38c058e0d0926b2daf120ab045944e3217bacad7
[WebKit-https.git] / PerformanceTests / SunSpider / ChangeLog
1 2013-01-16  Eric Seidel  <eric@webkit.org>
2
3         Remove --shark* support from sunspider/run-sunspider now that Shark is dead
4         https://bugs.webkit.org/show_bug.cgi?id=99512
5
6         Reviewed by Darin Adler.
7
8         I tried to use --instruments, but that also seems broken (in the same way before/after this patch).
9
10         * sunspider:
11         (runTestsOnce):
12
13 2012-08-02  Filip Pizlo  <fpizlo@apple.com>
14
15         Unreviewed, revert debugging stuff I landed in http://trac.webkit.org/changeset/124557
16         by accident.
17
18         * tests/v8-v6/v8-crypto.js:
19         (am1):
20         (am2):
21         (am3):
22         (am4):
23         (bnpCopyTo):
24         (bnpFromString):
25         (bnpClamp):
26
27 2012-03-11  Maciej Stachowiak  <mjs@apple.com>
28
29         Add --instruments option to sunspider to profile with Instruments
30         https://bugs.webkit.org/show_bug.cgi?id=80783
31
32         Reviewed by Filip Pizlo.
33
34         * resources/TimeProfile20us.tracetemplate: Added.
35         * sunspider:
36         (runTestsOnce):
37
38 2011-11-15  Andy Wingo  <wingo@igalia.com>
39
40         Extend sunspider driver to be able to run kraken
41         https://bugs.webkit.org/show_bug.cgi?id=71799
42
43         Reviewed by Filip Pizlo.
44
45         * resources/sunspider-standalone-driver.js: Try to load a -data
46         file.  If that succeeds, we have a kraken-like test, so we time
47         the test using `load'.  Otherwise fall back to using `run'.
48
49 2011-08-11  Mark Rowe  <mrowe@apple.com>
50
51         Fix sunspider-compare-results to default to the same suite version as the sunspider script.
52
53         Rubber-stamped by Maciej Stachowiak.
54
55         * sunspider-compare-results:
56
57 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
58
59         Update SunSpider versioning to 1.0
60         https://bugs.webkit.org/show_bug.cgi?id=63870
61
62         Reviewed by Dan Bernstein.
63
64         * hosted/sunspider.html:
65         * hosted/versions.html:
66         * resources/driver-TEMPLATE.html:
67         * resources/results-TEMPLATE.html:
68         * sunspider:
69
70 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
71
72         Sunspider 0.9.1 harness doesn't actually close() its test documents
73         https://bugs.webkit.org/show_bug.cgi?id=47045
74
75         Reviewed by Daniel Bates.
76
77         * hosted/versions.html:
78         * resources/driver-TEMPLATE.html:
79
80 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
81
82         Not Reviewed. 
83         
84         Fix obviou typo in previous commit.
85
86         * tests/sunspider-1.0/bitops-bitwise-and.js:
87
88 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
89
90         Reviewed by Anders Carlsson.
91
92         SunSpider: all four bitops benchmarks can be replaced with NOP
93         https://bugs.webkit.org/show_bug.cgi?id=38446
94
95         * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
96         (TimeFunc): Save the result.
97         * tests/sunspider-1.0/bitops-bits-in-byte.js:
98         (TimeFunc): Save the result.
99         * tests/sunspider-1.0/bitops-bitwise-and.js: More explicitly
100         save the results.
101         * tests/sunspider-1.0/bitops-nsieve-bits.js:
102         (sieve): Save the result.
103
104 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
105
106         SunSpider: string-validate-input.js uses an undeclared variable named 'name', which is a DOM API
107         https://bugs.webkit.org/show_bug.cgi?id=60937
108
109         Reviewed by Dan Bernstein.
110
111         * tests/sunspider-1.0/string-validate-input.js:
112         (doTest): Rename the variable to username
113
114 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
115
116         Reviewed by Darin Adler.
117
118         SunSpider: The main function of math-cordic is dead code
119         https://bugs.webkit.org/show_bug.cgi?id=63863
120         
121         Try to prevent math-cordic from being eliminated as dead code or
122         compiled down to a constant.
123
124         * tests/sunspider-1.0/math-cordic.js:
125         (cordicsincos):
126         (cordic):
127
128 2011-06-30  Maciej Stachowiak  <mjs@apple.com>
129
130         Reviewed by Adam Barth.
131
132         Create sunspider-1.0 directory in preparation for fixing a bunch of bugs
133         https://bugs.webkit.org/show_bug.cgi?id=63782
134
135         * make-hosted: Learn about the new directory.
136         * tests/sunspider-1.0: Copied from PerformanceTests/SunSpider/tests/sunspider-0.9.1.
137
138 2010-11-29  Geoffrey Garen  <ggaren@apple.com>
139
140         Reviewed by Gavin Barraclough.
141
142         Improved accuracy of command-line SunSpider.
143
144         * resources/sunspider-standalone-driver.js: Use the returned number of
145         elapsed milliseconds from "run" and "checkSyntax" instead of measuring
146         ourselves, for slightly more accurate numbers.
147
148 2010-09-16  Gavin Barraclough  <barraclough@apple.com>
149
150         Reviewed by Mark Rowe.
151         https://bugs.webkit.org/show_bug.cgi?id=45924
152
153         Modify the SunSpider harness to allow a path to be passed to the --suite
154         argument, supress the normal prepend-"tests/" behaviour for suite names
155         containing a '/'.
156
157         Also wrap the code in an anonymous function to move variables out of global
158         scope (currently the harness cannot handle running tests the overwrite the
159         variable 'j').
160
161         * resources/sunspider-standalone-driver.js:
162         * sunspider:
163
164 2010-08-27  Michael Saboff  <msaboff@apple.com>
165
166         Reviewed by Stephanie Lewis.
167
168         Added version 5 and version 6 of V8 tests with modifications similar
169         to the version 4 tests so they'll work with the SunSpider test harness.
170         Note that these files are external source and therefore don't conform 
171         to WebKit coding standards.
172         https://bugs.webkit.org/show_bug.cgi?id=44638
173
174         * tests/v8-v5: Added.
175         * tests/v8-v5/LIST: Added.
176         * tests/v8-v5/v8-crypto.js: Added.
177         * tests/v8-v5/v8-deltablue.js: Added.
178         * tests/v8-v5/v8-earley-boyer.js: Added.
179         * tests/v8-v5/v8-raytrace.js: Added.
180         * tests/v8-v5/v8-regexp.js: Added.
181         * tests/v8-v5/v8-richards.js: Added.
182         * tests/v8-v5/v8-splay.js: Added.
183         * tests/v8-v6: Added.
184         * tests/v8-v6/LIST: Added.
185         * tests/v8-v6/v8-crypto.js: Added.
186         * tests/v8-v6/v8-deltablue.js: Added.
187         * tests/v8-v6/v8-earley-boyer.js: Added.
188         * tests/v8-v6/v8-raytrace.js: Added.
189         * tests/v8-v6/v8-regexp.js: Added.
190         * tests/v8-v6/v8-richards.js: Added.
191         * tests/v8-v6/v8-splay.js: Added.
192
193 2010-07-15  Stephanie Lewis  <slewis@apple.com>
194
195         Reviewed by Geoff Garen.
196
197         http://bugs.webkit.org/show_bug.cgi?id=42406
198         Output location of results file.
199
200         * sunspider:
201
202 2010-05-17  Maciej Stachowiak  <mjs@apple.com>
203
204         Reviewed by Mark Rowe.
205
206         Browser-hosted version of SunSpider runs tests twice as many times as advertised
207         https://bugs.webkit.org/show_bug.cgi?id=39201
208
209         * make-hosted: Make sure to reset test list after every test suite.
210
211 2010-05-07  Adam Roben  <aroben@apple.com>
212
213         Fix typo in sunspider-compare-results.js that caused incorrect
214         two-sample t statistics to be computed
215
216         Fixes <http://webkit.org/b/38768> SunSpider computes incorrect
217         two-sample t statistics
218
219         Reviewed by Darin Adler.
220
221         * resources/sunspider-compare-results.js:
222         (sunspiderCompareResults.resultLine): Fixed the calculation of the t
223         variable to match the definition of a two-sample t statistic. Looks
224         like this was just a simple typo.
225
226 2010-04-05  Darin Adler  <darin@apple.com>
227
228         Updated the ignore property for changes to the contents here.
229
230         * .: Modified property svn:ignore.
231
232 2010-02-20  Maciej Stachowiak  <mjs@apple.com>
233
234         Reviewed by Adam Barth.
235
236         Add sunspider-0.9.1 to the site in preparation for announcing it.
237         https://bugs.webkit.org/show_bug.cgi?id=35206
238
239         * hosted/sunspider.css: Remove tabs, which I noticed where present while
240         copying to the WebKitSite directory.
241
242 2010-01-11  Oliver Hunt  <oliver@apple.com>
243
244         Reviewed by Darin Adler.
245
246         Sunspider parse-only tests don't run
247         https://bugs.webkit.org/show_bug.cgi?id=33489
248
249         Fix parse-only test list
250
251         * tests/parse-only/LIST:
252
253 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
254
255         Reviewed by Adele Peterson.
256
257         Make SunSpider version more prominent in the title
258         https://bugs.webkit.org/show_bug.cgi?id=32574
259
260         * hosted/sunspider.html:
261         * hosted/versions.html:
262         * resources/driver-TEMPLATE.html:
263         * resources/results-TEMPLATE.html:
264
265 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
266
267         Reviewed by Darin Adler.
268
269         Make SunSpider refuse to compare results across content versions
270         https://bugs.webkit.org/show_bug.cgi?id=32573
271         
272         The results URL now includes a version indicator, and compare mode
273         will refuse to compare between versions.
274
275         * resources/driver-TEMPLATE.html:
276         * resources/results-TEMPLATE.html:
277
278 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
279
280         Reviewed by Darin Adler.
281
282         Some Browser-hosted SunSpider files are not valid HTML5
283         https://bugs.webkit.org/show_bug.cgi?id=32536
284         
285         Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
286         close tags and adding a meta charset declaration).
287
288         * hosted/sunspider.html:
289         * hosted/versions.html:
290         * resources/TEMPLATE.html:
291         * resources/driver-TEMPLATE.html:
292         * resources/results-TEMPLATE.html:
293
294 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
295
296         Reviewed by Eric Seidel.
297
298         Make sunspider-0.9.1 the default content set (both command-line and hosted)
299         https://bugs.webkit.org/show_bug.cgi?id=32537
300
301         * hosted/sunspider.html: Change default suite link.
302         * sunspider: Change default suite.
303         * sunspider-compare-results: Likewise.
304
305 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
306
307         Reviewed by Sam Weinig.
308
309         in-browser SunSpider suffers excessive penalty under power management
310         https://bugs.webkit.org/show_bug.cgi?id=32505
311         
312         I have made a few changes to address this:
313         
314         1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
315         document.write to reduce triggering of progress UI.
316         2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
317         stepped all the way down the whole time.
318         3) Run the test cycle an extra time for warmup before the runs that count.
319
320         * hosted/sunspider-record-result.js: Removed.
321         * make-hosted:
322         * resources/TEMPLATE.html:
323         * resources/driver-TEMPLATE.html:
324
325 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
326
327         Reviewed by Gavin Barraclough.
328
329         SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
330         https://bugs.webkit.org/show_bug.cgi?id=16806
331
332         Based on a patch by Eric Seidel.
333         
334         Fix the base64 computation to actually compute correct results. The impact on runtime of
335         the test is pretty small, but noticeable for some browsers. But at least it's not
336         doing a wrong and meaningless computation any more.
337         
338         * tests/sunspider-0.9.1/string-base64.js:
339         ():
340         (base64ToString):
341
342 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
343
344         Fixing commit error...
345         
346         I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
347
348         * resources/driver-TEMPLATE.html:
349
350 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
351
352         Reviewed by Sam Weinig.
353
354         Use JSON.parse instead of eval for Web-hosted SunSpider results processing
355         https://bugs.webkit.org/show_bug.cgi?id=32490
356
357         * hosted/json2.js: Added. Incorporated from json.org
358         * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
359         * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
360         results.
361
362 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
363
364         Reviewed by Oliver Hunt.
365
366         Enable Web-hosted version of SunSpider to handle multiple versions
367         https://bugs.webkit.org/show_bug.cgi?id=32478
368
369         * make-hosted: Now generate the hosted version to be able to run both the
370         0.9 and the 0.9.1 test suites.
371         * hosted: Modified property svn:ignore.
372         * hosted/sunspider-driver.html: Removed.
373         * hosted/sunspider-results.html: Removed.
374         * hosted/sunspider.html:
375         * hosted/versions.html: Added.
376         * resources/TEMPLATE.html:
377         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
378         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
379
380 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
381
382         Reviewed by Oliver Hunt.
383
384         Give command-line sunspider the ability to handle multiple suites and versions
385         https://bugs.webkit.org/show_bug.cgi?id=32477
386         
387         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
388         enumerating the tests. Current suites are:
389         
390         sunspider-0.9 - riginal SunSpider content 
391         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
392         ubench - "ubench" microbenchmark suite
393         v8-v4 - version 4 of the v8 benchmark
394         parse-only - some parser tests
395         
396         Sunspider can now be run with the --suite= parameter to select a
397         suite. The default is --suite=sunspider-0.9.
398
399         * .: Modified property svn:ignore to ignore new results directories.
400         * make-hosted: Generate based on sunspider-0.9 suite.
401         * resources/TEMPLATE.html: Fix an HTML compliance problem.
402         * resources/sunspider-standalone-driver.js: Become aware of suite names.
403         * sunspider: Modified to handle multiple suites as described above.
404         * sunspider-compare-results: ditto
405         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
406         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
407         * tests/LIST-UBENCH: Removed.
408         * tests/LIST-V8: Removed.
409         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
410         * tests/3d-morph.js: ditto
411         * tests/3d-raytrace.js: ditto
412         * tests/access-binary-trees.js: ditto
413         * tests/access-fannkuch.js: ditto
414         * tests/access-nbody.js: ditto
415         * tests/access-nsieve.js: ditto
416         * tests/bitops-3bit-bits-in-byte.js: ditto
417         * tests/bitops-bits-in-byte.js: ditto
418         * tests/bitops-bitwise-and.js: ditto
419         * tests/bitops-nsieve-bits.js: ditto
420         * tests/controlflow-recursive.js: ditto
421         * tests/crypto-aes.js: ditto
422         * tests/crypto-md5.js: ditto
423         * tests/crypto-sha1.js: ditto
424         * tests/date-format-tofte.js: ditto
425         * tests/date-format-xparb.js: ditto
426         * tests/math-cordic.js: ditto
427         * tests/math-partial-sums.js: ditto
428         * tests/math-spectral-norm.js: ditto
429         * tests/regexp-dna.js: ditto
430         * tests/string-base64.js: ditto
431         * tests/string-fasta.js: ditto
432         * tests/string-tagcloud.js: ditto
433         * tests/string-unpack-code.js: ditto
434         * tests/string-validate-input.js: ditto
435         * tests/sunspider-0.9/: Added
436         * tests/sunspider-0.9/*.js: Moved from one level up
437         * tests/sunspider-0.9.1: Added.
438         * tests/sunspider-0.9.1/*.js: Moved from one level up.
439         * tests/v8-crypto.js: Moved to v8-v4/ directory.
440         * tests/v8-deltablue.js: ditto
441         * tests/v8-earley-boyer.js: ditto
442         * tests/v8-raytrace.js: ditto
443         * tests/v8-regexp.js: ditto
444         * tests/v8-richards.js: ditto
445         * tests/v8-splay.js: ditto
446         * tests/v8-v4: Added.
447
448 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
449
450         Reviewed by Darin Adler.
451
452         Platform specific null device replaced with a platform independent.
453         https://bugs.webkit.org/show_bug.cgi?id=29544
454
455         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
456
457 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
458
459         Reviewed by Maciej Stachowiak.
460
461         [Qt] Make sunspider script work on Windows platform.
462         https://bugs.webkit.org/show_bug.cgi?id=29544
463
464         * sunspider:
465         Using platform specific null device instead of /dev/null.
466
467 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
468
469         Reviewed by Oliver Hunt.
470
471         https://bugs.webkit.org/show_bug.cgi?id=26540
472         Currently the SunSpider test driver lacks an option to run a test suite that
473         will test JavaScriptCore parsing performance only.  This patch adds just such
474         a test suite and option to SunSpider as well as the jsc test shell.  I've included
475         three large javascript source files found in the wild: jquery, mootools and prototype.
476         Combined with the concatenation of all three, these form a new testsuite to measure
477         and test pure JavaScriptCore parsing performance.
478
479         * resources/sunspider-standalone-driver.js:
480           Load the js with parse-only flag if it is part of the parse-only suite.
481         * sunspider:
482           Add the parse-only suite to the list of options.
483         * sunspider-compare-results:
484           Add the parse-only suite to the list of options.
485         * tests/LIST-PARSE-ONLY: Added.
486         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
487           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
488         * tests/parse-only/jquery-1.3.2.js: Added.
489           Latest version from jquery.com.
490         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
491           Latest version from mootools.net.
492         * tests/parse-only/prototype-1.6.0.3.js: Added.
493           Latest version from prototypejs.org.
494
495 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
496
497         Update V8 benchmark to version 4.
498         
499         Reviewed by Cameron Zwarich.
500
501         * tests/LIST-V8:
502         * tests/v8-crypto.js:
503         * tests/v8-deltablue.js:
504         * tests/v8-earley-boyer.js:
505         * tests/v8-raytrace.js:
506         * tests/v8-regexp.js: Added.
507         * tests/v8-richards.js:
508         * tests/v8-splay.js: Added.
509
510 2009-05-26  Olivier DOLE  <odole@pleyo.com>
511
512         Reviewed by Maciej.
513
514         Store sunspider test suite results in a specific directory for each
515         different type of benchmark (sunspider, V8 or ubench).
516
517         * sunspider:
518         * sunspider-compare-results:
519
520 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
521
522         Build fix.
523
524         * SunSpider.make:
525
526 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
527
528         Include SunSpider tests in production builds.
529         
530         Reviewed by Adam Roben.
531
532         * SunSpider.make: Added.
533         * xcopy.excludes: Added.
534
535 2009-03-04  Oliver Hunt  <oliver@apple.com>
536
537         Added a few more tests to ubench
538
539         RS=Gavin Barraclough
540
541         Adding a couple of test to cover too few and too many arguments.  This
542         is needed to prevent regressions in all call paths from my current refactoring.
543
544         * tests/LIST-UBENCH:
545         * tests/ubench/function-correct-args.js: Added.
546         (f):
547         * tests/ubench/function-excess-args.js: Added.
548         (f):
549
550 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
551
552         Fix a bug where when V8's or UBENCH's test suite list file is missing
553         the error message is suggesting that SunSpider's list file is missing.
554
555         Reviewed by Sam Weinig.
556
557         * sunspider:
558         (loadTestsList): Fix the error message to use the right list name.
559
560 2008-10-07  David Hyatt  <hyatt@apple.com>
561
562         https://bugs.webkit.org/show_bug.cgi?id=18989
563
564         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
565         we want all browsers to be doing the same thing on the test.
566
567         Reviewed by Maciej
568
569         * tests/regexp-dna.js:
570
571 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
572
573         Rubber-stamped by Maciej Stachowiak.
574
575         The bleeding edge branch of V8 now supports passing fileanmes after
576         '-f', so the SunSpider script should be changed to do this again.
577         This allows all 3 major JavaScript engines to be used with the
578         unmodified SunSpider script.
579
580         * sunspider:
581
582 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
583
584         Reviewed by Maciej Stachowiak.
585
586         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
587         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
588         correct the punctuation in its documentation.
589
590         * sunspider:
591
592 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
593
594         Reviewed by Oliver Hunt.
595
596         Bug 20616: Incorporate V8 benchmarks in testing
597         <https://bugs.webkit.org/show_bug.cgi?id=20616>
598
599         This is a first attempt at running the V8 benchmarks within SunSpider,
600         with the --v8 option. The number of iterations is fixed for each
601         benchmark, and was chosen to make each of the benchmarks run between
602         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
603
604         * sunspider:
605         * tests/LIST-V8: Added.
606         * tests/v8-crypto.js: Added.
607         * tests/v8-deltablue.js: Added.
608         * tests/v8-earley-boyer.js: Added.
609         * tests/v8-raytrace.js: Added.
610         * tests/v8-richards.js: Added.
611
612 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
613
614         Not reviewed.
615
616         Remove a local change that was accidentally committed in r36034.
617
618         * resources/sunspider-standalone-driver.js:
619
620 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
621
622         Rubber-stamped by Oliver Hunt.
623
624         Make SunSpider work with JavaScript shells that don't take filenames
625         after a '-f' argument.
626
627         * sunspider:
628
629 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
630
631         Reviewed by Timothy.
632
633         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
634
635         * sunspider:
636         * tests/LIST-SQUIRRELFISH: Removed.
637
638 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
639
640         Rubber-stamped by Mark Rowe.
641
642         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
643
644         * sunspider:
645
646 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
647
648         Rubber stamped by Oliver.
649         
650         - add newly working tests to squirrelfish list
651
652         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
653
654 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
655
656         Rubber stamped by Oliver.
657         
658         - move ubench tests properly.
659
660         * tests/ubench: Added.
661         * tests/ubench/function-closure.js: Added.
662         * tests/ubench/function-empty.js: Added.
663         * tests/ubench/function-missing-args.js: Added.
664         * tests/ubench/function-sum.js: Added.
665         * tests/ubench/loop-empty-resolve.js: Added.
666         * tests/ubench/loop-empty.js: Added.
667         * tests/ubench/loop-sum.js: Added.
668
669 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
670
671         Rubber stamped by Oliver.
672
673         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
674
675 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
676
677         Reviewed by Oliver.
678
679         - make --squirrelfish mode use the real harness, but a reduced set of tests
680
681         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
682
683         * sunspider:
684         * tests/LIST-SQUIRRELFISH:
685         * tests/LIST-UBENCH: Added.
686         * tests/squirrelfish: Moved to tests/ubench
687
688 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
689
690         Reviewed by Maciej Stachowiak.
691         
692         Removed an unused test.
693
694         * tests/squirrelfish/loop-resolve.js: Removed.
695
696 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
697
698         Reviewed by Oliver.
699         
700         Add newly runnable tests to --squirrelfish mode.
701
702         * tests/LIST-SQUIRRELFISH:
703
704 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
705
706         Reviewed by Maciej Stachowiak.
707         
708         A little more love for --squirrelfish mode:
709         
710         Fixed a misplaced sort. Results properly sort now.
711         
712         Added a test list just for squirrelfish, and updated it to include
713         all currently passing SunSpider tests.
714         
715         Fixed the pruning regexp to match 3d-morph.js.
716
717         * sunspider:
718
719 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
720
721         Reviewed by Oliver Hunt.
722         
723         --squirrelfish mode: pared down tests for squirrelfish to chew on.
724
725         * sunspider:
726         * tests/LIST:
727         * tests/squirrelfish: Added.
728         * tests/squirrelfish/function-closure.js: Added.
729         * tests/squirrelfish/function-empty.js: Added.
730         * tests/squirrelfish/function-missing-args.js: Added.
731         * tests/squirrelfish/function-sum.js: Added.
732         * tests/squirrelfish/loop-empty-resolve.js: Added.
733         * tests/squirrelfish/loop-empty.js: Added.
734         * tests/squirrelfish/loop-resolve.js: Added.
735         * tests/squirrelfish/loop-sum.js: Added.
736
737 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
738
739         Reviewed by Geoff.
740
741         * sunspider: Pause update daemon on Darwin to make results reliable
742         again (workaround for <rdar://problem/5811127>).
743
744 2008-02-07  Eric Seidel  <eric@webkit.org>
745
746         Reviewed by Sam.
747
748         * sunspider: add an error message when ./tests/LIST is not found
749
750 2007-12-29  Alp Toker  <alp@atoker.com>
751
752         Fix typo.
753
754         * hosted/sunspider.html:
755
756 2007-12-29  Alp Toker  <alp@atoker.com>
757
758         Reviewed by Maciej.
759
760         Provide the current URL in the SunSpider results page. Useful for
761         copying benchmark results out of browsers without a location bar or
762         where the location bar can't handle long URLs.
763
764         * hosted/sunspider-results.html:
765
766 2007-12-20  Eric Seidel  <eric@webkit.org>
767
768         Reviewed by mjs.
769
770         * sunspider: increase default number of runs for more accurate numbers
771
772 2007-12-18  Mark Rowe  <mrowe@apple.com>
773
774         Fix typo.
775
776         * hosted/sunspider.html:
777
778 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
779
780         Reviewed by Eric.
781         
782         - give an extra digit of precision for small differences.
783
784         * resources/sunspider-compare-results.js:
785
786 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
787
788         Reviewed by Eric.
789         
790         - work around IE layout bugs
791
792         * hosted/sunspider.css:
793
794 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
795
796         Reviewed by Mark.
797         
798         - fix compare feature
799
800         * hosted/sunspider-results.html:
801
802 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
803
804         Reviewed by Mark.
805         
806         - Change the format of the output URL to be shorter, to meet IE constraints
807
808         * hosted/sunspider-driver.html:
809         * hosted/sunspider-results.html:
810
811 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
812
813         Reviewed by Oliver.
814         
815         - Add some explanatory test and a dumb logo to SunSpider web hosted version
816
817         * hosted/sunspider-driver.html:
818         * hosted/sunspider-results.html:
819         * hosted/sunspider.css:
820         * hosted/sunspider.html:
821
822 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
823
824         Reviewed by Oliver.
825         
826         - Make the web-hosted version of SunSpider look a little bit less drab.
827
828         * hosted/sunspider-driver.html:
829         * hosted/sunspider-results.html:
830         * hosted/sunspider.css: Added.
831         * hosted/sunspider.html:
832         * resources/TEMPLATE.html:
833
834 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
835
836         Reviewed by Sam.
837         
838         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
839         
840         A sample of what new-style results would look like:
841         
842         TEST                   COMPARISON            FROM                 TO             DETAILS
843         ...
844         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
845         ...
846             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
847         
848         * resources/sunspider-compare-results.js:
849
850 2007-11-14  Eric Seidel  <eric@webkit.org>
851
852         Reviewed by Sam.
853
854         * sunspider: add --shark-cache for L2 Cache Miss profiling
855
856 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
857
858         Reviewed by Darin Adler.
859
860         http://bugs.webkit.org/show_bug.cgi?id=15925
861         SunSpider should check for Shark being installed
862
863         * sunspider: Explicitly test that shark command line tool is installed.
864
865 2007-11-04  Darin Adler  <darin@apple.com>
866
867         * hosted/sunspider.html: Tweaked formatting a bit.
868
869 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
870
871         Reviewed by Darin Adler.
872
873         - Add compare mode and a description of the benchmark to the hosted version.
874
875         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
876         * hosted/sunspider.html: Added intro text.
877         * make-hosted: Copy new file.
878         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
879         a function.
880         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
881         expects.
882         * sunspider-compare-results: Use sunspider-standalone-compare.js.
883
884 2007-11-02  Darin Adler  <darin@apple.com>
885
886         * sunspider: Changed "--base" to "--set-baseline".
887
888 2007-11-02  Darin Adler  <darin@apple.com>
889
890         Reviewed by Maciej.
891
892         * sunspider: Added a "--base" option that records the current test as a baseline.
893         Moved the code to find the newest .mshark file into a function (since I needed to use
894         it in the other script and I prefer to copy/paste a function).
895
896         * sunspider-compare-results: Added logic to compare the baseline with the last
897         run when you don't pass any parameters.
898
899 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
900
901         Reviewed by Eric.
902         
903         - Added list of ECMAScript3 features that SunSpider does not cover currently.
904
905         * UNCOVERED: Added.
906
907 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
908
909         Reviewed by Oliver.
910         
911         - Give better error messages when sunspider-compare-results fails
912
913         * sunspider-compare-results:
914
915 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
916
917         Reviewed by Darin Adler.
918         
919         - fix Date tests to work in command-line SpiderMonkey
920
921         * tests/date-format-tofte.js:
922         * tests/date-format-xparb.js:
923
924 2007-10-29  Darin Adler  <darin@apple.com>
925
926         Reviewed by Maciej.
927
928         * sunspider: Added --shark20 option, to run Shark at its highest sample
929         resolution instead of its default.
930
931 2007-10-28  Adam Roben  <aroben@apple.com>
932
933         Fix SunSpider on Windows
934
935         Reviewed by Eric.
936
937         * sunspider: Don't use colons in filenames.
938
939 2007-10-28  Darin Adler  <darin@apple.com>
940
941         Reviewed by Adam.
942
943         * resources/sunspider-analyze-results.js: Tweak the output format a little.
944         Change so that when there's only one run we don't write out confidence
945         intervals at all rather than writing "NaN" over and over again.
946
947         * sunspider: Use the time and date as a suffix on the results file, that way
948         all the results are left behind in the tmp directory. This will make it easy
949         for us to add features that compare past results in the future. Also rename
950         the .mshark files using the same scheme.
951
952         * sunspider-compare-results: Relax the parsing rules so we can parse the
953         results file exactly as it's emitted from sunspider.
954
955 2007-10-25  Adam Roben  <aroben@apple.com>
956
957         Strip carriage returns from results
958
959         Carriage returns were screwing up sunspider-compare-results on
960         Windows.
961
962         Rubberstamped by Oliver.
963
964         * sunspider:
965
966 2007-10-24  Darin Adler  <darin@apple.com>
967
968         Reviewed by Eric.
969
970         * sunspider: Open the profile in shark after completing a run.
971         * sunspider-compare-results: Tiny perl tweaks.
972
973 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
974
975         Reviewed by Darin Adler.
976
977         - Don't hardcode my path to testkjs
978         
979         * sunspider-compare-results:
980
981 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
982
983         Reviewed by Darin Adler.
984         
985         - Add a compare script to compare results, and improve formatting of standard results a bit.
986
987         * resources/sunspider-analyze-results.js:
988         * resources/sunspider-compare-results.js: Added.
989         * sunspider-compare-results: Added.
990
991 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
992
993         Reviewed by Darin Adler.
994
995         * resources/sunspider-analyze-results.js: Do the error range
996         properly, using the t-distribution instead of 1.96 (which was
997         based on the normal distribution).
998         
999         * sunspider: Print results in a way that will be friendlier to the
1000         soon-to-come compare mode.
1001
1002 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
1003
1004         Reviewed by Review.
1005         
1006         - add more tests (probably the final set for now)
1007         
1008         Already balanced. Added date, regexp, control flow, and a few
1009         more string and object/array access tests.
1010         
1011         * tests/LIST:
1012         * tests/access-fannkuch.js: Added.
1013         * tests/access-nbody.js: Added.
1014         * tests/controlflow-recursive.js: Added.
1015         * tests/date-format-tofte.js: Added.
1016         * tests/date-format-xparb.js: Added.
1017         * tests/regexp-dna.js: Added.
1018         * tests/string-validate-input.js: Added.
1019
1020 2007-10-21  Eric Seidel  <eric@webkit.org>
1021
1022         Reviewed by mjs.
1023
1024         * sunspider: make --shark only sample from testkjs
1025
1026 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
1027
1028         Rubber stamped by Eric.
1029         
1030         - added crypto tests (already balanced)
1031
1032         * tests/LIST:
1033         * tests/crypto-aes.js: Added.
1034         * tests/crypto-md5.js: Added.
1035         * tests/crypto-sha1.js: Added.
1036
1037 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
1038
1039         Reviewed by Adam.
1040         
1041         - rebalance new tests
1042         
1043         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
1044         
1045         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1046
1047         This balances them with the existing tests.
1048
1049         * tests/access-binary-trees.js:
1050         * tests/access-nsieve.js:
1051         * tests/bitops-nsieve-bits.js:
1052         * tests/math-partial-sums.js:
1053         * tests/math-spectral-norm.js:
1054         * tests/string-fasta.js:
1055
1056 2007-10-20  Eric Seidel  <eric@webkit.org>
1057
1058         Reviewed by darin.
1059         
1060         Add improved argument handling to sunspider, including
1061         --runs=<number>, --tests=<pattern, --shark, and --help
1062         Also re-factor code into subroutines
1063
1064         * sunspider: improved argument handling
1065
1066 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1067
1068         Reviewed by Mark.
1069
1070         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
1071         
1072         * TODO:
1073         * tests/LIST:
1074         * tests/access-binary-trees.js: Added.
1075         * tests/access-nsieve.js: Added.
1076         * tests/bitops-nsieve-bits.js: Added.
1077         * tests/math-partial-sums.js: Added.
1078         * tests/math-spectral-norm.js: Added.
1079         * tests/string-fasta.js: Added.
1080
1081 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1082
1083         Reviewed by Darin Adler.
1084
1085         * TODO: Updated to note areas that are now well-covered.
1086         
1087         - Change the tests and the driver to avoid leaving large object
1088         graphs hanging around, since that throws off the subsequent tests.
1089         
1090         * resources/sunspider-standalone-driver.js:
1091         * tests/3d-cube.js:
1092         * tests/3d-morph.js:
1093         * tests/3d-raytrace.js:
1094         * tests/bitops-bitwise-and.js:
1095         * tests/string-base64.js:
1096         * tests/string-tagcloud.js:
1097
1098 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1099
1100         Reviewed by Nikolas.
1101         
1102         - Rebalance test complexity.
1103         
1104         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
1105         
1106         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1107
1108         However, for some tests, IE7 was a huge outlier compared to other
1109         major browsers. For those tests, I used Firefox 2.0.0.8 on the
1110         same OS and hardware as a normalization basis.
1111         
1112         * tests/3d-morph.js:
1113         * tests/3d-raytrace.js:
1114         * tests/bitops-3bit-bits-in-byte.js:
1115         * tests/bitops-bits-in-byte.js:
1116         * tests/bitops-bitwise-and.js:
1117         * tests/math-cordic.js:
1118         * tests/string-base64.js:
1119         * tests/string-tagcloud.js:
1120         * tests/string-unpack-code.js:
1121
1122 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1123
1124         Reviewed by Oliver.
1125         
1126         - Fix the browser-hosted driver to be compatible with Safari 2
1127
1128         * hosted/sunspider-driver.html:
1129
1130 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1131
1132         Rubber stamped by Hyatt.
1133         
1134         - Add BSD license notice to cordic test, the original author said to do so.
1135
1136         * tests/math-cordic.js:
1137
1138 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1139
1140         Reviewed by Eric.
1141         
1142         - Support Windows and Windows browsers.
1143
1144         * hosted/sunspider-results.html: Display results correctly in IE.
1145         * sunspider: tweaks to work on Windows, and with spaces in the path.
1146
1147 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1148
1149         Rubber stamped by Oliver.
1150
1151         * tests/string-tagcloud.js: Remove stray print()
1152
1153 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
1154
1155         Reviewed by Oliver.
1156         
1157         Add some string test cases.
1158
1159         * tests/string-base64.js: Added. Does base64 encoding and decoding.
1160         * tests/string-tagcloud.js: Added. Parses a JSON string and
1161         generates tagcloud HTML markup based on the data.        
1162         * tests/string-unpack-code.js: Added. Unpack compressed versions of
1163         four of the most popular JavaScript libraries.
1164         * tests/LIST: Added the new tests.
1165
1166 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
1167
1168         Reviewed by Anders.
1169
1170         * TODO: Updated with info on how to do the stats properly.
1171
1172         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
1173         avoid overly slowing down the benchmark.
1174
1175         Added some 3D-related test cases.
1176         
1177         * tests/3d-cube.js: Added.
1178         * tests/3d-morph.js: Added.
1179         * tests/3d-raytrace.js: Added.
1180         * tests/LIST:
1181
1182 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
1183
1184         Reviewed by Darin Adler.
1185
1186         - New JavaScript benchmark
1187         http://bugs.webkit.org/show_bug.cgi?id=15515
1188         
1189         A start on a new JavaScript benchmark, with standalone and
1190         browser-hosted drivers.
1191         
1192         The test content is still incomplete and the drivers could use
1193         some more features but there is enough here to make a good start.
1194         
1195         The drivers compute a 95% confidence interval on the mean for the
1196         whole test, each category, and each individual test to make it
1197         easier to tell whether differences are statistically
1198         significant. The confidence interval can be narrowed by running
1199         the test more times.
1200         
1201         * TODO: Added. Remaining things that need to be done.
1202         * sunspider: Added. Perl script that acts as the standalone test driver.
1203         * resources/sunspider-standalone-driver.js: Added. JavaScript part
1204         of standalone test driver.
1205         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
1206         analysis code.
1207         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
1208         * make-hosted: Added. Script to generate browser-hosted tests.
1209         * hosted/sunspider.html: Added. Start page for browser-hosted test.
1210         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
1211         * hosted/sunspider-results.html: Added. Results page for browser-hosted
1212         tests.
1213         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
1214         tests.
1215         * tests/LIST: Added. List of tests to use.
1216         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
1217         * tests/bitops-bits-in-byte.js: Added. Ditto.
1218         * tests/bitops-bitwise-and.js: Added. Ditto.
1219         * tests/math-cordic.js: Added. Ditto.
1220