2011-07-02 Maciej Stachowiak <mjs@apple.com>
[WebKit-https.git] / PerformanceTests / SunSpider / ChangeLog
1 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
2
3         Reviewed by Darin Adler.
4
5         SunSpider: The main function of math-cordic is dead code
6         https://bugs.webkit.org/show_bug.cgi?id=63863
7         
8         Try to prevent math-cordic from being eliminated as dead code or
9         compiled down to a constant.
10
11         * tests/sunspider-1.0/math-cordic.js:
12         (cordicsincos):
13         (cordic):
14
15 2011-06-30  Maciej Stachowiak  <mjs@apple.com>
16
17         Reviewed by Adam Barth.
18
19         Create sunspider-1.0 directory in preparation for fixing a bunch of bugs
20         https://bugs.webkit.org/show_bug.cgi?id=63782
21
22         * make-hosted: Learn about the new directory.
23         * tests/sunspider-1.0: Copied from PerformanceTests/SunSpider/tests/sunspider-0.9.1.
24
25 2010-11-29  Geoffrey Garen  <ggaren@apple.com>
26
27         Reviewed by Gavin Barraclough.
28
29         Improved accuracy of command-line SunSpider.
30
31         * resources/sunspider-standalone-driver.js: Use the returned number of
32         elapsed milliseconds from "run" and "checkSyntax" instead of measuring
33         ourselves, for slightly more accurate numbers.
34
35 2010-09-16  Gavin Barraclough  <barraclough@apple.com>
36
37         Reviewed by Mark Rowe.
38         https://bugs.webkit.org/show_bug.cgi?id=45924
39
40         Modify the SunSpider harness to allow a path to be passed to the --suite
41         argument, supress the normal prepend-"tests/" behaviour for suite names
42         containing a '/'.
43
44         Also wrap the code in an anonymous function to move variables out of global
45         scope (currently the harness cannot handle running tests the overwrite the
46         variable 'j').
47
48         * resources/sunspider-standalone-driver.js:
49         * sunspider:
50
51 2010-08-27  Michael Saboff  <msaboff@apple.com>
52
53         Reviewed by Stephanie Lewis.
54
55         Added version 5 and version 6 of V8 tests with modifications similar
56         to the version 4 tests so they'll work with the SunSpider test harness.
57         Note that these files are external source and therefore don't conform 
58         to WebKit coding standards.
59         https://bugs.webkit.org/show_bug.cgi?id=44638
60
61         * tests/v8-v5: Added.
62         * tests/v8-v5/LIST: Added.
63         * tests/v8-v5/v8-crypto.js: Added.
64         * tests/v8-v5/v8-deltablue.js: Added.
65         * tests/v8-v5/v8-earley-boyer.js: Added.
66         * tests/v8-v5/v8-raytrace.js: Added.
67         * tests/v8-v5/v8-regexp.js: Added.
68         * tests/v8-v5/v8-richards.js: Added.
69         * tests/v8-v5/v8-splay.js: Added.
70         * tests/v8-v6: Added.
71         * tests/v8-v6/LIST: Added.
72         * tests/v8-v6/v8-crypto.js: Added.
73         * tests/v8-v6/v8-deltablue.js: Added.
74         * tests/v8-v6/v8-earley-boyer.js: Added.
75         * tests/v8-v6/v8-raytrace.js: Added.
76         * tests/v8-v6/v8-regexp.js: Added.
77         * tests/v8-v6/v8-richards.js: Added.
78         * tests/v8-v6/v8-splay.js: Added.
79
80 2010-07-15  Stephanie Lewis  <slewis@apple.com>
81
82         Reviewed by Geoff Garen.
83
84         http://bugs.webkit.org/show_bug.cgi?id=42406
85         Output location of results file.
86
87         * sunspider:
88
89 2010-05-17  Maciej Stachowiak  <mjs@apple.com>
90
91         Reviewed by Mark Rowe.
92
93         Browser-hosted version of SunSpider runs tests twice as many times as advertised
94         https://bugs.webkit.org/show_bug.cgi?id=39201
95
96         * make-hosted: Make sure to reset test list after every test suite.
97
98 2010-05-07  Adam Roben  <aroben@apple.com>
99
100         Fix typo in sunspider-compare-results.js that caused incorrect
101         two-sample t statistics to be computed
102
103         Fixes <http://webkit.org/b/38768> SunSpider computes incorrect
104         two-sample t statistics
105
106         Reviewed by Darin Adler.
107
108         * resources/sunspider-compare-results.js:
109         (sunspiderCompareResults.resultLine): Fixed the calculation of the t
110         variable to match the definition of a two-sample t statistic. Looks
111         like this was just a simple typo.
112
113 2010-04-05  Darin Adler  <darin@apple.com>
114
115         Updated the ignore property for changes to the contents here.
116
117         * .: Modified property svn:ignore.
118
119 2010-02-20  Maciej Stachowiak  <mjs@apple.com>
120
121         Reviewed by Adam Barth.
122
123         Add sunspider-0.9.1 to the site in preparation for announcing it.
124         https://bugs.webkit.org/show_bug.cgi?id=35206
125
126         * hosted/sunspider.css: Remove tabs, which I noticed where present while
127         copying to the WebKitSite directory.
128
129 2010-01-11  Oliver Hunt  <oliver@apple.com>
130
131         Reviewed by Darin Adler.
132
133         Sunspider parse-only tests don't run
134         https://bugs.webkit.org/show_bug.cgi?id=33489
135
136         Fix parse-only test list
137
138         * tests/parse-only/LIST:
139
140 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
141
142         Reviewed by Adele Peterson.
143
144         Make SunSpider version more prominent in the title
145         https://bugs.webkit.org/show_bug.cgi?id=32574
146
147         * hosted/sunspider.html:
148         * hosted/versions.html:
149         * resources/driver-TEMPLATE.html:
150         * resources/results-TEMPLATE.html:
151
152 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
153
154         Reviewed by Darin Adler.
155
156         Make SunSpider refuse to compare results across content versions
157         https://bugs.webkit.org/show_bug.cgi?id=32573
158         
159         The results URL now includes a version indicator, and compare mode
160         will refuse to compare between versions.
161
162         * resources/driver-TEMPLATE.html:
163         * resources/results-TEMPLATE.html:
164
165 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
166
167         Reviewed by Darin Adler.
168
169         Some Browser-hosted SunSpider files are not valid HTML5
170         https://bugs.webkit.org/show_bug.cgi?id=32536
171         
172         Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
173         close tags and adding a meta charset declaration).
174
175         * hosted/sunspider.html:
176         * hosted/versions.html:
177         * resources/TEMPLATE.html:
178         * resources/driver-TEMPLATE.html:
179         * resources/results-TEMPLATE.html:
180
181 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
182
183         Reviewed by Eric Seidel.
184
185         Make sunspider-0.9.1 the default content set (both command-line and hosted)
186         https://bugs.webkit.org/show_bug.cgi?id=32537
187
188         * hosted/sunspider.html: Change default suite link.
189         * sunspider: Change default suite.
190         * sunspider-compare-results: Likewise.
191
192 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
193
194         Reviewed by Sam Weinig.
195
196         in-browser SunSpider suffers excessive penalty under power management
197         https://bugs.webkit.org/show_bug.cgi?id=32505
198         
199         I have made a few changes to address this:
200         
201         1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
202         document.write to reduce triggering of progress UI.
203         2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
204         stepped all the way down the whole time.
205         3) Run the test cycle an extra time for warmup before the runs that count.
206
207         * hosted/sunspider-record-result.js: Removed.
208         * make-hosted:
209         * resources/TEMPLATE.html:
210         * resources/driver-TEMPLATE.html:
211
212 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
213
214         Reviewed by Gavin Barraclough.
215
216         SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
217         https://bugs.webkit.org/show_bug.cgi?id=16806
218
219         Based on a patch by Eric Seidel.
220         
221         Fix the base64 computation to actually compute correct results. The impact on runtime of
222         the test is pretty small, but noticeable for some browsers. But at least it's not
223         doing a wrong and meaningless computation any more.
224         
225         * tests/sunspider-0.9.1/string-base64.js:
226         ():
227         (base64ToString):
228
229 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
230
231         Fixing commit error...
232         
233         I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
234
235         * resources/driver-TEMPLATE.html:
236
237 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
238
239         Reviewed by Sam Weinig.
240
241         Use JSON.parse instead of eval for Web-hosted SunSpider results processing
242         https://bugs.webkit.org/show_bug.cgi?id=32490
243
244         * hosted/json2.js: Added. Incorporated from json.org
245         * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
246         * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
247         results.
248
249 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
250
251         Reviewed by Oliver Hunt.
252
253         Enable Web-hosted version of SunSpider to handle multiple versions
254         https://bugs.webkit.org/show_bug.cgi?id=32478
255
256         * make-hosted: Now generate the hosted version to be able to run both the
257         0.9 and the 0.9.1 test suites.
258         * hosted: Modified property svn:ignore.
259         * hosted/sunspider-driver.html: Removed.
260         * hosted/sunspider-results.html: Removed.
261         * hosted/sunspider.html:
262         * hosted/versions.html: Added.
263         * resources/TEMPLATE.html:
264         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
265         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
266
267 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
268
269         Reviewed by Oliver Hunt.
270
271         Give command-line sunspider the ability to handle multiple suites and versions
272         https://bugs.webkit.org/show_bug.cgi?id=32477
273         
274         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
275         enumerating the tests. Current suites are:
276         
277         sunspider-0.9 - riginal SunSpider content 
278         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
279         ubench - "ubench" microbenchmark suite
280         v8-v4 - version 4 of the v8 benchmark
281         parse-only - some parser tests
282         
283         Sunspider can now be run with the --suite= parameter to select a
284         suite. The default is --suite=sunspider-0.9.
285
286         * .: Modified property svn:ignore to ignore new results directories.
287         * make-hosted: Generate based on sunspider-0.9 suite.
288         * resources/TEMPLATE.html: Fix an HTML compliance problem.
289         * resources/sunspider-standalone-driver.js: Become aware of suite names.
290         * sunspider: Modified to handle multiple suites as described above.
291         * sunspider-compare-results: ditto
292         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
293         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
294         * tests/LIST-UBENCH: Removed.
295         * tests/LIST-V8: Removed.
296         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
297         * tests/3d-morph.js: ditto
298         * tests/3d-raytrace.js: ditto
299         * tests/access-binary-trees.js: ditto
300         * tests/access-fannkuch.js: ditto
301         * tests/access-nbody.js: ditto
302         * tests/access-nsieve.js: ditto
303         * tests/bitops-3bit-bits-in-byte.js: ditto
304         * tests/bitops-bits-in-byte.js: ditto
305         * tests/bitops-bitwise-and.js: ditto
306         * tests/bitops-nsieve-bits.js: ditto
307         * tests/controlflow-recursive.js: ditto
308         * tests/crypto-aes.js: ditto
309         * tests/crypto-md5.js: ditto
310         * tests/crypto-sha1.js: ditto
311         * tests/date-format-tofte.js: ditto
312         * tests/date-format-xparb.js: ditto
313         * tests/math-cordic.js: ditto
314         * tests/math-partial-sums.js: ditto
315         * tests/math-spectral-norm.js: ditto
316         * tests/regexp-dna.js: ditto
317         * tests/string-base64.js: ditto
318         * tests/string-fasta.js: ditto
319         * tests/string-tagcloud.js: ditto
320         * tests/string-unpack-code.js: ditto
321         * tests/string-validate-input.js: ditto
322         * tests/sunspider-0.9/: Added
323         * tests/sunspider-0.9/*.js: Moved from one level up
324         * tests/sunspider-0.9.1: Added.
325         * tests/sunspider-0.9.1/*.js: Moved from one level up.
326         * tests/v8-crypto.js: Moved to v8-v4/ directory.
327         * tests/v8-deltablue.js: ditto
328         * tests/v8-earley-boyer.js: ditto
329         * tests/v8-raytrace.js: ditto
330         * tests/v8-regexp.js: ditto
331         * tests/v8-richards.js: ditto
332         * tests/v8-splay.js: ditto
333         * tests/v8-v4: Added.
334
335 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
336
337         Reviewed by Darin Adler.
338
339         Platform specific null device replaced with a platform independent.
340         https://bugs.webkit.org/show_bug.cgi?id=29544
341
342         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
343
344 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
345
346         Reviewed by Maciej Stachowiak.
347
348         [Qt] Make sunspider script work on Windows platform.
349         https://bugs.webkit.org/show_bug.cgi?id=29544
350
351         * sunspider:
352         Using platform specific null device instead of /dev/null.
353
354 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
355
356         Reviewed by Oliver Hunt.
357
358         https://bugs.webkit.org/show_bug.cgi?id=26540
359         Currently the SunSpider test driver lacks an option to run a test suite that
360         will test JavaScriptCore parsing performance only.  This patch adds just such
361         a test suite and option to SunSpider as well as the jsc test shell.  I've included
362         three large javascript source files found in the wild: jquery, mootools and prototype.
363         Combined with the concatenation of all three, these form a new testsuite to measure
364         and test pure JavaScriptCore parsing performance.
365
366         * resources/sunspider-standalone-driver.js:
367           Load the js with parse-only flag if it is part of the parse-only suite.
368         * sunspider:
369           Add the parse-only suite to the list of options.
370         * sunspider-compare-results:
371           Add the parse-only suite to the list of options.
372         * tests/LIST-PARSE-ONLY: Added.
373         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
374           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
375         * tests/parse-only/jquery-1.3.2.js: Added.
376           Latest version from jquery.com.
377         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
378           Latest version from mootools.net.
379         * tests/parse-only/prototype-1.6.0.3.js: Added.
380           Latest version from prototypejs.org.
381
382 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
383
384         Update V8 benchmark to version 4.
385         
386         Reviewed by Cameron Zwarich.
387
388         * tests/LIST-V8:
389         * tests/v8-crypto.js:
390         * tests/v8-deltablue.js:
391         * tests/v8-earley-boyer.js:
392         * tests/v8-raytrace.js:
393         * tests/v8-regexp.js: Added.
394         * tests/v8-richards.js:
395         * tests/v8-splay.js: Added.
396
397 2009-05-26  Olivier DOLE  <odole@pleyo.com>
398
399         Reviewed by Maciej.
400
401         Store sunspider test suite results in a specific directory for each
402         different type of benchmark (sunspider, V8 or ubench).
403
404         * sunspider:
405         * sunspider-compare-results:
406
407 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
408
409         Build fix.
410
411         * SunSpider.make:
412
413 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
414
415         Include SunSpider tests in production builds.
416         
417         Reviewed by Adam Roben.
418
419         * SunSpider.make: Added.
420         * xcopy.excludes: Added.
421
422 2009-03-04  Oliver Hunt  <oliver@apple.com>
423
424         Added a few more tests to ubench
425
426         RS=Gavin Barraclough
427
428         Adding a couple of test to cover too few and too many arguments.  This
429         is needed to prevent regressions in all call paths from my current refactoring.
430
431         * tests/LIST-UBENCH:
432         * tests/ubench/function-correct-args.js: Added.
433         (f):
434         * tests/ubench/function-excess-args.js: Added.
435         (f):
436
437 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
438
439         Fix a bug where when V8's or UBENCH's test suite list file is missing
440         the error message is suggesting that SunSpider's list file is missing.
441
442         Reviewed by Sam Weinig.
443
444         * sunspider:
445         (loadTestsList): Fix the error message to use the right list name.
446
447 2008-10-07  David Hyatt  <hyatt@apple.com>
448
449         https://bugs.webkit.org/show_bug.cgi?id=18989
450
451         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
452         we want all browsers to be doing the same thing on the test.
453
454         Reviewed by Maciej
455
456         * tests/regexp-dna.js:
457
458 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
459
460         Rubber-stamped by Maciej Stachowiak.
461
462         The bleeding edge branch of V8 now supports passing fileanmes after
463         '-f', so the SunSpider script should be changed to do this again.
464         This allows all 3 major JavaScript engines to be used with the
465         unmodified SunSpider script.
466
467         * sunspider:
468
469 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
470
471         Reviewed by Maciej Stachowiak.
472
473         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
474         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
475         correct the punctuation in its documentation.
476
477         * sunspider:
478
479 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
480
481         Reviewed by Oliver Hunt.
482
483         Bug 20616: Incorporate V8 benchmarks in testing
484         <https://bugs.webkit.org/show_bug.cgi?id=20616>
485
486         This is a first attempt at running the V8 benchmarks within SunSpider,
487         with the --v8 option. The number of iterations is fixed for each
488         benchmark, and was chosen to make each of the benchmarks run between
489         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
490
491         * sunspider:
492         * tests/LIST-V8: Added.
493         * tests/v8-crypto.js: Added.
494         * tests/v8-deltablue.js: Added.
495         * tests/v8-earley-boyer.js: Added.
496         * tests/v8-raytrace.js: Added.
497         * tests/v8-richards.js: Added.
498
499 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
500
501         Not reviewed.
502
503         Remove a local change that was accidentally committed in r36034.
504
505         * resources/sunspider-standalone-driver.js:
506
507 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
508
509         Rubber-stamped by Oliver Hunt.
510
511         Make SunSpider work with JavaScript shells that don't take filenames
512         after a '-f' argument.
513
514         * sunspider:
515
516 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
517
518         Reviewed by Timothy.
519
520         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
521
522         * sunspider:
523         * tests/LIST-SQUIRRELFISH: Removed.
524
525 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
526
527         Rubber-stamped by Mark Rowe.
528
529         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
530
531         * sunspider:
532
533 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
534
535         Rubber stamped by Oliver.
536         
537         - add newly working tests to squirrelfish list
538
539         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
540
541 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
542
543         Rubber stamped by Oliver.
544         
545         - move ubench tests properly.
546
547         * tests/ubench: Added.
548         * tests/ubench/function-closure.js: Added.
549         * tests/ubench/function-empty.js: Added.
550         * tests/ubench/function-missing-args.js: Added.
551         * tests/ubench/function-sum.js: Added.
552         * tests/ubench/loop-empty-resolve.js: Added.
553         * tests/ubench/loop-empty.js: Added.
554         * tests/ubench/loop-sum.js: Added.
555
556 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
557
558         Rubber stamped by Oliver.
559
560         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
561
562 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
563
564         Reviewed by Oliver.
565
566         - make --squirrelfish mode use the real harness, but a reduced set of tests
567
568         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
569
570         * sunspider:
571         * tests/LIST-SQUIRRELFISH:
572         * tests/LIST-UBENCH: Added.
573         * tests/squirrelfish: Moved to tests/ubench
574
575 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
576
577         Reviewed by Maciej Stachowiak.
578         
579         Removed an unused test.
580
581         * tests/squirrelfish/loop-resolve.js: Removed.
582
583 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
584
585         Reviewed by Oliver.
586         
587         Add newly runnable tests to --squirrelfish mode.
588
589         * tests/LIST-SQUIRRELFISH:
590
591 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
592
593         Reviewed by Maciej Stachowiak.
594         
595         A little more love for --squirrelfish mode:
596         
597         Fixed a misplaced sort. Results properly sort now.
598         
599         Added a test list just for squirrelfish, and updated it to include
600         all currently passing SunSpider tests.
601         
602         Fixed the pruning regexp to match 3d-morph.js.
603
604         * sunspider:
605
606 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
607
608         Reviewed by Oliver Hunt.
609         
610         --squirrelfish mode: pared down tests for squirrelfish to chew on.
611
612         * sunspider:
613         * tests/LIST:
614         * tests/squirrelfish: Added.
615         * tests/squirrelfish/function-closure.js: Added.
616         * tests/squirrelfish/function-empty.js: Added.
617         * tests/squirrelfish/function-missing-args.js: Added.
618         * tests/squirrelfish/function-sum.js: Added.
619         * tests/squirrelfish/loop-empty-resolve.js: Added.
620         * tests/squirrelfish/loop-empty.js: Added.
621         * tests/squirrelfish/loop-resolve.js: Added.
622         * tests/squirrelfish/loop-sum.js: Added.
623
624 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
625
626         Reviewed by Geoff.
627
628         * sunspider: Pause update daemon on Darwin to make results reliable
629         again (workaround for <rdar://problem/5811127>).
630
631 2008-02-07  Eric Seidel  <eric@webkit.org>
632
633         Reviewed by Sam.
634
635         * sunspider: add an error message when ./tests/LIST is not found
636
637 2007-12-29  Alp Toker  <alp@atoker.com>
638
639         Fix typo.
640
641         * hosted/sunspider.html:
642
643 2007-12-29  Alp Toker  <alp@atoker.com>
644
645         Reviewed by Maciej.
646
647         Provide the current URL in the SunSpider results page. Useful for
648         copying benchmark results out of browsers without a location bar or
649         where the location bar can't handle long URLs.
650
651         * hosted/sunspider-results.html:
652
653 2007-12-20  Eric Seidel  <eric@webkit.org>
654
655         Reviewed by mjs.
656
657         * sunspider: increase default number of runs for more accurate numbers
658
659 2007-12-18  Mark Rowe  <mrowe@apple.com>
660
661         Fix typo.
662
663         * hosted/sunspider.html:
664
665 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
666
667         Reviewed by Eric.
668         
669         - give an extra digit of precision for small differences.
670
671         * resources/sunspider-compare-results.js:
672
673 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
674
675         Reviewed by Eric.
676         
677         - work around IE layout bugs
678
679         * hosted/sunspider.css:
680
681 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
682
683         Reviewed by Mark.
684         
685         - fix compare feature
686
687         * hosted/sunspider-results.html:
688
689 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
690
691         Reviewed by Mark.
692         
693         - Change the format of the output URL to be shorter, to meet IE constraints
694
695         * hosted/sunspider-driver.html:
696         * hosted/sunspider-results.html:
697
698 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
699
700         Reviewed by Oliver.
701         
702         - Add some explanatory test and a dumb logo to SunSpider web hosted version
703
704         * hosted/sunspider-driver.html:
705         * hosted/sunspider-results.html:
706         * hosted/sunspider.css:
707         * hosted/sunspider.html:
708
709 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
710
711         Reviewed by Oliver.
712         
713         - Make the web-hosted version of SunSpider look a little bit less drab.
714
715         * hosted/sunspider-driver.html:
716         * hosted/sunspider-results.html:
717         * hosted/sunspider.css: Added.
718         * hosted/sunspider.html:
719         * resources/TEMPLATE.html:
720
721 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
722
723         Reviewed by Sam.
724         
725         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
726         
727         A sample of what new-style results would look like:
728         
729         TEST                   COMPARISON            FROM                 TO             DETAILS
730         ...
731         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
732         ...
733             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
734         
735         * resources/sunspider-compare-results.js:
736
737 2007-11-14  Eric Seidel  <eric@webkit.org>
738
739         Reviewed by Sam.
740
741         * sunspider: add --shark-cache for L2 Cache Miss profiling
742
743 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
744
745         Reviewed by Darin Adler.
746
747         http://bugs.webkit.org/show_bug.cgi?id=15925
748         SunSpider should check for Shark being installed
749
750         * sunspider: Explicitly test that shark command line tool is installed.
751
752 2007-11-04  Darin Adler  <darin@apple.com>
753
754         * hosted/sunspider.html: Tweaked formatting a bit.
755
756 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
757
758         Reviewed by Darin Adler.
759
760         - Add compare mode and a description of the benchmark to the hosted version.
761
762         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
763         * hosted/sunspider.html: Added intro text.
764         * make-hosted: Copy new file.
765         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
766         a function.
767         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
768         expects.
769         * sunspider-compare-results: Use sunspider-standalone-compare.js.
770
771 2007-11-02  Darin Adler  <darin@apple.com>
772
773         * sunspider: Changed "--base" to "--set-baseline".
774
775 2007-11-02  Darin Adler  <darin@apple.com>
776
777         Reviewed by Maciej.
778
779         * sunspider: Added a "--base" option that records the current test as a baseline.
780         Moved the code to find the newest .mshark file into a function (since I needed to use
781         it in the other script and I prefer to copy/paste a function).
782
783         * sunspider-compare-results: Added logic to compare the baseline with the last
784         run when you don't pass any parameters.
785
786 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
787
788         Reviewed by Eric.
789         
790         - Added list of ECMAScript3 features that SunSpider does not cover currently.
791
792         * UNCOVERED: Added.
793
794 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
795
796         Reviewed by Oliver.
797         
798         - Give better error messages when sunspider-compare-results fails
799
800         * sunspider-compare-results:
801
802 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
803
804         Reviewed by Darin Adler.
805         
806         - fix Date tests to work in command-line SpiderMonkey
807
808         * tests/date-format-tofte.js:
809         * tests/date-format-xparb.js:
810
811 2007-10-29  Darin Adler  <darin@apple.com>
812
813         Reviewed by Maciej.
814
815         * sunspider: Added --shark20 option, to run Shark at its highest sample
816         resolution instead of its default.
817
818 2007-10-28  Adam Roben  <aroben@apple.com>
819
820         Fix SunSpider on Windows
821
822         Reviewed by Eric.
823
824         * sunspider: Don't use colons in filenames.
825
826 2007-10-28  Darin Adler  <darin@apple.com>
827
828         Reviewed by Adam.
829
830         * resources/sunspider-analyze-results.js: Tweak the output format a little.
831         Change so that when there's only one run we don't write out confidence
832         intervals at all rather than writing "NaN" over and over again.
833
834         * sunspider: Use the time and date as a suffix on the results file, that way
835         all the results are left behind in the tmp directory. This will make it easy
836         for us to add features that compare past results in the future. Also rename
837         the .mshark files using the same scheme.
838
839         * sunspider-compare-results: Relax the parsing rules so we can parse the
840         results file exactly as it's emitted from sunspider.
841
842 2007-10-25  Adam Roben  <aroben@apple.com>
843
844         Strip carriage returns from results
845
846         Carriage returns were screwing up sunspider-compare-results on
847         Windows.
848
849         Rubberstamped by Oliver.
850
851         * sunspider:
852
853 2007-10-24  Darin Adler  <darin@apple.com>
854
855         Reviewed by Eric.
856
857         * sunspider: Open the profile in shark after completing a run.
858         * sunspider-compare-results: Tiny perl tweaks.
859
860 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
861
862         Reviewed by Darin Adler.
863
864         - Don't hardcode my path to testkjs
865         
866         * sunspider-compare-results:
867
868 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
869
870         Reviewed by Darin Adler.
871         
872         - Add a compare script to compare results, and improve formatting of standard results a bit.
873
874         * resources/sunspider-analyze-results.js:
875         * resources/sunspider-compare-results.js: Added.
876         * sunspider-compare-results: Added.
877
878 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
879
880         Reviewed by Darin Adler.
881
882         * resources/sunspider-analyze-results.js: Do the error range
883         properly, using the t-distribution instead of 1.96 (which was
884         based on the normal distribution).
885         
886         * sunspider: Print results in a way that will be friendlier to the
887         soon-to-come compare mode.
888
889 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
890
891         Reviewed by Review.
892         
893         - add more tests (probably the final set for now)
894         
895         Already balanced. Added date, regexp, control flow, and a few
896         more string and object/array access tests.
897         
898         * tests/LIST:
899         * tests/access-fannkuch.js: Added.
900         * tests/access-nbody.js: Added.
901         * tests/controlflow-recursive.js: Added.
902         * tests/date-format-tofte.js: Added.
903         * tests/date-format-xparb.js: Added.
904         * tests/regexp-dna.js: Added.
905         * tests/string-validate-input.js: Added.
906
907 2007-10-21  Eric Seidel  <eric@webkit.org>
908
909         Reviewed by mjs.
910
911         * sunspider: make --shark only sample from testkjs
912
913 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
914
915         Rubber stamped by Eric.
916         
917         - added crypto tests (already balanced)
918
919         * tests/LIST:
920         * tests/crypto-aes.js: Added.
921         * tests/crypto-md5.js: Added.
922         * tests/crypto-sha1.js: Added.
923
924 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
925
926         Reviewed by Adam.
927         
928         - rebalance new tests
929         
930         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
931         
932         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
933
934         This balances them with the existing tests.
935
936         * tests/access-binary-trees.js:
937         * tests/access-nsieve.js:
938         * tests/bitops-nsieve-bits.js:
939         * tests/math-partial-sums.js:
940         * tests/math-spectral-norm.js:
941         * tests/string-fasta.js:
942
943 2007-10-20  Eric Seidel  <eric@webkit.org>
944
945         Reviewed by darin.
946         
947         Add improved argument handling to sunspider, including
948         --runs=<number>, --tests=<pattern, --shark, and --help
949         Also re-factor code into subroutines
950
951         * sunspider: improved argument handling
952
953 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
954
955         Reviewed by Mark.
956
957         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
958         
959         * TODO:
960         * tests/LIST:
961         * tests/access-binary-trees.js: Added.
962         * tests/access-nsieve.js: Added.
963         * tests/bitops-nsieve-bits.js: Added.
964         * tests/math-partial-sums.js: Added.
965         * tests/math-spectral-norm.js: Added.
966         * tests/string-fasta.js: Added.
967
968 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
969
970         Reviewed by Darin Adler.
971
972         * TODO: Updated to note areas that are now well-covered.
973         
974         - Change the tests and the driver to avoid leaving large object
975         graphs hanging around, since that throws off the subsequent tests.
976         
977         * resources/sunspider-standalone-driver.js:
978         * tests/3d-cube.js:
979         * tests/3d-morph.js:
980         * tests/3d-raytrace.js:
981         * tests/bitops-bitwise-and.js:
982         * tests/string-base64.js:
983         * tests/string-tagcloud.js:
984
985 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
986
987         Reviewed by Nikolas.
988         
989         - Rebalance test complexity.
990         
991         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
992         
993         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
994
995         However, for some tests, IE7 was a huge outlier compared to other
996         major browsers. For those tests, I used Firefox 2.0.0.8 on the
997         same OS and hardware as a normalization basis.
998         
999         * tests/3d-morph.js:
1000         * tests/3d-raytrace.js:
1001         * tests/bitops-3bit-bits-in-byte.js:
1002         * tests/bitops-bits-in-byte.js:
1003         * tests/bitops-bitwise-and.js:
1004         * tests/math-cordic.js:
1005         * tests/string-base64.js:
1006         * tests/string-tagcloud.js:
1007         * tests/string-unpack-code.js:
1008
1009 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1010
1011         Reviewed by Oliver.
1012         
1013         - Fix the browser-hosted driver to be compatible with Safari 2
1014
1015         * hosted/sunspider-driver.html:
1016
1017 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1018
1019         Rubber stamped by Hyatt.
1020         
1021         - Add BSD license notice to cordic test, the original author said to do so.
1022
1023         * tests/math-cordic.js:
1024
1025 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1026
1027         Reviewed by Eric.
1028         
1029         - Support Windows and Windows browsers.
1030
1031         * hosted/sunspider-results.html: Display results correctly in IE.
1032         * sunspider: tweaks to work on Windows, and with spaces in the path.
1033
1034 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1035
1036         Rubber stamped by Oliver.
1037
1038         * tests/string-tagcloud.js: Remove stray print()
1039
1040 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
1041
1042         Reviewed by Oliver.
1043         
1044         Add some string test cases.
1045
1046         * tests/string-base64.js: Added. Does base64 encoding and decoding.
1047         * tests/string-tagcloud.js: Added. Parses a JSON string and
1048         generates tagcloud HTML markup based on the data.        
1049         * tests/string-unpack-code.js: Added. Unpack compressed versions of
1050         four of the most popular JavaScript libraries.
1051         * tests/LIST: Added the new tests.
1052
1053 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
1054
1055         Reviewed by Anders.
1056
1057         * TODO: Updated with info on how to do the stats properly.
1058
1059         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
1060         avoid overly slowing down the benchmark.
1061
1062         Added some 3D-related test cases.
1063         
1064         * tests/3d-cube.js: Added.
1065         * tests/3d-morph.js: Added.
1066         * tests/3d-raytrace.js: Added.
1067         * tests/LIST:
1068
1069 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
1070
1071         Reviewed by Darin Adler.
1072
1073         - New JavaScript benchmark
1074         http://bugs.webkit.org/show_bug.cgi?id=15515
1075         
1076         A start on a new JavaScript benchmark, with standalone and
1077         browser-hosted drivers.
1078         
1079         The test content is still incomplete and the drivers could use
1080         some more features but there is enough here to make a good start.
1081         
1082         The drivers compute a 95% confidence interval on the mean for the
1083         whole test, each category, and each individual test to make it
1084         easier to tell whether differences are statistically
1085         significant. The confidence interval can be narrowed by running
1086         the test more times.
1087         
1088         * TODO: Added. Remaining things that need to be done.
1089         * sunspider: Added. Perl script that acts as the standalone test driver.
1090         * resources/sunspider-standalone-driver.js: Added. JavaScript part
1091         of standalone test driver.
1092         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
1093         analysis code.
1094         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
1095         * make-hosted: Added. Script to generate browser-hosted tests.
1096         * hosted/sunspider.html: Added. Start page for browser-hosted test.
1097         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
1098         * hosted/sunspider-results.html: Added. Results page for browser-hosted
1099         tests.
1100         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
1101         tests.
1102         * tests/LIST: Added. List of tests to use.
1103         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
1104         * tests/bitops-bits-in-byte.js: Added. Ditto.
1105         * tests/bitops-bitwise-and.js: Added. Ditto.
1106         * tests/math-cordic.js: Added. Ditto.
1107