9a73676e6f91edc0bfd2e4aebe37bef957cdb56e
[WebKit.git] / SunSpider / ChangeLog
1 2010-05-07  Adam Roben  <aroben@apple.com>
2
3         Fix typo in sunspider-compare-results.js that caused incorrect
4         two-sample t statistics to be computed
5
6         Fixes <http://webkit.org/b/38768> SunSpider computes incorrect
7         two-sample t statistics
8
9         Reviewed by Darin Adler.
10
11         * resources/sunspider-compare-results.js:
12         (sunspiderCompareResults.resultLine): Fixed the calculation of the t
13         variable to match the definition of a two-sample t statistic. Looks
14         like this was just a simple typo.
15
16 2010-04-05  Darin Adler  <darin@apple.com>
17
18         Updated the ignore property for changes to the contents here.
19
20         * .: Modified property svn:ignore.
21
22 2010-02-20  Maciej Stachowiak  <mjs@apple.com>
23
24         Reviewed by Adam Barth.
25
26         Add sunspider-0.9.1 to the site in preparation for announcing it.
27         https://bugs.webkit.org/show_bug.cgi?id=35206
28
29         * hosted/sunspider.css: Remove tabs, which I noticed where present while
30         copying to the WebKitSite directory.
31
32 2010-01-11  Oliver Hunt  <oliver@apple.com>
33
34         Reviewed by Darin Adler.
35
36         Sunspider parse-only tests don't run
37         https://bugs.webkit.org/show_bug.cgi?id=33489
38
39         Fix parse-only test list
40
41         * tests/parse-only/LIST:
42
43 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
44
45         Reviewed by Adele Peterson.
46
47         Make SunSpider version more prominent in the title
48         https://bugs.webkit.org/show_bug.cgi?id=32574
49
50         * hosted/sunspider.html:
51         * hosted/versions.html:
52         * resources/driver-TEMPLATE.html:
53         * resources/results-TEMPLATE.html:
54
55 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
56
57         Reviewed by Darin Adler.
58
59         Make SunSpider refuse to compare results across content versions
60         https://bugs.webkit.org/show_bug.cgi?id=32573
61         
62         The results URL now includes a version indicator, and compare mode
63         will refuse to compare between versions.
64
65         * resources/driver-TEMPLATE.html:
66         * resources/results-TEMPLATE.html:
67
68 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
69
70         Reviewed by Darin Adler.
71
72         Some Browser-hosted SunSpider files are not valid HTML5
73         https://bugs.webkit.org/show_bug.cgi?id=32536
74         
75         Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
76         close tags and adding a meta charset declaration).
77
78         * hosted/sunspider.html:
79         * hosted/versions.html:
80         * resources/TEMPLATE.html:
81         * resources/driver-TEMPLATE.html:
82         * resources/results-TEMPLATE.html:
83
84 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
85
86         Reviewed by Eric Seidel.
87
88         Make sunspider-0.9.1 the default content set (both command-line and hosted)
89         https://bugs.webkit.org/show_bug.cgi?id=32537
90
91         * hosted/sunspider.html: Change default suite link.
92         * sunspider: Change default suite.
93         * sunspider-compare-results: Likewise.
94
95 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
96
97         Reviewed by Sam Weinig.
98
99         in-browser SunSpider suffers excessive penalty under power management
100         https://bugs.webkit.org/show_bug.cgi?id=32505
101         
102         I have made a few changes to address this:
103         
104         1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
105         document.write to reduce triggering of progress UI.
106         2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
107         stepped all the way down the whole time.
108         3) Run the test cycle an extra time for warmup before the runs that count.
109
110         * hosted/sunspider-record-result.js: Removed.
111         * make-hosted:
112         * resources/TEMPLATE.html:
113         * resources/driver-TEMPLATE.html:
114
115 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
116
117         Reviewed by Gavin Barraclough.
118
119         SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
120         https://bugs.webkit.org/show_bug.cgi?id=16806
121
122         Based on a patch by Eric Seidel.
123         
124         Fix the base64 computation to actually compute correct results. The impact on runtime of
125         the test is pretty small, but noticeable for some browsers. But at least it's not
126         doing a wrong and meaningless computation any more.
127         
128         * tests/sunspider-0.9.1/string-base64.js:
129         ():
130         (base64ToString):
131
132 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
133
134         Fixing commit error...
135         
136         I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
137
138         * resources/driver-TEMPLATE.html:
139
140 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
141
142         Reviewed by Sam Weinig.
143
144         Use JSON.parse instead of eval for Web-hosted SunSpider results processing
145         https://bugs.webkit.org/show_bug.cgi?id=32490
146
147         * hosted/json2.js: Added. Incorporated from json.org
148         * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
149         * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
150         results.
151
152 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
153
154         Reviewed by Oliver Hunt.
155
156         Enable Web-hosted version of SunSpider to handle multiple versions
157         https://bugs.webkit.org/show_bug.cgi?id=32478
158
159         * make-hosted: Now generate the hosted version to be able to run both the
160         0.9 and the 0.9.1 test suites.
161         * hosted: Modified property svn:ignore.
162         * hosted/sunspider-driver.html: Removed.
163         * hosted/sunspider-results.html: Removed.
164         * hosted/sunspider.html:
165         * hosted/versions.html: Added.
166         * resources/TEMPLATE.html:
167         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
168         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
169
170 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
171
172         Reviewed by Oliver Hunt.
173
174         Give command-line sunspider the ability to handle multiple suites and versions
175         https://bugs.webkit.org/show_bug.cgi?id=32477
176         
177         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
178         enumerating the tests. Current suites are:
179         
180         sunspider-0.9 - riginal SunSpider content 
181         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
182         ubench - "ubench" microbenchmark suite
183         v8-v4 - version 4 of the v8 benchmark
184         parse-only - some parser tests
185         
186         Sunspider can now be run with the --suite= parameter to select a
187         suite. The default is --suite=sunspider-0.9.
188
189         * .: Modified property svn:ignore to ignore new results directories.
190         * make-hosted: Generate based on sunspider-0.9 suite.
191         * resources/TEMPLATE.html: Fix an HTML compliance problem.
192         * resources/sunspider-standalone-driver.js: Become aware of suite names.
193         * sunspider: Modified to handle multiple suites as described above.
194         * sunspider-compare-results: ditto
195         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
196         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
197         * tests/LIST-UBENCH: Removed.
198         * tests/LIST-V8: Removed.
199         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
200         * tests/3d-morph.js: ditto
201         * tests/3d-raytrace.js: ditto
202         * tests/access-binary-trees.js: ditto
203         * tests/access-fannkuch.js: ditto
204         * tests/access-nbody.js: ditto
205         * tests/access-nsieve.js: ditto
206         * tests/bitops-3bit-bits-in-byte.js: ditto
207         * tests/bitops-bits-in-byte.js: ditto
208         * tests/bitops-bitwise-and.js: ditto
209         * tests/bitops-nsieve-bits.js: ditto
210         * tests/controlflow-recursive.js: ditto
211         * tests/crypto-aes.js: ditto
212         * tests/crypto-md5.js: ditto
213         * tests/crypto-sha1.js: ditto
214         * tests/date-format-tofte.js: ditto
215         * tests/date-format-xparb.js: ditto
216         * tests/math-cordic.js: ditto
217         * tests/math-partial-sums.js: ditto
218         * tests/math-spectral-norm.js: ditto
219         * tests/regexp-dna.js: ditto
220         * tests/string-base64.js: ditto
221         * tests/string-fasta.js: ditto
222         * tests/string-tagcloud.js: ditto
223         * tests/string-unpack-code.js: ditto
224         * tests/string-validate-input.js: ditto
225         * tests/sunspider-0.9/: Added
226         * tests/sunspider-0.9/*.js: Moved from one level up
227         * tests/sunspider-0.9.1: Added.
228         * tests/sunspider-0.9.1/*.js: Moved from one level up.
229         * tests/v8-crypto.js: Moved to v8-v4/ directory.
230         * tests/v8-deltablue.js: ditto
231         * tests/v8-earley-boyer.js: ditto
232         * tests/v8-raytrace.js: ditto
233         * tests/v8-regexp.js: ditto
234         * tests/v8-richards.js: ditto
235         * tests/v8-splay.js: ditto
236         * tests/v8-v4: Added.
237
238 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
239
240         Reviewed by Darin Adler.
241
242         Platform specific null device replaced with a platform independent.
243         https://bugs.webkit.org/show_bug.cgi?id=29544
244
245         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
246
247 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
248
249         Reviewed by Maciej Stachowiak.
250
251         [Qt] Make sunspider script work on Windows platform.
252         https://bugs.webkit.org/show_bug.cgi?id=29544
253
254         * sunspider:
255         Using platform specific null device instead of /dev/null.
256
257 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
258
259         Reviewed by Oliver Hunt.
260
261         https://bugs.webkit.org/show_bug.cgi?id=26540
262         Currently the SunSpider test driver lacks an option to run a test suite that
263         will test JavaScriptCore parsing performance only.  This patch adds just such
264         a test suite and option to SunSpider as well as the jsc test shell.  I've included
265         three large javascript source files found in the wild: jquery, mootools and prototype.
266         Combined with the concatenation of all three, these form a new testsuite to measure
267         and test pure JavaScriptCore parsing performance.
268
269         * resources/sunspider-standalone-driver.js:
270           Load the js with parse-only flag if it is part of the parse-only suite.
271         * sunspider:
272           Add the parse-only suite to the list of options.
273         * sunspider-compare-results:
274           Add the parse-only suite to the list of options.
275         * tests/LIST-PARSE-ONLY: Added.
276         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
277           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
278         * tests/parse-only/jquery-1.3.2.js: Added.
279           Latest version from jquery.com.
280         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
281           Latest version from mootools.net.
282         * tests/parse-only/prototype-1.6.0.3.js: Added.
283           Latest version from prototypejs.org.
284
285 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
286
287         Update V8 benchmark to version 4.
288         
289         Reviewed by Cameron Zwarich.
290
291         * tests/LIST-V8:
292         * tests/v8-crypto.js:
293         * tests/v8-deltablue.js:
294         * tests/v8-earley-boyer.js:
295         * tests/v8-raytrace.js:
296         * tests/v8-regexp.js: Added.
297         * tests/v8-richards.js:
298         * tests/v8-splay.js: Added.
299
300 2009-05-26  Olivier DOLE  <odole@pleyo.com>
301
302         Reviewed by Maciej.
303
304         Store sunspider test suite results in a specific directory for each
305         different type of benchmark (sunspider, V8 or ubench).
306
307         * sunspider:
308         * sunspider-compare-results:
309
310 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
311
312         Build fix.
313
314         * SunSpider.make:
315
316 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
317
318         Include SunSpider tests in production builds.
319         
320         Reviewed by Adam Roben.
321
322         * SunSpider.make: Added.
323         * xcopy.excludes: Added.
324
325 2009-03-04  Oliver Hunt  <oliver@apple.com>
326
327         Added a few more tests to ubench
328
329         RS=Gavin Barraclough
330
331         Adding a couple of test to cover too few and too many arguments.  This
332         is needed to prevent regressions in all call paths from my current refactoring.
333
334         * tests/LIST-UBENCH:
335         * tests/ubench/function-correct-args.js: Added.
336         (f):
337         * tests/ubench/function-excess-args.js: Added.
338         (f):
339
340 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
341
342         Fix a bug where when V8's or UBENCH's test suite list file is missing
343         the error message is suggesting that SunSpider's list file is missing.
344
345         Reviewed by Sam Weinig.
346
347         * sunspider:
348         (loadTestsList): Fix the error message to use the right list name.
349
350 2008-10-07  David Hyatt  <hyatt@apple.com>
351
352         https://bugs.webkit.org/show_bug.cgi?id=18989
353
354         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
355         we want all browsers to be doing the same thing on the test.
356
357         Reviewed by Maciej
358
359         * tests/regexp-dna.js:
360
361 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
362
363         Rubber-stamped by Maciej Stachowiak.
364
365         The bleeding edge branch of V8 now supports passing fileanmes after
366         '-f', so the SunSpider script should be changed to do this again.
367         This allows all 3 major JavaScript engines to be used with the
368         unmodified SunSpider script.
369
370         * sunspider:
371
372 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
373
374         Reviewed by Maciej Stachowiak.
375
376         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
377         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
378         correct the punctuation in its documentation.
379
380         * sunspider:
381
382 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
383
384         Reviewed by Oliver Hunt.
385
386         Bug 20616: Incorporate V8 benchmarks in testing
387         <https://bugs.webkit.org/show_bug.cgi?id=20616>
388
389         This is a first attempt at running the V8 benchmarks within SunSpider,
390         with the --v8 option. The number of iterations is fixed for each
391         benchmark, and was chosen to make each of the benchmarks run between
392         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
393
394         * sunspider:
395         * tests/LIST-V8: Added.
396         * tests/v8-crypto.js: Added.
397         * tests/v8-deltablue.js: Added.
398         * tests/v8-earley-boyer.js: Added.
399         * tests/v8-raytrace.js: Added.
400         * tests/v8-richards.js: Added.
401
402 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
403
404         Not reviewed.
405
406         Remove a local change that was accidentally committed in r36034.
407
408         * resources/sunspider-standalone-driver.js:
409
410 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
411
412         Rubber-stamped by Oliver Hunt.
413
414         Make SunSpider work with JavaScript shells that don't take filenames
415         after a '-f' argument.
416
417         * sunspider:
418
419 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
420
421         Reviewed by Timothy.
422
423         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
424
425         * sunspider:
426         * tests/LIST-SQUIRRELFISH: Removed.
427
428 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
429
430         Rubber-stamped by Mark Rowe.
431
432         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
433
434         * sunspider:
435
436 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
437
438         Rubber stamped by Oliver.
439         
440         - add newly working tests to squirrelfish list
441
442         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
443
444 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
445
446         Rubber stamped by Oliver.
447         
448         - move ubench tests properly.
449
450         * tests/ubench: Added.
451         * tests/ubench/function-closure.js: Added.
452         * tests/ubench/function-empty.js: Added.
453         * tests/ubench/function-missing-args.js: Added.
454         * tests/ubench/function-sum.js: Added.
455         * tests/ubench/loop-empty-resolve.js: Added.
456         * tests/ubench/loop-empty.js: Added.
457         * tests/ubench/loop-sum.js: Added.
458
459 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
460
461         Rubber stamped by Oliver.
462
463         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
464
465 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
466
467         Reviewed by Oliver.
468
469         - make --squirrelfish mode use the real harness, but a reduced set of tests
470
471         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
472
473         * sunspider:
474         * tests/LIST-SQUIRRELFISH:
475         * tests/LIST-UBENCH: Added.
476         * tests/squirrelfish: Moved to tests/ubench
477
478 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
479
480         Reviewed by Maciej Stachowiak.
481         
482         Removed an unused test.
483
484         * tests/squirrelfish/loop-resolve.js: Removed.
485
486 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
487
488         Reviewed by Oliver.
489         
490         Add newly runnable tests to --squirrelfish mode.
491
492         * tests/LIST-SQUIRRELFISH:
493
494 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
495
496         Reviewed by Maciej Stachowiak.
497         
498         A little more love for --squirrelfish mode:
499         
500         Fixed a misplaced sort. Results properly sort now.
501         
502         Added a test list just for squirrelfish, and updated it to include
503         all currently passing SunSpider tests.
504         
505         Fixed the pruning regexp to match 3d-morph.js.
506
507         * sunspider:
508
509 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
510
511         Reviewed by Oliver Hunt.
512         
513         --squirrelfish mode: pared down tests for squirrelfish to chew on.
514
515         * sunspider:
516         * tests/LIST:
517         * tests/squirrelfish: Added.
518         * tests/squirrelfish/function-closure.js: Added.
519         * tests/squirrelfish/function-empty.js: Added.
520         * tests/squirrelfish/function-missing-args.js: Added.
521         * tests/squirrelfish/function-sum.js: Added.
522         * tests/squirrelfish/loop-empty-resolve.js: Added.
523         * tests/squirrelfish/loop-empty.js: Added.
524         * tests/squirrelfish/loop-resolve.js: Added.
525         * tests/squirrelfish/loop-sum.js: Added.
526
527 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
528
529         Reviewed by Geoff.
530
531         * sunspider: Pause update daemon on Darwin to make results reliable
532         again (workaround for <rdar://problem/5811127>).
533
534 2008-02-07  Eric Seidel  <eric@webkit.org>
535
536         Reviewed by Sam.
537
538         * sunspider: add an error message when ./tests/LIST is not found
539
540 2007-12-29  Alp Toker  <alp@atoker.com>
541
542         Fix typo.
543
544         * hosted/sunspider.html:
545
546 2007-12-29  Alp Toker  <alp@atoker.com>
547
548         Reviewed by Maciej.
549
550         Provide the current URL in the SunSpider results page. Useful for
551         copying benchmark results out of browsers without a location bar or
552         where the location bar can't handle long URLs.
553
554         * hosted/sunspider-results.html:
555
556 2007-12-20  Eric Seidel  <eric@webkit.org>
557
558         Reviewed by mjs.
559
560         * sunspider: increase default number of runs for more accurate numbers
561
562 2007-12-18  Mark Rowe  <mrowe@apple.com>
563
564         Fix typo.
565
566         * hosted/sunspider.html:
567
568 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
569
570         Reviewed by Eric.
571         
572         - give an extra digit of precision for small differences.
573
574         * resources/sunspider-compare-results.js:
575
576 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
577
578         Reviewed by Eric.
579         
580         - work around IE layout bugs
581
582         * hosted/sunspider.css:
583
584 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
585
586         Reviewed by Mark.
587         
588         - fix compare feature
589
590         * hosted/sunspider-results.html:
591
592 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
593
594         Reviewed by Mark.
595         
596         - Change the format of the output URL to be shorter, to meet IE constraints
597
598         * hosted/sunspider-driver.html:
599         * hosted/sunspider-results.html:
600
601 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
602
603         Reviewed by Oliver.
604         
605         - Add some explanatory test and a dumb logo to SunSpider web hosted version
606
607         * hosted/sunspider-driver.html:
608         * hosted/sunspider-results.html:
609         * hosted/sunspider.css:
610         * hosted/sunspider.html:
611
612 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
613
614         Reviewed by Oliver.
615         
616         - Make the web-hosted version of SunSpider look a little bit less drab.
617
618         * hosted/sunspider-driver.html:
619         * hosted/sunspider-results.html:
620         * hosted/sunspider.css: Added.
621         * hosted/sunspider.html:
622         * resources/TEMPLATE.html:
623
624 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
625
626         Reviewed by Sam.
627         
628         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
629         
630         A sample of what new-style results would look like:
631         
632         TEST                   COMPARISON            FROM                 TO             DETAILS
633         ...
634         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
635         ...
636             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
637         
638         * resources/sunspider-compare-results.js:
639
640 2007-11-14  Eric Seidel  <eric@webkit.org>
641
642         Reviewed by Sam.
643
644         * sunspider: add --shark-cache for L2 Cache Miss profiling
645
646 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
647
648         Reviewed by Darin Adler.
649
650         http://bugs.webkit.org/show_bug.cgi?id=15925
651         SunSpider should check for Shark being installed
652
653         * sunspider: Explicitly test that shark command line tool is installed.
654
655 2007-11-04  Darin Adler  <darin@apple.com>
656
657         * hosted/sunspider.html: Tweaked formatting a bit.
658
659 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
660
661         Reviewed by Darin Adler.
662
663         - Add compare mode and a description of the benchmark to the hosted version.
664
665         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
666         * hosted/sunspider.html: Added intro text.
667         * make-hosted: Copy new file.
668         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
669         a function.
670         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
671         expects.
672         * sunspider-compare-results: Use sunspider-standalone-compare.js.
673
674 2007-11-02  Darin Adler  <darin@apple.com>
675
676         * sunspider: Changed "--base" to "--set-baseline".
677
678 2007-11-02  Darin Adler  <darin@apple.com>
679
680         Reviewed by Maciej.
681
682         * sunspider: Added a "--base" option that records the current test as a baseline.
683         Moved the code to find the newest .mshark file into a function (since I needed to use
684         it in the other script and I prefer to copy/paste a function).
685
686         * sunspider-compare-results: Added logic to compare the baseline with the last
687         run when you don't pass any parameters.
688
689 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
690
691         Reviewed by Eric.
692         
693         - Added list of ECMAScript3 features that SunSpider does not cover currently.
694
695         * UNCOVERED: Added.
696
697 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
698
699         Reviewed by Oliver.
700         
701         - Give better error messages when sunspider-compare-results fails
702
703         * sunspider-compare-results:
704
705 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
706
707         Reviewed by Darin Adler.
708         
709         - fix Date tests to work in command-line SpiderMonkey
710
711         * tests/date-format-tofte.js:
712         * tests/date-format-xparb.js:
713
714 2007-10-29  Darin Adler  <darin@apple.com>
715
716         Reviewed by Maciej.
717
718         * sunspider: Added --shark20 option, to run Shark at its highest sample
719         resolution instead of its default.
720
721 2007-10-28  Adam Roben  <aroben@apple.com>
722
723         Fix SunSpider on Windows
724
725         Reviewed by Eric.
726
727         * sunspider: Don't use colons in filenames.
728
729 2007-10-28  Darin Adler  <darin@apple.com>
730
731         Reviewed by Adam.
732
733         * resources/sunspider-analyze-results.js: Tweak the output format a little.
734         Change so that when there's only one run we don't write out confidence
735         intervals at all rather than writing "NaN" over and over again.
736
737         * sunspider: Use the time and date as a suffix on the results file, that way
738         all the results are left behind in the tmp directory. This will make it easy
739         for us to add features that compare past results in the future. Also rename
740         the .mshark files using the same scheme.
741
742         * sunspider-compare-results: Relax the parsing rules so we can parse the
743         results file exactly as it's emitted from sunspider.
744
745 2007-10-25  Adam Roben  <aroben@apple.com>
746
747         Strip carriage returns from results
748
749         Carriage returns were screwing up sunspider-compare-results on
750         Windows.
751
752         Rubberstamped by Oliver.
753
754         * sunspider:
755
756 2007-10-24  Darin Adler  <darin@apple.com>
757
758         Reviewed by Eric.
759
760         * sunspider: Open the profile in shark after completing a run.
761         * sunspider-compare-results: Tiny perl tweaks.
762
763 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
764
765         Reviewed by Darin Adler.
766
767         - Don't hardcode my path to testkjs
768         
769         * sunspider-compare-results:
770
771 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
772
773         Reviewed by Darin Adler.
774         
775         - Add a compare script to compare results, and improve formatting of standard results a bit.
776
777         * resources/sunspider-analyze-results.js:
778         * resources/sunspider-compare-results.js: Added.
779         * sunspider-compare-results: Added.
780
781 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
782
783         Reviewed by Darin Adler.
784
785         * resources/sunspider-analyze-results.js: Do the error range
786         properly, using the t-distribution instead of 1.96 (which was
787         based on the normal distribution).
788         
789         * sunspider: Print results in a way that will be friendlier to the
790         soon-to-come compare mode.
791
792 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
793
794         Reviewed by Review.
795         
796         - add more tests (probably the final set for now)
797         
798         Already balanced. Added date, regexp, control flow, and a few
799         more string and object/array access tests.
800         
801         * tests/LIST:
802         * tests/access-fannkuch.js: Added.
803         * tests/access-nbody.js: Added.
804         * tests/controlflow-recursive.js: Added.
805         * tests/date-format-tofte.js: Added.
806         * tests/date-format-xparb.js: Added.
807         * tests/regexp-dna.js: Added.
808         * tests/string-validate-input.js: Added.
809
810 2007-10-21  Eric Seidel  <eric@webkit.org>
811
812         Reviewed by mjs.
813
814         * sunspider: make --shark only sample from testkjs
815
816 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
817
818         Rubber stamped by Eric.
819         
820         - added crypto tests (already balanced)
821
822         * tests/LIST:
823         * tests/crypto-aes.js: Added.
824         * tests/crypto-md5.js: Added.
825         * tests/crypto-sha1.js: Added.
826
827 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
828
829         Reviewed by Adam.
830         
831         - rebalance new tests
832         
833         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
834         
835         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
836
837         This balances them with the existing tests.
838
839         * tests/access-binary-trees.js:
840         * tests/access-nsieve.js:
841         * tests/bitops-nsieve-bits.js:
842         * tests/math-partial-sums.js:
843         * tests/math-spectral-norm.js:
844         * tests/string-fasta.js:
845
846 2007-10-20  Eric Seidel  <eric@webkit.org>
847
848         Reviewed by darin.
849         
850         Add improved argument handling to sunspider, including
851         --runs=<number>, --tests=<pattern, --shark, and --help
852         Also re-factor code into subroutines
853
854         * sunspider: improved argument handling
855
856 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
857
858         Reviewed by Mark.
859
860         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
861         
862         * TODO:
863         * tests/LIST:
864         * tests/access-binary-trees.js: Added.
865         * tests/access-nsieve.js: Added.
866         * tests/bitops-nsieve-bits.js: Added.
867         * tests/math-partial-sums.js: Added.
868         * tests/math-spectral-norm.js: Added.
869         * tests/string-fasta.js: Added.
870
871 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
872
873         Reviewed by Darin Adler.
874
875         * TODO: Updated to note areas that are now well-covered.
876         
877         - Change the tests and the driver to avoid leaving large object
878         graphs hanging around, since that throws off the subsequent tests.
879         
880         * resources/sunspider-standalone-driver.js:
881         * tests/3d-cube.js:
882         * tests/3d-morph.js:
883         * tests/3d-raytrace.js:
884         * tests/bitops-bitwise-and.js:
885         * tests/string-base64.js:
886         * tests/string-tagcloud.js:
887
888 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
889
890         Reviewed by Nikolas.
891         
892         - Rebalance test complexity.
893         
894         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
895         
896         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
897
898         However, for some tests, IE7 was a huge outlier compared to other
899         major browsers. For those tests, I used Firefox 2.0.0.8 on the
900         same OS and hardware as a normalization basis.
901         
902         * tests/3d-morph.js:
903         * tests/3d-raytrace.js:
904         * tests/bitops-3bit-bits-in-byte.js:
905         * tests/bitops-bits-in-byte.js:
906         * tests/bitops-bitwise-and.js:
907         * tests/math-cordic.js:
908         * tests/string-base64.js:
909         * tests/string-tagcloud.js:
910         * tests/string-unpack-code.js:
911
912 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
913
914         Reviewed by Oliver.
915         
916         - Fix the browser-hosted driver to be compatible with Safari 2
917
918         * hosted/sunspider-driver.html:
919
920 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
921
922         Rubber stamped by Hyatt.
923         
924         - Add BSD license notice to cordic test, the original author said to do so.
925
926         * tests/math-cordic.js:
927
928 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
929
930         Reviewed by Eric.
931         
932         - Support Windows and Windows browsers.
933
934         * hosted/sunspider-results.html: Display results correctly in IE.
935         * sunspider: tweaks to work on Windows, and with spaces in the path.
936
937 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
938
939         Rubber stamped by Oliver.
940
941         * tests/string-tagcloud.js: Remove stray print()
942
943 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
944
945         Reviewed by Oliver.
946         
947         Add some string test cases.
948
949         * tests/string-base64.js: Added. Does base64 encoding and decoding.
950         * tests/string-tagcloud.js: Added. Parses a JSON string and
951         generates tagcloud HTML markup based on the data.        
952         * tests/string-unpack-code.js: Added. Unpack compressed versions of
953         four of the most popular JavaScript libraries.
954         * tests/LIST: Added the new tests.
955
956 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
957
958         Reviewed by Anders.
959
960         * TODO: Updated with info on how to do the stats properly.
961
962         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
963         avoid overly slowing down the benchmark.
964
965         Added some 3D-related test cases.
966         
967         * tests/3d-cube.js: Added.
968         * tests/3d-morph.js: Added.
969         * tests/3d-raytrace.js: Added.
970         * tests/LIST:
971
972 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
973
974         Reviewed by Darin Adler.
975
976         - New JavaScript benchmark
977         http://bugs.webkit.org/show_bug.cgi?id=15515
978         
979         A start on a new JavaScript benchmark, with standalone and
980         browser-hosted drivers.
981         
982         The test content is still incomplete and the drivers could use
983         some more features but there is enough here to make a good start.
984         
985         The drivers compute a 95% confidence interval on the mean for the
986         whole test, each category, and each individual test to make it
987         easier to tell whether differences are statistically
988         significant. The confidence interval can be narrowed by running
989         the test more times.
990         
991         * TODO: Added. Remaining things that need to be done.
992         * sunspider: Added. Perl script that acts as the standalone test driver.
993         * resources/sunspider-standalone-driver.js: Added. JavaScript part
994         of standalone test driver.
995         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
996         analysis code.
997         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
998         * make-hosted: Added. Script to generate browser-hosted tests.
999         * hosted/sunspider.html: Added. Start page for browser-hosted test.
1000         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
1001         * hosted/sunspider-results.html: Added. Results page for browser-hosted
1002         tests.
1003         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
1004         tests.
1005         * tests/LIST: Added. List of tests to use.
1006         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
1007         * tests/bitops-bits-in-byte.js: Added. Ditto.
1008         * tests/bitops-bitwise-and.js: Added. Ditto.
1009         * tests/math-cordic.js: Added. Ditto.
1010