2010-05-03 Abhishek Arya <inferno@chromium.org>
[WebKit-https.git] / SunSpider / ChangeLog
1 2010-04-05  Darin Adler  <darin@apple.com>
2
3         Updated the ignore property for changes to the contents here.
4
5         * .: Modified property svn:ignore.
6
7 2010-02-20  Maciej Stachowiak  <mjs@apple.com>
8
9         Reviewed by Adam Barth.
10
11         Add sunspider-0.9.1 to the site in preparation for announcing it.
12         https://bugs.webkit.org/show_bug.cgi?id=35206
13
14         * hosted/sunspider.css: Remove tabs, which I noticed where present while
15         copying to the WebKitSite directory.
16
17 2010-01-11  Oliver Hunt  <oliver@apple.com>
18
19         Reviewed by Darin Adler.
20
21         Sunspider parse-only tests don't run
22         https://bugs.webkit.org/show_bug.cgi?id=33489
23
24         Fix parse-only test list
25
26         * tests/parse-only/LIST:
27
28 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
29
30         Reviewed by Adele Peterson.
31
32         Make SunSpider version more prominent in the title
33         https://bugs.webkit.org/show_bug.cgi?id=32574
34
35         * hosted/sunspider.html:
36         * hosted/versions.html:
37         * resources/driver-TEMPLATE.html:
38         * resources/results-TEMPLATE.html:
39
40 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
41
42         Reviewed by Darin Adler.
43
44         Make SunSpider refuse to compare results across content versions
45         https://bugs.webkit.org/show_bug.cgi?id=32573
46         
47         The results URL now includes a version indicator, and compare mode
48         will refuse to compare between versions.
49
50         * resources/driver-TEMPLATE.html:
51         * resources/results-TEMPLATE.html:
52
53 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
54
55         Reviewed by Darin Adler.
56
57         Some Browser-hosted SunSpider files are not valid HTML5
58         https://bugs.webkit.org/show_bug.cgi?id=32536
59         
60         Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
61         close tags and adding a meta charset declaration).
62
63         * hosted/sunspider.html:
64         * hosted/versions.html:
65         * resources/TEMPLATE.html:
66         * resources/driver-TEMPLATE.html:
67         * resources/results-TEMPLATE.html:
68
69 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
70
71         Reviewed by Eric Seidel.
72
73         Make sunspider-0.9.1 the default content set (both command-line and hosted)
74         https://bugs.webkit.org/show_bug.cgi?id=32537
75
76         * hosted/sunspider.html: Change default suite link.
77         * sunspider: Change default suite.
78         * sunspider-compare-results: Likewise.
79
80 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
81
82         Reviewed by Sam Weinig.
83
84         in-browser SunSpider suffers excessive penalty under power management
85         https://bugs.webkit.org/show_bug.cgi?id=32505
86         
87         I have made a few changes to address this:
88         
89         1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
90         document.write to reduce triggering of progress UI.
91         2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
92         stepped all the way down the whole time.
93         3) Run the test cycle an extra time for warmup before the runs that count.
94
95         * hosted/sunspider-record-result.js: Removed.
96         * make-hosted:
97         * resources/TEMPLATE.html:
98         * resources/driver-TEMPLATE.html:
99
100 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
101
102         Reviewed by Gavin Barraclough.
103
104         SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
105         https://bugs.webkit.org/show_bug.cgi?id=16806
106
107         Based on a patch by Eric Seidel.
108         
109         Fix the base64 computation to actually compute correct results. The impact on runtime of
110         the test is pretty small, but noticeable for some browsers. But at least it's not
111         doing a wrong and meaningless computation any more.
112         
113         * tests/sunspider-0.9.1/string-base64.js:
114         ():
115         (base64ToString):
116
117 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
118
119         Fixing commit error...
120         
121         I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
122
123         * resources/driver-TEMPLATE.html:
124
125 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
126
127         Reviewed by Sam Weinig.
128
129         Use JSON.parse instead of eval for Web-hosted SunSpider results processing
130         https://bugs.webkit.org/show_bug.cgi?id=32490
131
132         * hosted/json2.js: Added. Incorporated from json.org
133         * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
134         * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
135         results.
136
137 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
138
139         Reviewed by Oliver Hunt.
140
141         Enable Web-hosted version of SunSpider to handle multiple versions
142         https://bugs.webkit.org/show_bug.cgi?id=32478
143
144         * make-hosted: Now generate the hosted version to be able to run both the
145         0.9 and the 0.9.1 test suites.
146         * hosted: Modified property svn:ignore.
147         * hosted/sunspider-driver.html: Removed.
148         * hosted/sunspider-results.html: Removed.
149         * hosted/sunspider.html:
150         * hosted/versions.html: Added.
151         * resources/TEMPLATE.html:
152         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
153         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
154
155 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
156
157         Reviewed by Oliver Hunt.
158
159         Give command-line sunspider the ability to handle multiple suites and versions
160         https://bugs.webkit.org/show_bug.cgi?id=32477
161         
162         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
163         enumerating the tests. Current suites are:
164         
165         sunspider-0.9 - riginal SunSpider content 
166         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
167         ubench - "ubench" microbenchmark suite
168         v8-v4 - version 4 of the v8 benchmark
169         parse-only - some parser tests
170         
171         Sunspider can now be run with the --suite= parameter to select a
172         suite. The default is --suite=sunspider-0.9.
173
174         * .: Modified property svn:ignore to ignore new results directories.
175         * make-hosted: Generate based on sunspider-0.9 suite.
176         * resources/TEMPLATE.html: Fix an HTML compliance problem.
177         * resources/sunspider-standalone-driver.js: Become aware of suite names.
178         * sunspider: Modified to handle multiple suites as described above.
179         * sunspider-compare-results: ditto
180         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
181         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
182         * tests/LIST-UBENCH: Removed.
183         * tests/LIST-V8: Removed.
184         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
185         * tests/3d-morph.js: ditto
186         * tests/3d-raytrace.js: ditto
187         * tests/access-binary-trees.js: ditto
188         * tests/access-fannkuch.js: ditto
189         * tests/access-nbody.js: ditto
190         * tests/access-nsieve.js: ditto
191         * tests/bitops-3bit-bits-in-byte.js: ditto
192         * tests/bitops-bits-in-byte.js: ditto
193         * tests/bitops-bitwise-and.js: ditto
194         * tests/bitops-nsieve-bits.js: ditto
195         * tests/controlflow-recursive.js: ditto
196         * tests/crypto-aes.js: ditto
197         * tests/crypto-md5.js: ditto
198         * tests/crypto-sha1.js: ditto
199         * tests/date-format-tofte.js: ditto
200         * tests/date-format-xparb.js: ditto
201         * tests/math-cordic.js: ditto
202         * tests/math-partial-sums.js: ditto
203         * tests/math-spectral-norm.js: ditto
204         * tests/regexp-dna.js: ditto
205         * tests/string-base64.js: ditto
206         * tests/string-fasta.js: ditto
207         * tests/string-tagcloud.js: ditto
208         * tests/string-unpack-code.js: ditto
209         * tests/string-validate-input.js: ditto
210         * tests/sunspider-0.9/: Added
211         * tests/sunspider-0.9/*.js: Moved from one level up
212         * tests/sunspider-0.9.1: Added.
213         * tests/sunspider-0.9.1/*.js: Moved from one level up.
214         * tests/v8-crypto.js: Moved to v8-v4/ directory.
215         * tests/v8-deltablue.js: ditto
216         * tests/v8-earley-boyer.js: ditto
217         * tests/v8-raytrace.js: ditto
218         * tests/v8-regexp.js: ditto
219         * tests/v8-richards.js: ditto
220         * tests/v8-splay.js: ditto
221         * tests/v8-v4: Added.
222
223 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
224
225         Reviewed by Darin Adler.
226
227         Platform specific null device replaced with a platform independent.
228         https://bugs.webkit.org/show_bug.cgi?id=29544
229
230         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
231
232 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
233
234         Reviewed by Maciej Stachowiak.
235
236         [Qt] Make sunspider script work on Windows platform.
237         https://bugs.webkit.org/show_bug.cgi?id=29544
238
239         * sunspider:
240         Using platform specific null device instead of /dev/null.
241
242 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
243
244         Reviewed by Oliver Hunt.
245
246         https://bugs.webkit.org/show_bug.cgi?id=26540
247         Currently the SunSpider test driver lacks an option to run a test suite that
248         will test JavaScriptCore parsing performance only.  This patch adds just such
249         a test suite and option to SunSpider as well as the jsc test shell.  I've included
250         three large javascript source files found in the wild: jquery, mootools and prototype.
251         Combined with the concatenation of all three, these form a new testsuite to measure
252         and test pure JavaScriptCore parsing performance.
253
254         * resources/sunspider-standalone-driver.js:
255           Load the js with parse-only flag if it is part of the parse-only suite.
256         * sunspider:
257           Add the parse-only suite to the list of options.
258         * sunspider-compare-results:
259           Add the parse-only suite to the list of options.
260         * tests/LIST-PARSE-ONLY: Added.
261         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
262           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
263         * tests/parse-only/jquery-1.3.2.js: Added.
264           Latest version from jquery.com.
265         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
266           Latest version from mootools.net.
267         * tests/parse-only/prototype-1.6.0.3.js: Added.
268           Latest version from prototypejs.org.
269
270 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
271
272         Update V8 benchmark to version 4.
273         
274         Reviewed by Cameron Zwarich.
275
276         * tests/LIST-V8:
277         * tests/v8-crypto.js:
278         * tests/v8-deltablue.js:
279         * tests/v8-earley-boyer.js:
280         * tests/v8-raytrace.js:
281         * tests/v8-regexp.js: Added.
282         * tests/v8-richards.js:
283         * tests/v8-splay.js: Added.
284
285 2009-05-26  Olivier DOLE  <odole@pleyo.com>
286
287         Reviewed by Maciej.
288
289         Store sunspider test suite results in a specific directory for each
290         different type of benchmark (sunspider, V8 or ubench).
291
292         * sunspider:
293         * sunspider-compare-results:
294
295 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
296
297         Build fix.
298
299         * SunSpider.make:
300
301 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
302
303         Include SunSpider tests in production builds.
304         
305         Reviewed by Adam Roben.
306
307         * SunSpider.make: Added.
308         * xcopy.excludes: Added.
309
310 2009-03-04  Oliver Hunt  <oliver@apple.com>
311
312         Added a few more tests to ubench
313
314         RS=Gavin Barraclough
315
316         Adding a couple of test to cover too few and too many arguments.  This
317         is needed to prevent regressions in all call paths from my current refactoring.
318
319         * tests/LIST-UBENCH:
320         * tests/ubench/function-correct-args.js: Added.
321         (f):
322         * tests/ubench/function-excess-args.js: Added.
323         (f):
324
325 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
326
327         Fix a bug where when V8's or UBENCH's test suite list file is missing
328         the error message is suggesting that SunSpider's list file is missing.
329
330         Reviewed by Sam Weinig.
331
332         * sunspider:
333         (loadTestsList): Fix the error message to use the right list name.
334
335 2008-10-07  David Hyatt  <hyatt@apple.com>
336
337         https://bugs.webkit.org/show_bug.cgi?id=18989
338
339         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
340         we want all browsers to be doing the same thing on the test.
341
342         Reviewed by Maciej
343
344         * tests/regexp-dna.js:
345
346 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
347
348         Rubber-stamped by Maciej Stachowiak.
349
350         The bleeding edge branch of V8 now supports passing fileanmes after
351         '-f', so the SunSpider script should be changed to do this again.
352         This allows all 3 major JavaScript engines to be used with the
353         unmodified SunSpider script.
354
355         * sunspider:
356
357 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
358
359         Reviewed by Maciej Stachowiak.
360
361         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
362         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
363         correct the punctuation in its documentation.
364
365         * sunspider:
366
367 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
368
369         Reviewed by Oliver Hunt.
370
371         Bug 20616: Incorporate V8 benchmarks in testing
372         <https://bugs.webkit.org/show_bug.cgi?id=20616>
373
374         This is a first attempt at running the V8 benchmarks within SunSpider,
375         with the --v8 option. The number of iterations is fixed for each
376         benchmark, and was chosen to make each of the benchmarks run between
377         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
378
379         * sunspider:
380         * tests/LIST-V8: Added.
381         * tests/v8-crypto.js: Added.
382         * tests/v8-deltablue.js: Added.
383         * tests/v8-earley-boyer.js: Added.
384         * tests/v8-raytrace.js: Added.
385         * tests/v8-richards.js: Added.
386
387 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
388
389         Not reviewed.
390
391         Remove a local change that was accidentally committed in r36034.
392
393         * resources/sunspider-standalone-driver.js:
394
395 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
396
397         Rubber-stamped by Oliver Hunt.
398
399         Make SunSpider work with JavaScript shells that don't take filenames
400         after a '-f' argument.
401
402         * sunspider:
403
404 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
405
406         Reviewed by Timothy.
407
408         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
409
410         * sunspider:
411         * tests/LIST-SQUIRRELFISH: Removed.
412
413 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
414
415         Rubber-stamped by Mark Rowe.
416
417         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
418
419         * sunspider:
420
421 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
422
423         Rubber stamped by Oliver.
424         
425         - add newly working tests to squirrelfish list
426
427         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
428
429 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
430
431         Rubber stamped by Oliver.
432         
433         - move ubench tests properly.
434
435         * tests/ubench: Added.
436         * tests/ubench/function-closure.js: Added.
437         * tests/ubench/function-empty.js: Added.
438         * tests/ubench/function-missing-args.js: Added.
439         * tests/ubench/function-sum.js: Added.
440         * tests/ubench/loop-empty-resolve.js: Added.
441         * tests/ubench/loop-empty.js: Added.
442         * tests/ubench/loop-sum.js: Added.
443
444 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
445
446         Rubber stamped by Oliver.
447
448         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
449
450 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
451
452         Reviewed by Oliver.
453
454         - make --squirrelfish mode use the real harness, but a reduced set of tests
455
456         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
457
458         * sunspider:
459         * tests/LIST-SQUIRRELFISH:
460         * tests/LIST-UBENCH: Added.
461         * tests/squirrelfish: Moved to tests/ubench
462
463 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
464
465         Reviewed by Maciej Stachowiak.
466         
467         Removed an unused test.
468
469         * tests/squirrelfish/loop-resolve.js: Removed.
470
471 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
472
473         Reviewed by Oliver.
474         
475         Add newly runnable tests to --squirrelfish mode.
476
477         * tests/LIST-SQUIRRELFISH:
478
479 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
480
481         Reviewed by Maciej Stachowiak.
482         
483         A little more love for --squirrelfish mode:
484         
485         Fixed a misplaced sort. Results properly sort now.
486         
487         Added a test list just for squirrelfish, and updated it to include
488         all currently passing SunSpider tests.
489         
490         Fixed the pruning regexp to match 3d-morph.js.
491
492         * sunspider:
493
494 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
495
496         Reviewed by Oliver Hunt.
497         
498         --squirrelfish mode: pared down tests for squirrelfish to chew on.
499
500         * sunspider:
501         * tests/LIST:
502         * tests/squirrelfish: Added.
503         * tests/squirrelfish/function-closure.js: Added.
504         * tests/squirrelfish/function-empty.js: Added.
505         * tests/squirrelfish/function-missing-args.js: Added.
506         * tests/squirrelfish/function-sum.js: Added.
507         * tests/squirrelfish/loop-empty-resolve.js: Added.
508         * tests/squirrelfish/loop-empty.js: Added.
509         * tests/squirrelfish/loop-resolve.js: Added.
510         * tests/squirrelfish/loop-sum.js: Added.
511
512 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
513
514         Reviewed by Geoff.
515
516         * sunspider: Pause update daemon on Darwin to make results reliable
517         again (workaround for <rdar://problem/5811127>).
518
519 2008-02-07  Eric Seidel  <eric@webkit.org>
520
521         Reviewed by Sam.
522
523         * sunspider: add an error message when ./tests/LIST is not found
524
525 2007-12-29  Alp Toker  <alp@atoker.com>
526
527         Fix typo.
528
529         * hosted/sunspider.html:
530
531 2007-12-29  Alp Toker  <alp@atoker.com>
532
533         Reviewed by Maciej.
534
535         Provide the current URL in the SunSpider results page. Useful for
536         copying benchmark results out of browsers without a location bar or
537         where the location bar can't handle long URLs.
538
539         * hosted/sunspider-results.html:
540
541 2007-12-20  Eric Seidel  <eric@webkit.org>
542
543         Reviewed by mjs.
544
545         * sunspider: increase default number of runs for more accurate numbers
546
547 2007-12-18  Mark Rowe  <mrowe@apple.com>
548
549         Fix typo.
550
551         * hosted/sunspider.html:
552
553 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
554
555         Reviewed by Eric.
556         
557         - give an extra digit of precision for small differences.
558
559         * resources/sunspider-compare-results.js:
560
561 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
562
563         Reviewed by Eric.
564         
565         - work around IE layout bugs
566
567         * hosted/sunspider.css:
568
569 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
570
571         Reviewed by Mark.
572         
573         - fix compare feature
574
575         * hosted/sunspider-results.html:
576
577 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
578
579         Reviewed by Mark.
580         
581         - Change the format of the output URL to be shorter, to meet IE constraints
582
583         * hosted/sunspider-driver.html:
584         * hosted/sunspider-results.html:
585
586 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
587
588         Reviewed by Oliver.
589         
590         - Add some explanatory test and a dumb logo to SunSpider web hosted version
591
592         * hosted/sunspider-driver.html:
593         * hosted/sunspider-results.html:
594         * hosted/sunspider.css:
595         * hosted/sunspider.html:
596
597 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
598
599         Reviewed by Oliver.
600         
601         - Make the web-hosted version of SunSpider look a little bit less drab.
602
603         * hosted/sunspider-driver.html:
604         * hosted/sunspider-results.html:
605         * hosted/sunspider.css: Added.
606         * hosted/sunspider.html:
607         * resources/TEMPLATE.html:
608
609 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
610
611         Reviewed by Sam.
612         
613         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
614         
615         A sample of what new-style results would look like:
616         
617         TEST                   COMPARISON            FROM                 TO             DETAILS
618         ...
619         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
620         ...
621             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
622         
623         * resources/sunspider-compare-results.js:
624
625 2007-11-14  Eric Seidel  <eric@webkit.org>
626
627         Reviewed by Sam.
628
629         * sunspider: add --shark-cache for L2 Cache Miss profiling
630
631 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
632
633         Reviewed by Darin Adler.
634
635         http://bugs.webkit.org/show_bug.cgi?id=15925
636         SunSpider should check for Shark being installed
637
638         * sunspider: Explicitly test that shark command line tool is installed.
639
640 2007-11-04  Darin Adler  <darin@apple.com>
641
642         * hosted/sunspider.html: Tweaked formatting a bit.
643
644 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
645
646         Reviewed by Darin Adler.
647
648         - Add compare mode and a description of the benchmark to the hosted version.
649
650         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
651         * hosted/sunspider.html: Added intro text.
652         * make-hosted: Copy new file.
653         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
654         a function.
655         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
656         expects.
657         * sunspider-compare-results: Use sunspider-standalone-compare.js.
658
659 2007-11-02  Darin Adler  <darin@apple.com>
660
661         * sunspider: Changed "--base" to "--set-baseline".
662
663 2007-11-02  Darin Adler  <darin@apple.com>
664
665         Reviewed by Maciej.
666
667         * sunspider: Added a "--base" option that records the current test as a baseline.
668         Moved the code to find the newest .mshark file into a function (since I needed to use
669         it in the other script and I prefer to copy/paste a function).
670
671         * sunspider-compare-results: Added logic to compare the baseline with the last
672         run when you don't pass any parameters.
673
674 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
675
676         Reviewed by Eric.
677         
678         - Added list of ECMAScript3 features that SunSpider does not cover currently.
679
680         * UNCOVERED: Added.
681
682 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
683
684         Reviewed by Oliver.
685         
686         - Give better error messages when sunspider-compare-results fails
687
688         * sunspider-compare-results:
689
690 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
691
692         Reviewed by Darin Adler.
693         
694         - fix Date tests to work in command-line SpiderMonkey
695
696         * tests/date-format-tofte.js:
697         * tests/date-format-xparb.js:
698
699 2007-10-29  Darin Adler  <darin@apple.com>
700
701         Reviewed by Maciej.
702
703         * sunspider: Added --shark20 option, to run Shark at its highest sample
704         resolution instead of its default.
705
706 2007-10-28  Adam Roben  <aroben@apple.com>
707
708         Fix SunSpider on Windows
709
710         Reviewed by Eric.
711
712         * sunspider: Don't use colons in filenames.
713
714 2007-10-28  Darin Adler  <darin@apple.com>
715
716         Reviewed by Adam.
717
718         * resources/sunspider-analyze-results.js: Tweak the output format a little.
719         Change so that when there's only one run we don't write out confidence
720         intervals at all rather than writing "NaN" over and over again.
721
722         * sunspider: Use the time and date as a suffix on the results file, that way
723         all the results are left behind in the tmp directory. This will make it easy
724         for us to add features that compare past results in the future. Also rename
725         the .mshark files using the same scheme.
726
727         * sunspider-compare-results: Relax the parsing rules so we can parse the
728         results file exactly as it's emitted from sunspider.
729
730 2007-10-25  Adam Roben  <aroben@apple.com>
731
732         Strip carriage returns from results
733
734         Carriage returns were screwing up sunspider-compare-results on
735         Windows.
736
737         Rubberstamped by Oliver.
738
739         * sunspider:
740
741 2007-10-24  Darin Adler  <darin@apple.com>
742
743         Reviewed by Eric.
744
745         * sunspider: Open the profile in shark after completing a run.
746         * sunspider-compare-results: Tiny perl tweaks.
747
748 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
749
750         Reviewed by Darin Adler.
751
752         - Don't hardcode my path to testkjs
753         
754         * sunspider-compare-results:
755
756 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
757
758         Reviewed by Darin Adler.
759         
760         - Add a compare script to compare results, and improve formatting of standard results a bit.
761
762         * resources/sunspider-analyze-results.js:
763         * resources/sunspider-compare-results.js: Added.
764         * sunspider-compare-results: Added.
765
766 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
767
768         Reviewed by Darin Adler.
769
770         * resources/sunspider-analyze-results.js: Do the error range
771         properly, using the t-distribution instead of 1.96 (which was
772         based on the normal distribution).
773         
774         * sunspider: Print results in a way that will be friendlier to the
775         soon-to-come compare mode.
776
777 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
778
779         Reviewed by Review.
780         
781         - add more tests (probably the final set for now)
782         
783         Already balanced. Added date, regexp, control flow, and a few
784         more string and object/array access tests.
785         
786         * tests/LIST:
787         * tests/access-fannkuch.js: Added.
788         * tests/access-nbody.js: Added.
789         * tests/controlflow-recursive.js: Added.
790         * tests/date-format-tofte.js: Added.
791         * tests/date-format-xparb.js: Added.
792         * tests/regexp-dna.js: Added.
793         * tests/string-validate-input.js: Added.
794
795 2007-10-21  Eric Seidel  <eric@webkit.org>
796
797         Reviewed by mjs.
798
799         * sunspider: make --shark only sample from testkjs
800
801 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
802
803         Rubber stamped by Eric.
804         
805         - added crypto tests (already balanced)
806
807         * tests/LIST:
808         * tests/crypto-aes.js: Added.
809         * tests/crypto-md5.js: Added.
810         * tests/crypto-sha1.js: Added.
811
812 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
813
814         Reviewed by Adam.
815         
816         - rebalance new tests
817         
818         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
819         
820         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
821
822         This balances them with the existing tests.
823
824         * tests/access-binary-trees.js:
825         * tests/access-nsieve.js:
826         * tests/bitops-nsieve-bits.js:
827         * tests/math-partial-sums.js:
828         * tests/math-spectral-norm.js:
829         * tests/string-fasta.js:
830
831 2007-10-20  Eric Seidel  <eric@webkit.org>
832
833         Reviewed by darin.
834         
835         Add improved argument handling to sunspider, including
836         --runs=<number>, --tests=<pattern, --shark, and --help
837         Also re-factor code into subroutines
838
839         * sunspider: improved argument handling
840
841 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
842
843         Reviewed by Mark.
844
845         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
846         
847         * TODO:
848         * tests/LIST:
849         * tests/access-binary-trees.js: Added.
850         * tests/access-nsieve.js: Added.
851         * tests/bitops-nsieve-bits.js: Added.
852         * tests/math-partial-sums.js: Added.
853         * tests/math-spectral-norm.js: Added.
854         * tests/string-fasta.js: Added.
855
856 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
857
858         Reviewed by Darin Adler.
859
860         * TODO: Updated to note areas that are now well-covered.
861         
862         - Change the tests and the driver to avoid leaving large object
863         graphs hanging around, since that throws off the subsequent tests.
864         
865         * resources/sunspider-standalone-driver.js:
866         * tests/3d-cube.js:
867         * tests/3d-morph.js:
868         * tests/3d-raytrace.js:
869         * tests/bitops-bitwise-and.js:
870         * tests/string-base64.js:
871         * tests/string-tagcloud.js:
872
873 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
874
875         Reviewed by Nikolas.
876         
877         - Rebalance test complexity.
878         
879         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
880         
881         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
882
883         However, for some tests, IE7 was a huge outlier compared to other
884         major browsers. For those tests, I used Firefox 2.0.0.8 on the
885         same OS and hardware as a normalization basis.
886         
887         * tests/3d-morph.js:
888         * tests/3d-raytrace.js:
889         * tests/bitops-3bit-bits-in-byte.js:
890         * tests/bitops-bits-in-byte.js:
891         * tests/bitops-bitwise-and.js:
892         * tests/math-cordic.js:
893         * tests/string-base64.js:
894         * tests/string-tagcloud.js:
895         * tests/string-unpack-code.js:
896
897 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
898
899         Reviewed by Oliver.
900         
901         - Fix the browser-hosted driver to be compatible with Safari 2
902
903         * hosted/sunspider-driver.html:
904
905 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
906
907         Rubber stamped by Hyatt.
908         
909         - Add BSD license notice to cordic test, the original author said to do so.
910
911         * tests/math-cordic.js:
912
913 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
914
915         Reviewed by Eric.
916         
917         - Support Windows and Windows browsers.
918
919         * hosted/sunspider-results.html: Display results correctly in IE.
920         * sunspider: tweaks to work on Windows, and with spaces in the path.
921
922 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
923
924         Rubber stamped by Oliver.
925
926         * tests/string-tagcloud.js: Remove stray print()
927
928 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
929
930         Reviewed by Oliver.
931         
932         Add some string test cases.
933
934         * tests/string-base64.js: Added. Does base64 encoding and decoding.
935         * tests/string-tagcloud.js: Added. Parses a JSON string and
936         generates tagcloud HTML markup based on the data.        
937         * tests/string-unpack-code.js: Added. Unpack compressed versions of
938         four of the most popular JavaScript libraries.
939         * tests/LIST: Added the new tests.
940
941 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
942
943         Reviewed by Anders.
944
945         * TODO: Updated with info on how to do the stats properly.
946
947         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
948         avoid overly slowing down the benchmark.
949
950         Added some 3D-related test cases.
951         
952         * tests/3d-cube.js: Added.
953         * tests/3d-morph.js: Added.
954         * tests/3d-raytrace.js: Added.
955         * tests/LIST:
956
957 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
958
959         Reviewed by Darin Adler.
960
961         - New JavaScript benchmark
962         http://bugs.webkit.org/show_bug.cgi?id=15515
963         
964         A start on a new JavaScript benchmark, with standalone and
965         browser-hosted drivers.
966         
967         The test content is still incomplete and the drivers could use
968         some more features but there is enough here to make a good start.
969         
970         The drivers compute a 95% confidence interval on the mean for the
971         whole test, each category, and each individual test to make it
972         easier to tell whether differences are statistically
973         significant. The confidence interval can be narrowed by running
974         the test more times.
975         
976         * TODO: Added. Remaining things that need to be done.
977         * sunspider: Added. Perl script that acts as the standalone test driver.
978         * resources/sunspider-standalone-driver.js: Added. JavaScript part
979         of standalone test driver.
980         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
981         analysis code.
982         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
983         * make-hosted: Added. Script to generate browser-hosted tests.
984         * hosted/sunspider.html: Added. Start page for browser-hosted test.
985         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
986         * hosted/sunspider-results.html: Added. Results page for browser-hosted
987         tests.
988         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
989         tests.
990         * tests/LIST: Added. List of tests to use.
991         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
992         * tests/bitops-bits-in-byte.js: Added. Ditto.
993         * tests/bitops-bitwise-and.js: Added. Ditto.
994         * tests/math-cordic.js: Added. Ditto.
995