2010-02-23 Arno Renevier <arno@renevier.net>
[WebKit-https.git] / SunSpider / ChangeLog
1 2010-02-20  Maciej Stachowiak  <mjs@apple.com>
2
3         Reviewed by Adam Barth.
4
5         Add sunspider-0.9.1 to the site in preparation for announcing it.
6         https://bugs.webkit.org/show_bug.cgi?id=35206
7
8         * hosted/sunspider.css: Remove tabs, which I noticed where present while
9         copying to the WebKitSite directory.
10
11 2010-01-11  Oliver Hunt  <oliver@apple.com>
12
13         Reviewed by Darin Adler.
14
15         Sunspider parse-only tests don't run
16         https://bugs.webkit.org/show_bug.cgi?id=33489
17
18         Fix parse-only test list
19
20         * tests/parse-only/LIST:
21
22 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
23
24         Reviewed by Adele Peterson.
25
26         Make SunSpider version more prominent in the title
27         https://bugs.webkit.org/show_bug.cgi?id=32574
28
29         * hosted/sunspider.html:
30         * hosted/versions.html:
31         * resources/driver-TEMPLATE.html:
32         * resources/results-TEMPLATE.html:
33
34 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
35
36         Reviewed by Darin Adler.
37
38         Make SunSpider refuse to compare results across content versions
39         https://bugs.webkit.org/show_bug.cgi?id=32573
40         
41         The results URL now includes a version indicator, and compare mode
42         will refuse to compare between versions.
43
44         * resources/driver-TEMPLATE.html:
45         * resources/results-TEMPLATE.html:
46
47 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
48
49         Reviewed by Darin Adler.
50
51         Some Browser-hosted SunSpider files are not valid HTML5
52         https://bugs.webkit.org/show_bug.cgi?id=32536
53         
54         Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
55         close tags and adding a meta charset declaration).
56
57         * hosted/sunspider.html:
58         * hosted/versions.html:
59         * resources/TEMPLATE.html:
60         * resources/driver-TEMPLATE.html:
61         * resources/results-TEMPLATE.html:
62
63 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
64
65         Reviewed by Eric Seidel.
66
67         Make sunspider-0.9.1 the default content set (both command-line and hosted)
68         https://bugs.webkit.org/show_bug.cgi?id=32537
69
70         * hosted/sunspider.html: Change default suite link.
71         * sunspider: Change default suite.
72         * sunspider-compare-results: Likewise.
73
74 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
75
76         Reviewed by Sam Weinig.
77
78         in-browser SunSpider suffers excessive penalty under power management
79         https://bugs.webkit.org/show_bug.cgi?id=32505
80         
81         I have made a few changes to address this:
82         
83         1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
84         document.write to reduce triggering of progress UI.
85         2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
86         stepped all the way down the whole time.
87         3) Run the test cycle an extra time for warmup before the runs that count.
88
89         * hosted/sunspider-record-result.js: Removed.
90         * make-hosted:
91         * resources/TEMPLATE.html:
92         * resources/driver-TEMPLATE.html:
93
94 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
95
96         Reviewed by Gavin Barraclough.
97
98         SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
99         https://bugs.webkit.org/show_bug.cgi?id=16806
100
101         Based on a patch by Eric Seidel.
102         
103         Fix the base64 computation to actually compute correct results. The impact on runtime of
104         the test is pretty small, but noticeable for some browsers. But at least it's not
105         doing a wrong and meaningless computation any more.
106         
107         * tests/sunspider-0.9.1/string-base64.js:
108         ():
109         (base64ToString):
110
111 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
112
113         Fixing commit error...
114         
115         I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
116
117         * resources/driver-TEMPLATE.html:
118
119 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
120
121         Reviewed by Sam Weinig.
122
123         Use JSON.parse instead of eval for Web-hosted SunSpider results processing
124         https://bugs.webkit.org/show_bug.cgi?id=32490
125
126         * hosted/json2.js: Added. Incorporated from json.org
127         * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
128         * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
129         results.
130
131 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
132
133         Reviewed by Oliver Hunt.
134
135         Enable Web-hosted version of SunSpider to handle multiple versions
136         https://bugs.webkit.org/show_bug.cgi?id=32478
137
138         * make-hosted: Now generate the hosted version to be able to run both the
139         0.9 and the 0.9.1 test suites.
140         * hosted: Modified property svn:ignore.
141         * hosted/sunspider-driver.html: Removed.
142         * hosted/sunspider-results.html: Removed.
143         * hosted/sunspider.html:
144         * hosted/versions.html: Added.
145         * resources/TEMPLATE.html:
146         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
147         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
148
149 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
150
151         Reviewed by Oliver Hunt.
152
153         Give command-line sunspider the ability to handle multiple suites and versions
154         https://bugs.webkit.org/show_bug.cgi?id=32477
155         
156         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
157         enumerating the tests. Current suites are:
158         
159         sunspider-0.9 - riginal SunSpider content 
160         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
161         ubench - "ubench" microbenchmark suite
162         v8-v4 - version 4 of the v8 benchmark
163         parse-only - some parser tests
164         
165         Sunspider can now be run with the --suite= parameter to select a
166         suite. The default is --suite=sunspider-0.9.
167
168         * .: Modified property svn:ignore to ignore new results directories.
169         * make-hosted: Generate based on sunspider-0.9 suite.
170         * resources/TEMPLATE.html: Fix an HTML compliance problem.
171         * resources/sunspider-standalone-driver.js: Become aware of suite names.
172         * sunspider: Modified to handle multiple suites as described above.
173         * sunspider-compare-results: ditto
174         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
175         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
176         * tests/LIST-UBENCH: Removed.
177         * tests/LIST-V8: Removed.
178         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
179         * tests/3d-morph.js: ditto
180         * tests/3d-raytrace.js: ditto
181         * tests/access-binary-trees.js: ditto
182         * tests/access-fannkuch.js: ditto
183         * tests/access-nbody.js: ditto
184         * tests/access-nsieve.js: ditto
185         * tests/bitops-3bit-bits-in-byte.js: ditto
186         * tests/bitops-bits-in-byte.js: ditto
187         * tests/bitops-bitwise-and.js: ditto
188         * tests/bitops-nsieve-bits.js: ditto
189         * tests/controlflow-recursive.js: ditto
190         * tests/crypto-aes.js: ditto
191         * tests/crypto-md5.js: ditto
192         * tests/crypto-sha1.js: ditto
193         * tests/date-format-tofte.js: ditto
194         * tests/date-format-xparb.js: ditto
195         * tests/math-cordic.js: ditto
196         * tests/math-partial-sums.js: ditto
197         * tests/math-spectral-norm.js: ditto
198         * tests/regexp-dna.js: ditto
199         * tests/string-base64.js: ditto
200         * tests/string-fasta.js: ditto
201         * tests/string-tagcloud.js: ditto
202         * tests/string-unpack-code.js: ditto
203         * tests/string-validate-input.js: ditto
204         * tests/sunspider-0.9/: Added
205         * tests/sunspider-0.9/*.js: Moved from one level up
206         * tests/sunspider-0.9.1: Added.
207         * tests/sunspider-0.9.1/*.js: Moved from one level up.
208         * tests/v8-crypto.js: Moved to v8-v4/ directory.
209         * tests/v8-deltablue.js: ditto
210         * tests/v8-earley-boyer.js: ditto
211         * tests/v8-raytrace.js: ditto
212         * tests/v8-regexp.js: ditto
213         * tests/v8-richards.js: ditto
214         * tests/v8-splay.js: ditto
215         * tests/v8-v4: Added.
216
217 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
218
219         Reviewed by Darin Adler.
220
221         Platform specific null device replaced with a platform independent.
222         https://bugs.webkit.org/show_bug.cgi?id=29544
223
224         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
225
226 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
227
228         Reviewed by Maciej Stachowiak.
229
230         [Qt] Make sunspider script work on Windows platform.
231         https://bugs.webkit.org/show_bug.cgi?id=29544
232
233         * sunspider:
234         Using platform specific null device instead of /dev/null.
235
236 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
237
238         Reviewed by Oliver Hunt.
239
240         https://bugs.webkit.org/show_bug.cgi?id=26540
241         Currently the SunSpider test driver lacks an option to run a test suite that
242         will test JavaScriptCore parsing performance only.  This patch adds just such
243         a test suite and option to SunSpider as well as the jsc test shell.  I've included
244         three large javascript source files found in the wild: jquery, mootools and prototype.
245         Combined with the concatenation of all three, these form a new testsuite to measure
246         and test pure JavaScriptCore parsing performance.
247
248         * resources/sunspider-standalone-driver.js:
249           Load the js with parse-only flag if it is part of the parse-only suite.
250         * sunspider:
251           Add the parse-only suite to the list of options.
252         * sunspider-compare-results:
253           Add the parse-only suite to the list of options.
254         * tests/LIST-PARSE-ONLY: Added.
255         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
256           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
257         * tests/parse-only/jquery-1.3.2.js: Added.
258           Latest version from jquery.com.
259         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
260           Latest version from mootools.net.
261         * tests/parse-only/prototype-1.6.0.3.js: Added.
262           Latest version from prototypejs.org.
263
264 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
265
266         Update V8 benchmark to version 4.
267         
268         Reviewed by Cameron Zwarich.
269
270         * tests/LIST-V8:
271         * tests/v8-crypto.js:
272         * tests/v8-deltablue.js:
273         * tests/v8-earley-boyer.js:
274         * tests/v8-raytrace.js:
275         * tests/v8-regexp.js: Added.
276         * tests/v8-richards.js:
277         * tests/v8-splay.js: Added.
278
279 2009-05-26  Olivier DOLE  <odole@pleyo.com>
280
281         Reviewed by Maciej.
282
283         Store sunspider test suite results in a specific directory for each
284         different type of benchmark (sunspider, V8 or ubench).
285
286         * sunspider:
287         * sunspider-compare-results:
288
289 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
290
291         Build fix.
292
293         * SunSpider.make:
294
295 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
296
297         Include SunSpider tests in production builds.
298         
299         Reviewed by Adam Roben.
300
301         * SunSpider.make: Added.
302         * xcopy.excludes: Added.
303
304 2009-03-04  Oliver Hunt  <oliver@apple.com>
305
306         Added a few more tests to ubench
307
308         RS=Gavin Barraclough
309
310         Adding a couple of test to cover too few and too many arguments.  This
311         is needed to prevent regressions in all call paths from my current refactoring.
312
313         * tests/LIST-UBENCH:
314         * tests/ubench/function-correct-args.js: Added.
315         (f):
316         * tests/ubench/function-excess-args.js: Added.
317         (f):
318
319 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
320
321         Fix a bug where when V8's or UBENCH's test suite list file is missing
322         the error message is suggesting that SunSpider's list file is missing.
323
324         Reviewed by Sam Weinig.
325
326         * sunspider:
327         (loadTestsList): Fix the error message to use the right list name.
328
329 2008-10-07  David Hyatt  <hyatt@apple.com>
330
331         https://bugs.webkit.org/show_bug.cgi?id=18989
332
333         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
334         we want all browsers to be doing the same thing on the test.
335
336         Reviewed by Maciej
337
338         * tests/regexp-dna.js:
339
340 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
341
342         Rubber-stamped by Maciej Stachowiak.
343
344         The bleeding edge branch of V8 now supports passing fileanmes after
345         '-f', so the SunSpider script should be changed to do this again.
346         This allows all 3 major JavaScript engines to be used with the
347         unmodified SunSpider script.
348
349         * sunspider:
350
351 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
352
353         Reviewed by Maciej Stachowiak.
354
355         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
356         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
357         correct the punctuation in its documentation.
358
359         * sunspider:
360
361 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
362
363         Reviewed by Oliver Hunt.
364
365         Bug 20616: Incorporate V8 benchmarks in testing
366         <https://bugs.webkit.org/show_bug.cgi?id=20616>
367
368         This is a first attempt at running the V8 benchmarks within SunSpider,
369         with the --v8 option. The number of iterations is fixed for each
370         benchmark, and was chosen to make each of the benchmarks run between
371         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
372
373         * sunspider:
374         * tests/LIST-V8: Added.
375         * tests/v8-crypto.js: Added.
376         * tests/v8-deltablue.js: Added.
377         * tests/v8-earley-boyer.js: Added.
378         * tests/v8-raytrace.js: Added.
379         * tests/v8-richards.js: Added.
380
381 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
382
383         Not reviewed.
384
385         Remove a local change that was accidentally committed in r36034.
386
387         * resources/sunspider-standalone-driver.js:
388
389 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
390
391         Rubber-stamped by Oliver Hunt.
392
393         Make SunSpider work with JavaScript shells that don't take filenames
394         after a '-f' argument.
395
396         * sunspider:
397
398 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
399
400         Reviewed by Timothy.
401
402         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
403
404         * sunspider:
405         * tests/LIST-SQUIRRELFISH: Removed.
406
407 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
408
409         Rubber-stamped by Mark Rowe.
410
411         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
412
413         * sunspider:
414
415 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
416
417         Rubber stamped by Oliver.
418         
419         - add newly working tests to squirrelfish list
420
421         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
422
423 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
424
425         Rubber stamped by Oliver.
426         
427         - move ubench tests properly.
428
429         * tests/ubench: Added.
430         * tests/ubench/function-closure.js: Added.
431         * tests/ubench/function-empty.js: Added.
432         * tests/ubench/function-missing-args.js: Added.
433         * tests/ubench/function-sum.js: Added.
434         * tests/ubench/loop-empty-resolve.js: Added.
435         * tests/ubench/loop-empty.js: Added.
436         * tests/ubench/loop-sum.js: Added.
437
438 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
439
440         Rubber stamped by Oliver.
441
442         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
443
444 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
445
446         Reviewed by Oliver.
447
448         - make --squirrelfish mode use the real harness, but a reduced set of tests
449
450         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
451
452         * sunspider:
453         * tests/LIST-SQUIRRELFISH:
454         * tests/LIST-UBENCH: Added.
455         * tests/squirrelfish: Moved to tests/ubench
456
457 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
458
459         Reviewed by Maciej Stachowiak.
460         
461         Removed an unused test.
462
463         * tests/squirrelfish/loop-resolve.js: Removed.
464
465 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
466
467         Reviewed by Oliver.
468         
469         Add newly runnable tests to --squirrelfish mode.
470
471         * tests/LIST-SQUIRRELFISH:
472
473 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
474
475         Reviewed by Maciej Stachowiak.
476         
477         A little more love for --squirrelfish mode:
478         
479         Fixed a misplaced sort. Results properly sort now.
480         
481         Added a test list just for squirrelfish, and updated it to include
482         all currently passing SunSpider tests.
483         
484         Fixed the pruning regexp to match 3d-morph.js.
485
486         * sunspider:
487
488 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
489
490         Reviewed by Oliver Hunt.
491         
492         --squirrelfish mode: pared down tests for squirrelfish to chew on.
493
494         * sunspider:
495         * tests/LIST:
496         * tests/squirrelfish: Added.
497         * tests/squirrelfish/function-closure.js: Added.
498         * tests/squirrelfish/function-empty.js: Added.
499         * tests/squirrelfish/function-missing-args.js: Added.
500         * tests/squirrelfish/function-sum.js: Added.
501         * tests/squirrelfish/loop-empty-resolve.js: Added.
502         * tests/squirrelfish/loop-empty.js: Added.
503         * tests/squirrelfish/loop-resolve.js: Added.
504         * tests/squirrelfish/loop-sum.js: Added.
505
506 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
507
508         Reviewed by Geoff.
509
510         * sunspider: Pause update daemon on Darwin to make results reliable
511         again (workaround for <rdar://problem/5811127>).
512
513 2008-02-07  Eric Seidel  <eric@webkit.org>
514
515         Reviewed by Sam.
516
517         * sunspider: add an error message when ./tests/LIST is not found
518
519 2007-12-29  Alp Toker  <alp@atoker.com>
520
521         Fix typo.
522
523         * hosted/sunspider.html:
524
525 2007-12-29  Alp Toker  <alp@atoker.com>
526
527         Reviewed by Maciej.
528
529         Provide the current URL in the SunSpider results page. Useful for
530         copying benchmark results out of browsers without a location bar or
531         where the location bar can't handle long URLs.
532
533         * hosted/sunspider-results.html:
534
535 2007-12-20  Eric Seidel  <eric@webkit.org>
536
537         Reviewed by mjs.
538
539         * sunspider: increase default number of runs for more accurate numbers
540
541 2007-12-18  Mark Rowe  <mrowe@apple.com>
542
543         Fix typo.
544
545         * hosted/sunspider.html:
546
547 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
548
549         Reviewed by Eric.
550         
551         - give an extra digit of precision for small differences.
552
553         * resources/sunspider-compare-results.js:
554
555 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
556
557         Reviewed by Eric.
558         
559         - work around IE layout bugs
560
561         * hosted/sunspider.css:
562
563 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
564
565         Reviewed by Mark.
566         
567         - fix compare feature
568
569         * hosted/sunspider-results.html:
570
571 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
572
573         Reviewed by Mark.
574         
575         - Change the format of the output URL to be shorter, to meet IE constraints
576
577         * hosted/sunspider-driver.html:
578         * hosted/sunspider-results.html:
579
580 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
581
582         Reviewed by Oliver.
583         
584         - Add some explanatory test and a dumb logo to SunSpider web hosted version
585
586         * hosted/sunspider-driver.html:
587         * hosted/sunspider-results.html:
588         * hosted/sunspider.css:
589         * hosted/sunspider.html:
590
591 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
592
593         Reviewed by Oliver.
594         
595         - Make the web-hosted version of SunSpider look a little bit less drab.
596
597         * hosted/sunspider-driver.html:
598         * hosted/sunspider-results.html:
599         * hosted/sunspider.css: Added.
600         * hosted/sunspider.html:
601         * resources/TEMPLATE.html:
602
603 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
604
605         Reviewed by Sam.
606         
607         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
608         
609         A sample of what new-style results would look like:
610         
611         TEST                   COMPARISON            FROM                 TO             DETAILS
612         ...
613         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
614         ...
615             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
616         
617         * resources/sunspider-compare-results.js:
618
619 2007-11-14  Eric Seidel  <eric@webkit.org>
620
621         Reviewed by Sam.
622
623         * sunspider: add --shark-cache for L2 Cache Miss profiling
624
625 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
626
627         Reviewed by Darin Adler.
628
629         http://bugs.webkit.org/show_bug.cgi?id=15925
630         SunSpider should check for Shark being installed
631
632         * sunspider: Explicitly test that shark command line tool is installed.
633
634 2007-11-04  Darin Adler  <darin@apple.com>
635
636         * hosted/sunspider.html: Tweaked formatting a bit.
637
638 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
639
640         Reviewed by Darin Adler.
641
642         - Add compare mode and a description of the benchmark to the hosted version.
643
644         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
645         * hosted/sunspider.html: Added intro text.
646         * make-hosted: Copy new file.
647         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
648         a function.
649         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
650         expects.
651         * sunspider-compare-results: Use sunspider-standalone-compare.js.
652
653 2007-11-02  Darin Adler  <darin@apple.com>
654
655         * sunspider: Changed "--base" to "--set-baseline".
656
657 2007-11-02  Darin Adler  <darin@apple.com>
658
659         Reviewed by Maciej.
660
661         * sunspider: Added a "--base" option that records the current test as a baseline.
662         Moved the code to find the newest .mshark file into a function (since I needed to use
663         it in the other script and I prefer to copy/paste a function).
664
665         * sunspider-compare-results: Added logic to compare the baseline with the last
666         run when you don't pass any parameters.
667
668 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
669
670         Reviewed by Eric.
671         
672         - Added list of ECMAScript3 features that SunSpider does not cover currently.
673
674         * UNCOVERED: Added.
675
676 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
677
678         Reviewed by Oliver.
679         
680         - Give better error messages when sunspider-compare-results fails
681
682         * sunspider-compare-results:
683
684 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
685
686         Reviewed by Darin Adler.
687         
688         - fix Date tests to work in command-line SpiderMonkey
689
690         * tests/date-format-tofte.js:
691         * tests/date-format-xparb.js:
692
693 2007-10-29  Darin Adler  <darin@apple.com>
694
695         Reviewed by Maciej.
696
697         * sunspider: Added --shark20 option, to run Shark at its highest sample
698         resolution instead of its default.
699
700 2007-10-28  Adam Roben  <aroben@apple.com>
701
702         Fix SunSpider on Windows
703
704         Reviewed by Eric.
705
706         * sunspider: Don't use colons in filenames.
707
708 2007-10-28  Darin Adler  <darin@apple.com>
709
710         Reviewed by Adam.
711
712         * resources/sunspider-analyze-results.js: Tweak the output format a little.
713         Change so that when there's only one run we don't write out confidence
714         intervals at all rather than writing "NaN" over and over again.
715
716         * sunspider: Use the time and date as a suffix on the results file, that way
717         all the results are left behind in the tmp directory. This will make it easy
718         for us to add features that compare past results in the future. Also rename
719         the .mshark files using the same scheme.
720
721         * sunspider-compare-results: Relax the parsing rules so we can parse the
722         results file exactly as it's emitted from sunspider.
723
724 2007-10-25  Adam Roben  <aroben@apple.com>
725
726         Strip carriage returns from results
727
728         Carriage returns were screwing up sunspider-compare-results on
729         Windows.
730
731         Rubberstamped by Oliver.
732
733         * sunspider:
734
735 2007-10-24  Darin Adler  <darin@apple.com>
736
737         Reviewed by Eric.
738
739         * sunspider: Open the profile in shark after completing a run.
740         * sunspider-compare-results: Tiny perl tweaks.
741
742 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
743
744         Reviewed by Darin Adler.
745
746         - Don't hardcode my path to testkjs
747         
748         * sunspider-compare-results:
749
750 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
751
752         Reviewed by Darin Adler.
753         
754         - Add a compare script to compare results, and improve formatting of standard results a bit.
755
756         * resources/sunspider-analyze-results.js:
757         * resources/sunspider-compare-results.js: Added.
758         * sunspider-compare-results: Added.
759
760 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
761
762         Reviewed by Darin Adler.
763
764         * resources/sunspider-analyze-results.js: Do the error range
765         properly, using the t-distribution instead of 1.96 (which was
766         based on the normal distribution).
767         
768         * sunspider: Print results in a way that will be friendlier to the
769         soon-to-come compare mode.
770
771 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
772
773         Reviewed by Review.
774         
775         - add more tests (probably the final set for now)
776         
777         Already balanced. Added date, regexp, control flow, and a few
778         more string and object/array access tests.
779         
780         * tests/LIST:
781         * tests/access-fannkuch.js: Added.
782         * tests/access-nbody.js: Added.
783         * tests/controlflow-recursive.js: Added.
784         * tests/date-format-tofte.js: Added.
785         * tests/date-format-xparb.js: Added.
786         * tests/regexp-dna.js: Added.
787         * tests/string-validate-input.js: Added.
788
789 2007-10-21  Eric Seidel  <eric@webkit.org>
790
791         Reviewed by mjs.
792
793         * sunspider: make --shark only sample from testkjs
794
795 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
796
797         Rubber stamped by Eric.
798         
799         - added crypto tests (already balanced)
800
801         * tests/LIST:
802         * tests/crypto-aes.js: Added.
803         * tests/crypto-md5.js: Added.
804         * tests/crypto-sha1.js: Added.
805
806 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
807
808         Reviewed by Adam.
809         
810         - rebalance new tests
811         
812         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
813         
814         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
815
816         This balances them with the existing tests.
817
818         * tests/access-binary-trees.js:
819         * tests/access-nsieve.js:
820         * tests/bitops-nsieve-bits.js:
821         * tests/math-partial-sums.js:
822         * tests/math-spectral-norm.js:
823         * tests/string-fasta.js:
824
825 2007-10-20  Eric Seidel  <eric@webkit.org>
826
827         Reviewed by darin.
828         
829         Add improved argument handling to sunspider, including
830         --runs=<number>, --tests=<pattern, --shark, and --help
831         Also re-factor code into subroutines
832
833         * sunspider: improved argument handling
834
835 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
836
837         Reviewed by Mark.
838
839         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
840         
841         * TODO:
842         * tests/LIST:
843         * tests/access-binary-trees.js: Added.
844         * tests/access-nsieve.js: Added.
845         * tests/bitops-nsieve-bits.js: Added.
846         * tests/math-partial-sums.js: Added.
847         * tests/math-spectral-norm.js: Added.
848         * tests/string-fasta.js: Added.
849
850 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
851
852         Reviewed by Darin Adler.
853
854         * TODO: Updated to note areas that are now well-covered.
855         
856         - Change the tests and the driver to avoid leaving large object
857         graphs hanging around, since that throws off the subsequent tests.
858         
859         * resources/sunspider-standalone-driver.js:
860         * tests/3d-cube.js:
861         * tests/3d-morph.js:
862         * tests/3d-raytrace.js:
863         * tests/bitops-bitwise-and.js:
864         * tests/string-base64.js:
865         * tests/string-tagcloud.js:
866
867 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
868
869         Reviewed by Nikolas.
870         
871         - Rebalance test complexity.
872         
873         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
874         
875         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
876
877         However, for some tests, IE7 was a huge outlier compared to other
878         major browsers. For those tests, I used Firefox 2.0.0.8 on the
879         same OS and hardware as a normalization basis.
880         
881         * tests/3d-morph.js:
882         * tests/3d-raytrace.js:
883         * tests/bitops-3bit-bits-in-byte.js:
884         * tests/bitops-bits-in-byte.js:
885         * tests/bitops-bitwise-and.js:
886         * tests/math-cordic.js:
887         * tests/string-base64.js:
888         * tests/string-tagcloud.js:
889         * tests/string-unpack-code.js:
890
891 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
892
893         Reviewed by Oliver.
894         
895         - Fix the browser-hosted driver to be compatible with Safari 2
896
897         * hosted/sunspider-driver.html:
898
899 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
900
901         Rubber stamped by Hyatt.
902         
903         - Add BSD license notice to cordic test, the original author said to do so.
904
905         * tests/math-cordic.js:
906
907 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
908
909         Reviewed by Eric.
910         
911         - Support Windows and Windows browsers.
912
913         * hosted/sunspider-results.html: Display results correctly in IE.
914         * sunspider: tweaks to work on Windows, and with spaces in the path.
915
916 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
917
918         Rubber stamped by Oliver.
919
920         * tests/string-tagcloud.js: Remove stray print()
921
922 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
923
924         Reviewed by Oliver.
925         
926         Add some string test cases.
927
928         * tests/string-base64.js: Added. Does base64 encoding and decoding.
929         * tests/string-tagcloud.js: Added. Parses a JSON string and
930         generates tagcloud HTML markup based on the data.        
931         * tests/string-unpack-code.js: Added. Unpack compressed versions of
932         four of the most popular JavaScript libraries.
933         * tests/LIST: Added the new tests.
934
935 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
936
937         Reviewed by Anders.
938
939         * TODO: Updated with info on how to do the stats properly.
940
941         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
942         avoid overly slowing down the benchmark.
943
944         Added some 3D-related test cases.
945         
946         * tests/3d-cube.js: Added.
947         * tests/3d-morph.js: Added.
948         * tests/3d-raytrace.js: Added.
949         * tests/LIST:
950
951 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
952
953         Reviewed by Darin Adler.
954
955         - New JavaScript benchmark
956         http://bugs.webkit.org/show_bug.cgi?id=15515
957         
958         A start on a new JavaScript benchmark, with standalone and
959         browser-hosted drivers.
960         
961         The test content is still incomplete and the drivers could use
962         some more features but there is enough here to make a good start.
963         
964         The drivers compute a 95% confidence interval on the mean for the
965         whole test, each category, and each individual test to make it
966         easier to tell whether differences are statistically
967         significant. The confidence interval can be narrowed by running
968         the test more times.
969         
970         * TODO: Added. Remaining things that need to be done.
971         * sunspider: Added. Perl script that acts as the standalone test driver.
972         * resources/sunspider-standalone-driver.js: Added. JavaScript part
973         of standalone test driver.
974         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
975         analysis code.
976         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
977         * make-hosted: Added. Script to generate browser-hosted tests.
978         * hosted/sunspider.html: Added. Start page for browser-hosted test.
979         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
980         * hosted/sunspider-results.html: Added. Results page for browser-hosted
981         tests.
982         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
983         tests.
984         * tests/LIST: Added. List of tests to use.
985         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
986         * tests/bitops-bits-in-byte.js: Added. Ditto.
987         * tests/bitops-bitwise-and.js: Added. Ditto.
988         * tests/math-cordic.js: Added. Ditto.
989