2009-12-12 Maciej Stachowiak <mjs@apple.com>
[WebKit.git] / SunSpider / ChangeLog
1 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
2
3         Reviewed by Oliver Hunt.
4
5         Enable Web-hosted version of SunSpider to handle multiple versions
6         https://bugs.webkit.org/show_bug.cgi?id=32478
7
8         * make-hosted: Now generate the hosted version to be able to run both the
9         0.9 and the 0.9.1 test suites.
10         * hosted: Modified property svn:ignore.
11         * hosted/sunspider-driver.html: Removed.
12         * hosted/sunspider-results.html: Removed.
13         * hosted/sunspider.html:
14         * hosted/versions.html: Added.
15         * resources/TEMPLATE.html:
16         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
17         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
18
19 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
20
21         Reviewed by Oliver Hunt.
22
23         Give command-line sunspider the ability to handle multiple suites and versions
24         https://bugs.webkit.org/show_bug.cgi?id=32477
25         
26         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
27         enumerating the tests. Current suites are:
28         
29         sunspider-0.9 - riginal SunSpider content 
30         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
31         ubench - "ubench" microbenchmark suite
32         v8-v4 - version 4 of the v8 benchmark
33         parse-only - some parser tests
34         
35         Sunspider can now be run with the --suite= parameter to select a
36         suite. The default is --suite=sunspider-0.9.
37
38         * .: Modified property svn:ignore to ignore new results directories.
39         * make-hosted: Generate based on sunspider-0.9 suite.
40         * resources/TEMPLATE.html: Fix an HTML compliance problem.
41         * resources/sunspider-standalone-driver.js: Become aware of suite names.
42         * sunspider: Modified to handle multiple suites as described above.
43         * sunspider-compare-results: ditto
44         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
45         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
46         * tests/LIST-UBENCH: Removed.
47         * tests/LIST-V8: Removed.
48         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
49         * tests/3d-morph.js: ditto
50         * tests/3d-raytrace.js: ditto
51         * tests/access-binary-trees.js: ditto
52         * tests/access-fannkuch.js: ditto
53         * tests/access-nbody.js: ditto
54         * tests/access-nsieve.js: ditto
55         * tests/bitops-3bit-bits-in-byte.js: ditto
56         * tests/bitops-bits-in-byte.js: ditto
57         * tests/bitops-bitwise-and.js: ditto
58         * tests/bitops-nsieve-bits.js: ditto
59         * tests/controlflow-recursive.js: ditto
60         * tests/crypto-aes.js: ditto
61         * tests/crypto-md5.js: ditto
62         * tests/crypto-sha1.js: ditto
63         * tests/date-format-tofte.js: ditto
64         * tests/date-format-xparb.js: ditto
65         * tests/math-cordic.js: ditto
66         * tests/math-partial-sums.js: ditto
67         * tests/math-spectral-norm.js: ditto
68         * tests/regexp-dna.js: ditto
69         * tests/string-base64.js: ditto
70         * tests/string-fasta.js: ditto
71         * tests/string-tagcloud.js: ditto
72         * tests/string-unpack-code.js: ditto
73         * tests/string-validate-input.js: ditto
74         * tests/sunspider-0.9/: Added
75         * tests/sunspider-0.9/*.js: Moved from one level up
76         * tests/sunspider-0.9.1: Added.
77         * tests/sunspider-0.9.1/*.js: Moved from one level up.
78         * tests/v8-crypto.js: Moved to v8-v4/ directory.
79         * tests/v8-deltablue.js: ditto
80         * tests/v8-earley-boyer.js: ditto
81         * tests/v8-raytrace.js: ditto
82         * tests/v8-regexp.js: ditto
83         * tests/v8-richards.js: ditto
84         * tests/v8-splay.js: ditto
85         * tests/v8-v4: Added.
86
87 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
88
89         Reviewed by Darin Adler.
90
91         Platform specific null device replaced with a platform independent.
92         https://bugs.webkit.org/show_bug.cgi?id=29544
93
94         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
95
96 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
97
98         Reviewed by Maciej Stachowiak.
99
100         [Qt] Make sunspider script work on Windows platform.
101         https://bugs.webkit.org/show_bug.cgi?id=29544
102
103         * sunspider:
104         Using platform specific null device instead of /dev/null.
105
106 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
107
108         Reviewed by Oliver Hunt.
109
110         https://bugs.webkit.org/show_bug.cgi?id=26540
111         Currently the SunSpider test driver lacks an option to run a test suite that
112         will test JavaScriptCore parsing performance only.  This patch adds just such
113         a test suite and option to SunSpider as well as the jsc test shell.  I've included
114         three large javascript source files found in the wild: jquery, mootools and prototype.
115         Combined with the concatenation of all three, these form a new testsuite to measure
116         and test pure JavaScriptCore parsing performance.
117
118         * resources/sunspider-standalone-driver.js:
119           Load the js with parse-only flag if it is part of the parse-only suite.
120         * sunspider:
121           Add the parse-only suite to the list of options.
122         * sunspider-compare-results:
123           Add the parse-only suite to the list of options.
124         * tests/LIST-PARSE-ONLY: Added.
125         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
126           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
127         * tests/parse-only/jquery-1.3.2.js: Added.
128           Latest version from jquery.com.
129         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
130           Latest version from mootools.net.
131         * tests/parse-only/prototype-1.6.0.3.js: Added.
132           Latest version from prototypejs.org.
133
134 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
135
136         Update V8 benchmark to version 4.
137         
138         Reviewed by Cameron Zwarich.
139
140         * tests/LIST-V8:
141         * tests/v8-crypto.js:
142         * tests/v8-deltablue.js:
143         * tests/v8-earley-boyer.js:
144         * tests/v8-raytrace.js:
145         * tests/v8-regexp.js: Added.
146         * tests/v8-richards.js:
147         * tests/v8-splay.js: Added.
148
149 2009-05-26  Olivier DOLE  <odole@pleyo.com>
150
151         Reviewed by Maciej.
152
153         Store sunspider test suite results in a specific directory for each
154         different type of benchmark (sunspider, V8 or ubench).
155
156         * sunspider:
157         * sunspider-compare-results:
158
159 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
160
161         Build fix.
162
163         * SunSpider.make:
164
165 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
166
167         Include SunSpider tests in production builds.
168         
169         Reviewed by Adam Roben.
170
171         * SunSpider.make: Added.
172         * xcopy.excludes: Added.
173
174 2009-03-04  Oliver Hunt  <oliver@apple.com>
175
176         Added a few more tests to ubench
177
178         RS=Gavin Barraclough
179
180         Adding a couple of test to cover too few and too many arguments.  This
181         is needed to prevent regressions in all call paths from my current refactoring.
182
183         * tests/LIST-UBENCH:
184         * tests/ubench/function-correct-args.js: Added.
185         (f):
186         * tests/ubench/function-excess-args.js: Added.
187         (f):
188
189 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
190
191         Fix a bug where when V8's or UBENCH's test suite list file is missing
192         the error message is suggesting that SunSpider's list file is missing.
193
194         Reviewed by Sam Weinig.
195
196         * sunspider:
197         (loadTestsList): Fix the error message to use the right list name.
198
199 2008-10-07  David Hyatt  <hyatt@apple.com>
200
201         https://bugs.webkit.org/show_bug.cgi?id=18989
202
203         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
204         we want all browsers to be doing the same thing on the test.
205
206         Reviewed by Maciej
207
208         * tests/regexp-dna.js:
209
210 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
211
212         Rubber-stamped by Maciej Stachowiak.
213
214         The bleeding edge branch of V8 now supports passing fileanmes after
215         '-f', so the SunSpider script should be changed to do this again.
216         This allows all 3 major JavaScript engines to be used with the
217         unmodified SunSpider script.
218
219         * sunspider:
220
221 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
222
223         Reviewed by Maciej Stachowiak.
224
225         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
226         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
227         correct the punctuation in its documentation.
228
229         * sunspider:
230
231 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
232
233         Reviewed by Oliver Hunt.
234
235         Bug 20616: Incorporate V8 benchmarks in testing
236         <https://bugs.webkit.org/show_bug.cgi?id=20616>
237
238         This is a first attempt at running the V8 benchmarks within SunSpider,
239         with the --v8 option. The number of iterations is fixed for each
240         benchmark, and was chosen to make each of the benchmarks run between
241         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
242
243         * sunspider:
244         * tests/LIST-V8: Added.
245         * tests/v8-crypto.js: Added.
246         * tests/v8-deltablue.js: Added.
247         * tests/v8-earley-boyer.js: Added.
248         * tests/v8-raytrace.js: Added.
249         * tests/v8-richards.js: Added.
250
251 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
252
253         Not reviewed.
254
255         Remove a local change that was accidentally committed in r36034.
256
257         * resources/sunspider-standalone-driver.js:
258
259 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
260
261         Rubber-stamped by Oliver Hunt.
262
263         Make SunSpider work with JavaScript shells that don't take filenames
264         after a '-f' argument.
265
266         * sunspider:
267
268 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
269
270         Reviewed by Timothy.
271
272         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
273
274         * sunspider:
275         * tests/LIST-SQUIRRELFISH: Removed.
276
277 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
278
279         Rubber-stamped by Mark Rowe.
280
281         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
282
283         * sunspider:
284
285 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
286
287         Rubber stamped by Oliver.
288         
289         - add newly working tests to squirrelfish list
290
291         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
292
293 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
294
295         Rubber stamped by Oliver.
296         
297         - move ubench tests properly.
298
299         * tests/ubench: Added.
300         * tests/ubench/function-closure.js: Added.
301         * tests/ubench/function-empty.js: Added.
302         * tests/ubench/function-missing-args.js: Added.
303         * tests/ubench/function-sum.js: Added.
304         * tests/ubench/loop-empty-resolve.js: Added.
305         * tests/ubench/loop-empty.js: Added.
306         * tests/ubench/loop-sum.js: Added.
307
308 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
309
310         Rubber stamped by Oliver.
311
312         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
313
314 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
315
316         Reviewed by Oliver.
317
318         - make --squirrelfish mode use the real harness, but a reduced set of tests
319
320         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
321
322         * sunspider:
323         * tests/LIST-SQUIRRELFISH:
324         * tests/LIST-UBENCH: Added.
325         * tests/squirrelfish: Moved to tests/ubench
326
327 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
328
329         Reviewed by Maciej Stachowiak.
330         
331         Removed an unused test.
332
333         * tests/squirrelfish/loop-resolve.js: Removed.
334
335 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
336
337         Reviewed by Oliver.
338         
339         Add newly runnable tests to --squirrelfish mode.
340
341         * tests/LIST-SQUIRRELFISH:
342
343 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
344
345         Reviewed by Maciej Stachowiak.
346         
347         A little more love for --squirrelfish mode:
348         
349         Fixed a misplaced sort. Results properly sort now.
350         
351         Added a test list just for squirrelfish, and updated it to include
352         all currently passing SunSpider tests.
353         
354         Fixed the pruning regexp to match 3d-morph.js.
355
356         * sunspider:
357
358 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
359
360         Reviewed by Oliver Hunt.
361         
362         --squirrelfish mode: pared down tests for squirrelfish to chew on.
363
364         * sunspider:
365         * tests/LIST:
366         * tests/squirrelfish: Added.
367         * tests/squirrelfish/function-closure.js: Added.
368         * tests/squirrelfish/function-empty.js: Added.
369         * tests/squirrelfish/function-missing-args.js: Added.
370         * tests/squirrelfish/function-sum.js: Added.
371         * tests/squirrelfish/loop-empty-resolve.js: Added.
372         * tests/squirrelfish/loop-empty.js: Added.
373         * tests/squirrelfish/loop-resolve.js: Added.
374         * tests/squirrelfish/loop-sum.js: Added.
375
376 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
377
378         Reviewed by Geoff.
379
380         * sunspider: Pause update daemon on Darwin to make results reliable
381         again (workaround for <rdar://problem/5811127>).
382
383 2008-02-07  Eric Seidel  <eric@webkit.org>
384
385         Reviewed by Sam.
386
387         * sunspider: add an error message when ./tests/LIST is not found
388
389 2007-12-29  Alp Toker  <alp@atoker.com>
390
391         Fix typo.
392
393         * hosted/sunspider.html:
394
395 2007-12-29  Alp Toker  <alp@atoker.com>
396
397         Reviewed by Maciej.
398
399         Provide the current URL in the SunSpider results page. Useful for
400         copying benchmark results out of browsers without a location bar or
401         where the location bar can't handle long URLs.
402
403         * hosted/sunspider-results.html:
404
405 2007-12-20  Eric Seidel  <eric@webkit.org>
406
407         Reviewed by mjs.
408
409         * sunspider: increase default number of runs for more accurate numbers
410
411 2007-12-18  Mark Rowe  <mrowe@apple.com>
412
413         Fix typo.
414
415         * hosted/sunspider.html:
416
417 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
418
419         Reviewed by Eric.
420         
421         - give an extra digit of precision for small differences.
422
423         * resources/sunspider-compare-results.js:
424
425 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
426
427         Reviewed by Eric.
428         
429         - work around IE layout bugs
430
431         * hosted/sunspider.css:
432
433 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
434
435         Reviewed by Mark.
436         
437         - fix compare feature
438
439         * hosted/sunspider-results.html:
440
441 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
442
443         Reviewed by Mark.
444         
445         - Change the format of the output URL to be shorter, to meet IE constraints
446
447         * hosted/sunspider-driver.html:
448         * hosted/sunspider-results.html:
449
450 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
451
452         Reviewed by Oliver.
453         
454         - Add some explanatory test and a dumb logo to SunSpider web hosted version
455
456         * hosted/sunspider-driver.html:
457         * hosted/sunspider-results.html:
458         * hosted/sunspider.css:
459         * hosted/sunspider.html:
460
461 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
462
463         Reviewed by Oliver.
464         
465         - Make the web-hosted version of SunSpider look a little bit less drab.
466
467         * hosted/sunspider-driver.html:
468         * hosted/sunspider-results.html:
469         * hosted/sunspider.css: Added.
470         * hosted/sunspider.html:
471         * resources/TEMPLATE.html:
472
473 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
474
475         Reviewed by Sam.
476         
477         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
478         
479         A sample of what new-style results would look like:
480         
481         TEST                   COMPARISON            FROM                 TO             DETAILS
482         ...
483         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
484         ...
485             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
486         
487         * resources/sunspider-compare-results.js:
488
489 2007-11-14  Eric Seidel  <eric@webkit.org>
490
491         Reviewed by Sam.
492
493         * sunspider: add --shark-cache for L2 Cache Miss profiling
494
495 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
496
497         Reviewed by Darin Adler.
498
499         http://bugs.webkit.org/show_bug.cgi?id=15925
500         SunSpider should check for Shark being installed
501
502         * sunspider: Explicitly test that shark command line tool is installed.
503
504 2007-11-04  Darin Adler  <darin@apple.com>
505
506         * hosted/sunspider.html: Tweaked formatting a bit.
507
508 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
509
510         Reviewed by Darin Adler.
511
512         - Add compare mode and a description of the benchmark to the hosted version.
513
514         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
515         * hosted/sunspider.html: Added intro text.
516         * make-hosted: Copy new file.
517         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
518         a function.
519         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
520         expects.
521         * sunspider-compare-results: Use sunspider-standalone-compare.js.
522
523 2007-11-02  Darin Adler  <darin@apple.com>
524
525         * sunspider: Changed "--base" to "--set-baseline".
526
527 2007-11-02  Darin Adler  <darin@apple.com>
528
529         Reviewed by Maciej.
530
531         * sunspider: Added a "--base" option that records the current test as a baseline.
532         Moved the code to find the newest .mshark file into a function (since I needed to use
533         it in the other script and I prefer to copy/paste a function).
534
535         * sunspider-compare-results: Added logic to compare the baseline with the last
536         run when you don't pass any parameters.
537
538 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
539
540         Reviewed by Eric.
541         
542         - Added list of ECMAScript3 features that SunSpider does not cover currently.
543
544         * UNCOVERED: Added.
545
546 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
547
548         Reviewed by Oliver.
549         
550         - Give better error messages when sunspider-compare-results fails
551
552         * sunspider-compare-results:
553
554 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
555
556         Reviewed by Darin Adler.
557         
558         - fix Date tests to work in command-line SpiderMonkey
559
560         * tests/date-format-tofte.js:
561         * tests/date-format-xparb.js:
562
563 2007-10-29  Darin Adler  <darin@apple.com>
564
565         Reviewed by Maciej.
566
567         * sunspider: Added --shark20 option, to run Shark at its highest sample
568         resolution instead of its default.
569
570 2007-10-28  Adam Roben  <aroben@apple.com>
571
572         Fix SunSpider on Windows
573
574         Reviewed by Eric.
575
576         * sunspider: Don't use colons in filenames.
577
578 2007-10-28  Darin Adler  <darin@apple.com>
579
580         Reviewed by Adam.
581
582         * resources/sunspider-analyze-results.js: Tweak the output format a little.
583         Change so that when there's only one run we don't write out confidence
584         intervals at all rather than writing "NaN" over and over again.
585
586         * sunspider: Use the time and date as a suffix on the results file, that way
587         all the results are left behind in the tmp directory. This will make it easy
588         for us to add features that compare past results in the future. Also rename
589         the .mshark files using the same scheme.
590
591         * sunspider-compare-results: Relax the parsing rules so we can parse the
592         results file exactly as it's emitted from sunspider.
593
594 2007-10-25  Adam Roben  <aroben@apple.com>
595
596         Strip carriage returns from results
597
598         Carriage returns were screwing up sunspider-compare-results on
599         Windows.
600
601         Rubberstamped by Oliver.
602
603         * sunspider:
604
605 2007-10-24  Darin Adler  <darin@apple.com>
606
607         Reviewed by Eric.
608
609         * sunspider: Open the profile in shark after completing a run.
610         * sunspider-compare-results: Tiny perl tweaks.
611
612 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
613
614         Reviewed by Darin Adler.
615
616         - Don't hardcode my path to testkjs
617         
618         * sunspider-compare-results:
619
620 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
621
622         Reviewed by Darin Adler.
623         
624         - Add a compare script to compare results, and improve formatting of standard results a bit.
625
626         * resources/sunspider-analyze-results.js:
627         * resources/sunspider-compare-results.js: Added.
628         * sunspider-compare-results: Added.
629
630 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
631
632         Reviewed by Darin Adler.
633
634         * resources/sunspider-analyze-results.js: Do the error range
635         properly, using the t-distribution instead of 1.96 (which was
636         based on the normal distribution).
637         
638         * sunspider: Print results in a way that will be friendlier to the
639         soon-to-come compare mode.
640
641 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
642
643         Reviewed by Review.
644         
645         - add more tests (probably the final set for now)
646         
647         Already balanced. Added date, regexp, control flow, and a few
648         more string and object/array access tests.
649         
650         * tests/LIST:
651         * tests/access-fannkuch.js: Added.
652         * tests/access-nbody.js: Added.
653         * tests/controlflow-recursive.js: Added.
654         * tests/date-format-tofte.js: Added.
655         * tests/date-format-xparb.js: Added.
656         * tests/regexp-dna.js: Added.
657         * tests/string-validate-input.js: Added.
658
659 2007-10-21  Eric Seidel  <eric@webkit.org>
660
661         Reviewed by mjs.
662
663         * sunspider: make --shark only sample from testkjs
664
665 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
666
667         Rubber stamped by Eric.
668         
669         - added crypto tests (already balanced)
670
671         * tests/LIST:
672         * tests/crypto-aes.js: Added.
673         * tests/crypto-md5.js: Added.
674         * tests/crypto-sha1.js: Added.
675
676 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
677
678         Reviewed by Adam.
679         
680         - rebalance new tests
681         
682         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
683         
684         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
685
686         This balances them with the existing tests.
687
688         * tests/access-binary-trees.js:
689         * tests/access-nsieve.js:
690         * tests/bitops-nsieve-bits.js:
691         * tests/math-partial-sums.js:
692         * tests/math-spectral-norm.js:
693         * tests/string-fasta.js:
694
695 2007-10-20  Eric Seidel  <eric@webkit.org>
696
697         Reviewed by darin.
698         
699         Add improved argument handling to sunspider, including
700         --runs=<number>, --tests=<pattern, --shark, and --help
701         Also re-factor code into subroutines
702
703         * sunspider: improved argument handling
704
705 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
706
707         Reviewed by Mark.
708
709         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
710         
711         * TODO:
712         * tests/LIST:
713         * tests/access-binary-trees.js: Added.
714         * tests/access-nsieve.js: Added.
715         * tests/bitops-nsieve-bits.js: Added.
716         * tests/math-partial-sums.js: Added.
717         * tests/math-spectral-norm.js: Added.
718         * tests/string-fasta.js: Added.
719
720 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
721
722         Reviewed by Darin Adler.
723
724         * TODO: Updated to note areas that are now well-covered.
725         
726         - Change the tests and the driver to avoid leaving large object
727         graphs hanging around, since that throws off the subsequent tests.
728         
729         * resources/sunspider-standalone-driver.js:
730         * tests/3d-cube.js:
731         * tests/3d-morph.js:
732         * tests/3d-raytrace.js:
733         * tests/bitops-bitwise-and.js:
734         * tests/string-base64.js:
735         * tests/string-tagcloud.js:
736
737 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
738
739         Reviewed by Nikolas.
740         
741         - Rebalance test complexity.
742         
743         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
744         
745         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
746
747         However, for some tests, IE7 was a huge outlier compared to other
748         major browsers. For those tests, I used Firefox 2.0.0.8 on the
749         same OS and hardware as a normalization basis.
750         
751         * tests/3d-morph.js:
752         * tests/3d-raytrace.js:
753         * tests/bitops-3bit-bits-in-byte.js:
754         * tests/bitops-bits-in-byte.js:
755         * tests/bitops-bitwise-and.js:
756         * tests/math-cordic.js:
757         * tests/string-base64.js:
758         * tests/string-tagcloud.js:
759         * tests/string-unpack-code.js:
760
761 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
762
763         Reviewed by Oliver.
764         
765         - Fix the browser-hosted driver to be compatible with Safari 2
766
767         * hosted/sunspider-driver.html:
768
769 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
770
771         Rubber stamped by Hyatt.
772         
773         - Add BSD license notice to cordic test, the original author said to do so.
774
775         * tests/math-cordic.js:
776
777 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
778
779         Reviewed by Eric.
780         
781         - Support Windows and Windows browsers.
782
783         * hosted/sunspider-results.html: Display results correctly in IE.
784         * sunspider: tweaks to work on Windows, and with spaces in the path.
785
786 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
787
788         Rubber stamped by Oliver.
789
790         * tests/string-tagcloud.js: Remove stray print()
791
792 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
793
794         Reviewed by Oliver.
795         
796         Add some string test cases.
797
798         * tests/string-base64.js: Added. Does base64 encoding and decoding.
799         * tests/string-tagcloud.js: Added. Parses a JSON string and
800         generates tagcloud HTML markup based on the data.        
801         * tests/string-unpack-code.js: Added. Unpack compressed versions of
802         four of the most popular JavaScript libraries.
803         * tests/LIST: Added the new tests.
804
805 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
806
807         Reviewed by Anders.
808
809         * TODO: Updated with info on how to do the stats properly.
810
811         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
812         avoid overly slowing down the benchmark.
813
814         Added some 3D-related test cases.
815         
816         * tests/3d-cube.js: Added.
817         * tests/3d-morph.js: Added.
818         * tests/3d-raytrace.js: Added.
819         * tests/LIST:
820
821 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
822
823         Reviewed by Darin Adler.
824
825         - New JavaScript benchmark
826         http://bugs.webkit.org/show_bug.cgi?id=15515
827         
828         A start on a new JavaScript benchmark, with standalone and
829         browser-hosted drivers.
830         
831         The test content is still incomplete and the drivers could use
832         some more features but there is enough here to make a good start.
833         
834         The drivers compute a 95% confidence interval on the mean for the
835         whole test, each category, and each individual test to make it
836         easier to tell whether differences are statistically
837         significant. The confidence interval can be narrowed by running
838         the test more times.
839         
840         * TODO: Added. Remaining things that need to be done.
841         * sunspider: Added. Perl script that acts as the standalone test driver.
842         * resources/sunspider-standalone-driver.js: Added. JavaScript part
843         of standalone test driver.
844         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
845         analysis code.
846         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
847         * make-hosted: Added. Script to generate browser-hosted tests.
848         * hosted/sunspider.html: Added. Start page for browser-hosted test.
849         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
850         * hosted/sunspider-results.html: Added. Results page for browser-hosted
851         tests.
852         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
853         tests.
854         * tests/LIST: Added. List of tests to use.
855         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
856         * tests/bitops-bits-in-byte.js: Added. Ditto.
857         * tests/bitops-bitwise-and.js: Added. Ditto.
858         * tests/math-cordic.js: Added. Ditto.
859