2009-12-14 Maciej Stachowiak <mjs@apple.com>
[WebKit.git] / SunSpider / ChangeLog
1 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
2
3         Reviewed by Sam Weinig.
4
5         in-browser SunSpider suffers excessive penalty under power management
6         https://bugs.webkit.org/show_bug.cgi?id=32505
7         
8         I have made a few changes to address this:
9         
10         1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
11         document.write to reduce triggering of progress UI.
12         2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
13         stepped all the way down the whole time.
14         3) Run the test cycle an extra time for warmup before the runs that count.
15
16         * hosted/sunspider-record-result.js: Removed.
17         * make-hosted:
18         * resources/TEMPLATE.html:
19         * resources/driver-TEMPLATE.html:
20
21 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
22
23         Reviewed by Gavin Barraclough.
24
25         SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
26         https://bugs.webkit.org/show_bug.cgi?id=16806
27
28         Based on a patch by Eric Seidel.
29         
30         Fix the base64 computation to actually compute correct results. The impact on runtime of
31         the test is pretty small, but noticeable for some browsers. But at least it's not
32         doing a wrong and meaningless computation any more.
33         
34         * tests/sunspider-0.9.1/string-base64.js:
35         ():
36         (base64ToString):
37
38 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
39
40         Fixing commit error...
41         
42         I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
43
44         * resources/driver-TEMPLATE.html:
45
46 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
47
48         Reviewed by Sam Weinig.
49
50         Use JSON.parse instead of eval for Web-hosted SunSpider results processing
51         https://bugs.webkit.org/show_bug.cgi?id=32490
52
53         * hosted/json2.js: Added. Incorporated from json.org
54         * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
55         * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
56         results.
57
58 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
59
60         Reviewed by Oliver Hunt.
61
62         Enable Web-hosted version of SunSpider to handle multiple versions
63         https://bugs.webkit.org/show_bug.cgi?id=32478
64
65         * make-hosted: Now generate the hosted version to be able to run both the
66         0.9 and the 0.9.1 test suites.
67         * hosted: Modified property svn:ignore.
68         * hosted/sunspider-driver.html: Removed.
69         * hosted/sunspider-results.html: Removed.
70         * hosted/sunspider.html:
71         * hosted/versions.html: Added.
72         * resources/TEMPLATE.html:
73         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
74         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
75
76 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
77
78         Reviewed by Oliver Hunt.
79
80         Give command-line sunspider the ability to handle multiple suites and versions
81         https://bugs.webkit.org/show_bug.cgi?id=32477
82         
83         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
84         enumerating the tests. Current suites are:
85         
86         sunspider-0.9 - riginal SunSpider content 
87         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
88         ubench - "ubench" microbenchmark suite
89         v8-v4 - version 4 of the v8 benchmark
90         parse-only - some parser tests
91         
92         Sunspider can now be run with the --suite= parameter to select a
93         suite. The default is --suite=sunspider-0.9.
94
95         * .: Modified property svn:ignore to ignore new results directories.
96         * make-hosted: Generate based on sunspider-0.9 suite.
97         * resources/TEMPLATE.html: Fix an HTML compliance problem.
98         * resources/sunspider-standalone-driver.js: Become aware of suite names.
99         * sunspider: Modified to handle multiple suites as described above.
100         * sunspider-compare-results: ditto
101         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
102         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
103         * tests/LIST-UBENCH: Removed.
104         * tests/LIST-V8: Removed.
105         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
106         * tests/3d-morph.js: ditto
107         * tests/3d-raytrace.js: ditto
108         * tests/access-binary-trees.js: ditto
109         * tests/access-fannkuch.js: ditto
110         * tests/access-nbody.js: ditto
111         * tests/access-nsieve.js: ditto
112         * tests/bitops-3bit-bits-in-byte.js: ditto
113         * tests/bitops-bits-in-byte.js: ditto
114         * tests/bitops-bitwise-and.js: ditto
115         * tests/bitops-nsieve-bits.js: ditto
116         * tests/controlflow-recursive.js: ditto
117         * tests/crypto-aes.js: ditto
118         * tests/crypto-md5.js: ditto
119         * tests/crypto-sha1.js: ditto
120         * tests/date-format-tofte.js: ditto
121         * tests/date-format-xparb.js: ditto
122         * tests/math-cordic.js: ditto
123         * tests/math-partial-sums.js: ditto
124         * tests/math-spectral-norm.js: ditto
125         * tests/regexp-dna.js: ditto
126         * tests/string-base64.js: ditto
127         * tests/string-fasta.js: ditto
128         * tests/string-tagcloud.js: ditto
129         * tests/string-unpack-code.js: ditto
130         * tests/string-validate-input.js: ditto
131         * tests/sunspider-0.9/: Added
132         * tests/sunspider-0.9/*.js: Moved from one level up
133         * tests/sunspider-0.9.1: Added.
134         * tests/sunspider-0.9.1/*.js: Moved from one level up.
135         * tests/v8-crypto.js: Moved to v8-v4/ directory.
136         * tests/v8-deltablue.js: ditto
137         * tests/v8-earley-boyer.js: ditto
138         * tests/v8-raytrace.js: ditto
139         * tests/v8-regexp.js: ditto
140         * tests/v8-richards.js: ditto
141         * tests/v8-splay.js: ditto
142         * tests/v8-v4: Added.
143
144 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
145
146         Reviewed by Darin Adler.
147
148         Platform specific null device replaced with a platform independent.
149         https://bugs.webkit.org/show_bug.cgi?id=29544
150
151         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
152
153 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
154
155         Reviewed by Maciej Stachowiak.
156
157         [Qt] Make sunspider script work on Windows platform.
158         https://bugs.webkit.org/show_bug.cgi?id=29544
159
160         * sunspider:
161         Using platform specific null device instead of /dev/null.
162
163 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
164
165         Reviewed by Oliver Hunt.
166
167         https://bugs.webkit.org/show_bug.cgi?id=26540
168         Currently the SunSpider test driver lacks an option to run a test suite that
169         will test JavaScriptCore parsing performance only.  This patch adds just such
170         a test suite and option to SunSpider as well as the jsc test shell.  I've included
171         three large javascript source files found in the wild: jquery, mootools and prototype.
172         Combined with the concatenation of all three, these form a new testsuite to measure
173         and test pure JavaScriptCore parsing performance.
174
175         * resources/sunspider-standalone-driver.js:
176           Load the js with parse-only flag if it is part of the parse-only suite.
177         * sunspider:
178           Add the parse-only suite to the list of options.
179         * sunspider-compare-results:
180           Add the parse-only suite to the list of options.
181         * tests/LIST-PARSE-ONLY: Added.
182         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
183           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
184         * tests/parse-only/jquery-1.3.2.js: Added.
185           Latest version from jquery.com.
186         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
187           Latest version from mootools.net.
188         * tests/parse-only/prototype-1.6.0.3.js: Added.
189           Latest version from prototypejs.org.
190
191 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
192
193         Update V8 benchmark to version 4.
194         
195         Reviewed by Cameron Zwarich.
196
197         * tests/LIST-V8:
198         * tests/v8-crypto.js:
199         * tests/v8-deltablue.js:
200         * tests/v8-earley-boyer.js:
201         * tests/v8-raytrace.js:
202         * tests/v8-regexp.js: Added.
203         * tests/v8-richards.js:
204         * tests/v8-splay.js: Added.
205
206 2009-05-26  Olivier DOLE  <odole@pleyo.com>
207
208         Reviewed by Maciej.
209
210         Store sunspider test suite results in a specific directory for each
211         different type of benchmark (sunspider, V8 or ubench).
212
213         * sunspider:
214         * sunspider-compare-results:
215
216 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
217
218         Build fix.
219
220         * SunSpider.make:
221
222 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
223
224         Include SunSpider tests in production builds.
225         
226         Reviewed by Adam Roben.
227
228         * SunSpider.make: Added.
229         * xcopy.excludes: Added.
230
231 2009-03-04  Oliver Hunt  <oliver@apple.com>
232
233         Added a few more tests to ubench
234
235         RS=Gavin Barraclough
236
237         Adding a couple of test to cover too few and too many arguments.  This
238         is needed to prevent regressions in all call paths from my current refactoring.
239
240         * tests/LIST-UBENCH:
241         * tests/ubench/function-correct-args.js: Added.
242         (f):
243         * tests/ubench/function-excess-args.js: Added.
244         (f):
245
246 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
247
248         Fix a bug where when V8's or UBENCH's test suite list file is missing
249         the error message is suggesting that SunSpider's list file is missing.
250
251         Reviewed by Sam Weinig.
252
253         * sunspider:
254         (loadTestsList): Fix the error message to use the right list name.
255
256 2008-10-07  David Hyatt  <hyatt@apple.com>
257
258         https://bugs.webkit.org/show_bug.cgi?id=18989
259
260         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
261         we want all browsers to be doing the same thing on the test.
262
263         Reviewed by Maciej
264
265         * tests/regexp-dna.js:
266
267 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
268
269         Rubber-stamped by Maciej Stachowiak.
270
271         The bleeding edge branch of V8 now supports passing fileanmes after
272         '-f', so the SunSpider script should be changed to do this again.
273         This allows all 3 major JavaScript engines to be used with the
274         unmodified SunSpider script.
275
276         * sunspider:
277
278 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
279
280         Reviewed by Maciej Stachowiak.
281
282         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
283         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
284         correct the punctuation in its documentation.
285
286         * sunspider:
287
288 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
289
290         Reviewed by Oliver Hunt.
291
292         Bug 20616: Incorporate V8 benchmarks in testing
293         <https://bugs.webkit.org/show_bug.cgi?id=20616>
294
295         This is a first attempt at running the V8 benchmarks within SunSpider,
296         with the --v8 option. The number of iterations is fixed for each
297         benchmark, and was chosen to make each of the benchmarks run between
298         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
299
300         * sunspider:
301         * tests/LIST-V8: Added.
302         * tests/v8-crypto.js: Added.
303         * tests/v8-deltablue.js: Added.
304         * tests/v8-earley-boyer.js: Added.
305         * tests/v8-raytrace.js: Added.
306         * tests/v8-richards.js: Added.
307
308 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
309
310         Not reviewed.
311
312         Remove a local change that was accidentally committed in r36034.
313
314         * resources/sunspider-standalone-driver.js:
315
316 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
317
318         Rubber-stamped by Oliver Hunt.
319
320         Make SunSpider work with JavaScript shells that don't take filenames
321         after a '-f' argument.
322
323         * sunspider:
324
325 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
326
327         Reviewed by Timothy.
328
329         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
330
331         * sunspider:
332         * tests/LIST-SQUIRRELFISH: Removed.
333
334 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
335
336         Rubber-stamped by Mark Rowe.
337
338         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
339
340         * sunspider:
341
342 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
343
344         Rubber stamped by Oliver.
345         
346         - add newly working tests to squirrelfish list
347
348         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
349
350 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
351
352         Rubber stamped by Oliver.
353         
354         - move ubench tests properly.
355
356         * tests/ubench: Added.
357         * tests/ubench/function-closure.js: Added.
358         * tests/ubench/function-empty.js: Added.
359         * tests/ubench/function-missing-args.js: Added.
360         * tests/ubench/function-sum.js: Added.
361         * tests/ubench/loop-empty-resolve.js: Added.
362         * tests/ubench/loop-empty.js: Added.
363         * tests/ubench/loop-sum.js: Added.
364
365 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
366
367         Rubber stamped by Oliver.
368
369         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
370
371 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
372
373         Reviewed by Oliver.
374
375         - make --squirrelfish mode use the real harness, but a reduced set of tests
376
377         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
378
379         * sunspider:
380         * tests/LIST-SQUIRRELFISH:
381         * tests/LIST-UBENCH: Added.
382         * tests/squirrelfish: Moved to tests/ubench
383
384 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
385
386         Reviewed by Maciej Stachowiak.
387         
388         Removed an unused test.
389
390         * tests/squirrelfish/loop-resolve.js: Removed.
391
392 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
393
394         Reviewed by Oliver.
395         
396         Add newly runnable tests to --squirrelfish mode.
397
398         * tests/LIST-SQUIRRELFISH:
399
400 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
401
402         Reviewed by Maciej Stachowiak.
403         
404         A little more love for --squirrelfish mode:
405         
406         Fixed a misplaced sort. Results properly sort now.
407         
408         Added a test list just for squirrelfish, and updated it to include
409         all currently passing SunSpider tests.
410         
411         Fixed the pruning regexp to match 3d-morph.js.
412
413         * sunspider:
414
415 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
416
417         Reviewed by Oliver Hunt.
418         
419         --squirrelfish mode: pared down tests for squirrelfish to chew on.
420
421         * sunspider:
422         * tests/LIST:
423         * tests/squirrelfish: Added.
424         * tests/squirrelfish/function-closure.js: Added.
425         * tests/squirrelfish/function-empty.js: Added.
426         * tests/squirrelfish/function-missing-args.js: Added.
427         * tests/squirrelfish/function-sum.js: Added.
428         * tests/squirrelfish/loop-empty-resolve.js: Added.
429         * tests/squirrelfish/loop-empty.js: Added.
430         * tests/squirrelfish/loop-resolve.js: Added.
431         * tests/squirrelfish/loop-sum.js: Added.
432
433 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
434
435         Reviewed by Geoff.
436
437         * sunspider: Pause update daemon on Darwin to make results reliable
438         again (workaround for <rdar://problem/5811127>).
439
440 2008-02-07  Eric Seidel  <eric@webkit.org>
441
442         Reviewed by Sam.
443
444         * sunspider: add an error message when ./tests/LIST is not found
445
446 2007-12-29  Alp Toker  <alp@atoker.com>
447
448         Fix typo.
449
450         * hosted/sunspider.html:
451
452 2007-12-29  Alp Toker  <alp@atoker.com>
453
454         Reviewed by Maciej.
455
456         Provide the current URL in the SunSpider results page. Useful for
457         copying benchmark results out of browsers without a location bar or
458         where the location bar can't handle long URLs.
459
460         * hosted/sunspider-results.html:
461
462 2007-12-20  Eric Seidel  <eric@webkit.org>
463
464         Reviewed by mjs.
465
466         * sunspider: increase default number of runs for more accurate numbers
467
468 2007-12-18  Mark Rowe  <mrowe@apple.com>
469
470         Fix typo.
471
472         * hosted/sunspider.html:
473
474 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
475
476         Reviewed by Eric.
477         
478         - give an extra digit of precision for small differences.
479
480         * resources/sunspider-compare-results.js:
481
482 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
483
484         Reviewed by Eric.
485         
486         - work around IE layout bugs
487
488         * hosted/sunspider.css:
489
490 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
491
492         Reviewed by Mark.
493         
494         - fix compare feature
495
496         * hosted/sunspider-results.html:
497
498 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
499
500         Reviewed by Mark.
501         
502         - Change the format of the output URL to be shorter, to meet IE constraints
503
504         * hosted/sunspider-driver.html:
505         * hosted/sunspider-results.html:
506
507 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
508
509         Reviewed by Oliver.
510         
511         - Add some explanatory test and a dumb logo to SunSpider web hosted version
512
513         * hosted/sunspider-driver.html:
514         * hosted/sunspider-results.html:
515         * hosted/sunspider.css:
516         * hosted/sunspider.html:
517
518 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
519
520         Reviewed by Oliver.
521         
522         - Make the web-hosted version of SunSpider look a little bit less drab.
523
524         * hosted/sunspider-driver.html:
525         * hosted/sunspider-results.html:
526         * hosted/sunspider.css: Added.
527         * hosted/sunspider.html:
528         * resources/TEMPLATE.html:
529
530 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
531
532         Reviewed by Sam.
533         
534         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
535         
536         A sample of what new-style results would look like:
537         
538         TEST                   COMPARISON            FROM                 TO             DETAILS
539         ...
540         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
541         ...
542             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
543         
544         * resources/sunspider-compare-results.js:
545
546 2007-11-14  Eric Seidel  <eric@webkit.org>
547
548         Reviewed by Sam.
549
550         * sunspider: add --shark-cache for L2 Cache Miss profiling
551
552 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
553
554         Reviewed by Darin Adler.
555
556         http://bugs.webkit.org/show_bug.cgi?id=15925
557         SunSpider should check for Shark being installed
558
559         * sunspider: Explicitly test that shark command line tool is installed.
560
561 2007-11-04  Darin Adler  <darin@apple.com>
562
563         * hosted/sunspider.html: Tweaked formatting a bit.
564
565 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
566
567         Reviewed by Darin Adler.
568
569         - Add compare mode and a description of the benchmark to the hosted version.
570
571         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
572         * hosted/sunspider.html: Added intro text.
573         * make-hosted: Copy new file.
574         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
575         a function.
576         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
577         expects.
578         * sunspider-compare-results: Use sunspider-standalone-compare.js.
579
580 2007-11-02  Darin Adler  <darin@apple.com>
581
582         * sunspider: Changed "--base" to "--set-baseline".
583
584 2007-11-02  Darin Adler  <darin@apple.com>
585
586         Reviewed by Maciej.
587
588         * sunspider: Added a "--base" option that records the current test as a baseline.
589         Moved the code to find the newest .mshark file into a function (since I needed to use
590         it in the other script and I prefer to copy/paste a function).
591
592         * sunspider-compare-results: Added logic to compare the baseline with the last
593         run when you don't pass any parameters.
594
595 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
596
597         Reviewed by Eric.
598         
599         - Added list of ECMAScript3 features that SunSpider does not cover currently.
600
601         * UNCOVERED: Added.
602
603 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
604
605         Reviewed by Oliver.
606         
607         - Give better error messages when sunspider-compare-results fails
608
609         * sunspider-compare-results:
610
611 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
612
613         Reviewed by Darin Adler.
614         
615         - fix Date tests to work in command-line SpiderMonkey
616
617         * tests/date-format-tofte.js:
618         * tests/date-format-xparb.js:
619
620 2007-10-29  Darin Adler  <darin@apple.com>
621
622         Reviewed by Maciej.
623
624         * sunspider: Added --shark20 option, to run Shark at its highest sample
625         resolution instead of its default.
626
627 2007-10-28  Adam Roben  <aroben@apple.com>
628
629         Fix SunSpider on Windows
630
631         Reviewed by Eric.
632
633         * sunspider: Don't use colons in filenames.
634
635 2007-10-28  Darin Adler  <darin@apple.com>
636
637         Reviewed by Adam.
638
639         * resources/sunspider-analyze-results.js: Tweak the output format a little.
640         Change so that when there's only one run we don't write out confidence
641         intervals at all rather than writing "NaN" over and over again.
642
643         * sunspider: Use the time and date as a suffix on the results file, that way
644         all the results are left behind in the tmp directory. This will make it easy
645         for us to add features that compare past results in the future. Also rename
646         the .mshark files using the same scheme.
647
648         * sunspider-compare-results: Relax the parsing rules so we can parse the
649         results file exactly as it's emitted from sunspider.
650
651 2007-10-25  Adam Roben  <aroben@apple.com>
652
653         Strip carriage returns from results
654
655         Carriage returns were screwing up sunspider-compare-results on
656         Windows.
657
658         Rubberstamped by Oliver.
659
660         * sunspider:
661
662 2007-10-24  Darin Adler  <darin@apple.com>
663
664         Reviewed by Eric.
665
666         * sunspider: Open the profile in shark after completing a run.
667         * sunspider-compare-results: Tiny perl tweaks.
668
669 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
670
671         Reviewed by Darin Adler.
672
673         - Don't hardcode my path to testkjs
674         
675         * sunspider-compare-results:
676
677 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
678
679         Reviewed by Darin Adler.
680         
681         - Add a compare script to compare results, and improve formatting of standard results a bit.
682
683         * resources/sunspider-analyze-results.js:
684         * resources/sunspider-compare-results.js: Added.
685         * sunspider-compare-results: Added.
686
687 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
688
689         Reviewed by Darin Adler.
690
691         * resources/sunspider-analyze-results.js: Do the error range
692         properly, using the t-distribution instead of 1.96 (which was
693         based on the normal distribution).
694         
695         * sunspider: Print results in a way that will be friendlier to the
696         soon-to-come compare mode.
697
698 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
699
700         Reviewed by Review.
701         
702         - add more tests (probably the final set for now)
703         
704         Already balanced. Added date, regexp, control flow, and a few
705         more string and object/array access tests.
706         
707         * tests/LIST:
708         * tests/access-fannkuch.js: Added.
709         * tests/access-nbody.js: Added.
710         * tests/controlflow-recursive.js: Added.
711         * tests/date-format-tofte.js: Added.
712         * tests/date-format-xparb.js: Added.
713         * tests/regexp-dna.js: Added.
714         * tests/string-validate-input.js: Added.
715
716 2007-10-21  Eric Seidel  <eric@webkit.org>
717
718         Reviewed by mjs.
719
720         * sunspider: make --shark only sample from testkjs
721
722 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
723
724         Rubber stamped by Eric.
725         
726         - added crypto tests (already balanced)
727
728         * tests/LIST:
729         * tests/crypto-aes.js: Added.
730         * tests/crypto-md5.js: Added.
731         * tests/crypto-sha1.js: Added.
732
733 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
734
735         Reviewed by Adam.
736         
737         - rebalance new tests
738         
739         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
740         
741         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
742
743         This balances them with the existing tests.
744
745         * tests/access-binary-trees.js:
746         * tests/access-nsieve.js:
747         * tests/bitops-nsieve-bits.js:
748         * tests/math-partial-sums.js:
749         * tests/math-spectral-norm.js:
750         * tests/string-fasta.js:
751
752 2007-10-20  Eric Seidel  <eric@webkit.org>
753
754         Reviewed by darin.
755         
756         Add improved argument handling to sunspider, including
757         --runs=<number>, --tests=<pattern, --shark, and --help
758         Also re-factor code into subroutines
759
760         * sunspider: improved argument handling
761
762 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
763
764         Reviewed by Mark.
765
766         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
767         
768         * TODO:
769         * tests/LIST:
770         * tests/access-binary-trees.js: Added.
771         * tests/access-nsieve.js: Added.
772         * tests/bitops-nsieve-bits.js: Added.
773         * tests/math-partial-sums.js: Added.
774         * tests/math-spectral-norm.js: Added.
775         * tests/string-fasta.js: Added.
776
777 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
778
779         Reviewed by Darin Adler.
780
781         * TODO: Updated to note areas that are now well-covered.
782         
783         - Change the tests and the driver to avoid leaving large object
784         graphs hanging around, since that throws off the subsequent tests.
785         
786         * resources/sunspider-standalone-driver.js:
787         * tests/3d-cube.js:
788         * tests/3d-morph.js:
789         * tests/3d-raytrace.js:
790         * tests/bitops-bitwise-and.js:
791         * tests/string-base64.js:
792         * tests/string-tagcloud.js:
793
794 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
795
796         Reviewed by Nikolas.
797         
798         - Rebalance test complexity.
799         
800         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
801         
802         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
803
804         However, for some tests, IE7 was a huge outlier compared to other
805         major browsers. For those tests, I used Firefox 2.0.0.8 on the
806         same OS and hardware as a normalization basis.
807         
808         * tests/3d-morph.js:
809         * tests/3d-raytrace.js:
810         * tests/bitops-3bit-bits-in-byte.js:
811         * tests/bitops-bits-in-byte.js:
812         * tests/bitops-bitwise-and.js:
813         * tests/math-cordic.js:
814         * tests/string-base64.js:
815         * tests/string-tagcloud.js:
816         * tests/string-unpack-code.js:
817
818 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
819
820         Reviewed by Oliver.
821         
822         - Fix the browser-hosted driver to be compatible with Safari 2
823
824         * hosted/sunspider-driver.html:
825
826 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
827
828         Rubber stamped by Hyatt.
829         
830         - Add BSD license notice to cordic test, the original author said to do so.
831
832         * tests/math-cordic.js:
833
834 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
835
836         Reviewed by Eric.
837         
838         - Support Windows and Windows browsers.
839
840         * hosted/sunspider-results.html: Display results correctly in IE.
841         * sunspider: tweaks to work on Windows, and with spaces in the path.
842
843 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
844
845         Rubber stamped by Oliver.
846
847         * tests/string-tagcloud.js: Remove stray print()
848
849 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
850
851         Reviewed by Oliver.
852         
853         Add some string test cases.
854
855         * tests/string-base64.js: Added. Does base64 encoding and decoding.
856         * tests/string-tagcloud.js: Added. Parses a JSON string and
857         generates tagcloud HTML markup based on the data.        
858         * tests/string-unpack-code.js: Added. Unpack compressed versions of
859         four of the most popular JavaScript libraries.
860         * tests/LIST: Added the new tests.
861
862 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
863
864         Reviewed by Anders.
865
866         * TODO: Updated with info on how to do the stats properly.
867
868         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
869         avoid overly slowing down the benchmark.
870
871         Added some 3D-related test cases.
872         
873         * tests/3d-cube.js: Added.
874         * tests/3d-morph.js: Added.
875         * tests/3d-raytrace.js: Added.
876         * tests/LIST:
877
878 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
879
880         Reviewed by Darin Adler.
881
882         - New JavaScript benchmark
883         http://bugs.webkit.org/show_bug.cgi?id=15515
884         
885         A start on a new JavaScript benchmark, with standalone and
886         browser-hosted drivers.
887         
888         The test content is still incomplete and the drivers could use
889         some more features but there is enough here to make a good start.
890         
891         The drivers compute a 95% confidence interval on the mean for the
892         whole test, each category, and each individual test to make it
893         easier to tell whether differences are statistically
894         significant. The confidence interval can be narrowed by running
895         the test more times.
896         
897         * TODO: Added. Remaining things that need to be done.
898         * sunspider: Added. Perl script that acts as the standalone test driver.
899         * resources/sunspider-standalone-driver.js: Added. JavaScript part
900         of standalone test driver.
901         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
902         analysis code.
903         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
904         * make-hosted: Added. Script to generate browser-hosted tests.
905         * hosted/sunspider.html: Added. Start page for browser-hosted test.
906         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
907         * hosted/sunspider-results.html: Added. Results page for browser-hosted
908         tests.
909         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
910         tests.
911         * tests/LIST: Added. List of tests to use.
912         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
913         * tests/bitops-bits-in-byte.js: Added. Ditto.
914         * tests/bitops-bitwise-and.js: Added. Ditto.
915         * tests/math-cordic.js: Added. Ditto.
916