9d62bc16c6d6ea61cf10a8a1973be96be7ced330
[WebKit-https.git] / PerformanceTests / SunSpider / ChangeLog
1 2014-07-08  Gergo Balogh  <gbalogh.u-szeged@partner.samsung.com>
2
3         Add option to specify output file location in sunspider.
4         https://bugs.webkit.org/show_bug.cgi?id=134339
5
6         Reviewed by Darin Adler.
7
8         * sunspider:
9
10 2014-06-03  Eva Balazsfalvi  <evab.u-szeged@partner.samsung.com>
11
12         REGRESSION(r169092 and r169102): Skip failing JSC tests poperly on non-x86 Darwin platforms
13         https://bugs.webkit.org/show_bug.cgi?id=133149
14
15         Reviewed by Csaba Osztrogon√°c.
16
17         * profiler-test.yaml: Skip profiler tests only if the architecture isn't x86 and the host is Darwin.
18
19 2014-05-21  Filip Pizlo  <fpizlo@apple.com>
20
21         Unreviewed, roll out http://trac.webkit.org/changeset/169159.
22         
23         This was a unilateral change and wasn't properly reviewed.
24
25         * profiler-test.yaml:
26
27 2014-05-19  Filip Pizlo  <fpizlo@apple.com>
28
29         Take care of some ARM64 test failures
30         https://bugs.webkit.org/show_bug.cgi?id=133090
31
32         Reviewed by Geoffrey Garen.
33
34         * profiler-test.yaml: Skip these on not-x86.
35
36 2014-03-14  Maciej Stachowiak  <mjs@apple.com>
37
38         Replace "Apple Computer, Inc." with "Apple Inc." in copyright headers
39         https://bugs.webkit.org/show_bug.cgi?id=130276
40         <rdar://problem/16266927>
41
42         Reviewed by Simon Fraser.
43
44         * hosted/sunspider.html:
45         * hosted/versions.html:
46         * make-hosted:
47         * resources/TEMPLATE.html:
48         * resources/driver-TEMPLATE.html:
49         * resources/results-TEMPLATE.html:
50         * resources/sunspider-analyze-results.js:
51         * resources/sunspider-compare-results.js:
52         * resources/sunspider-standalone-compare.js:
53         * resources/sunspider-standalone-driver.js:
54         * sunspider:
55         * sunspider-compare-results:
56         * tests/sunspider-0.9.1/3d-morph.js:
57         * tests/sunspider-0.9.1/3d-raytrace.js:
58         * tests/sunspider-0.9.1/bitops-bitwise-and.js:
59         * tests/sunspider-0.9.1/math-cordic.js:
60         * tests/sunspider-0.9.1/string-tagcloud.js:
61         * tests/sunspider-0.9/3d-morph.js:
62         * tests/sunspider-0.9/3d-raytrace.js:
63         * tests/sunspider-0.9/bitops-bitwise-and.js:
64         * tests/sunspider-0.9/math-cordic.js:
65         * tests/sunspider-0.9/string-tagcloud.js:
66         * tests/sunspider-1.0.1/3d-morph.js:
67         * tests/sunspider-1.0.1/3d-raytrace.js:
68         * tests/sunspider-1.0.1/bitops-bitwise-and.js:
69         * tests/sunspider-1.0.1/math-cordic.js:
70         * tests/sunspider-1.0.1/string-tagcloud.js:
71         * tests/sunspider-1.0.2/3d-morph.js:
72         * tests/sunspider-1.0.2/3d-raytrace.js:
73         * tests/sunspider-1.0.2/bitops-bitwise-and.js:
74         * tests/sunspider-1.0.2/math-cordic.js:
75         * tests/sunspider-1.0.2/string-tagcloud.js:
76         * tests/sunspider-1.0/3d-morph.js:
77         * tests/sunspider-1.0/3d-raytrace.js:
78         * tests/sunspider-1.0/bitops-bitwise-and.js:
79         * tests/sunspider-1.0/math-cordic.js:
80         * tests/sunspider-1.0/string-tagcloud.js:
81
82 2014-02-10  Filip Pizlo  <fpizlo@apple.com>
83
84         Unreviewed, revert the accidental change from r163789.
85
86         * tests/v8-v6/v8-deltablue.js:
87
88 2013-12-10  Filip Pizlo  <fpizlo@apple.com>
89
90         Get rid of forward exit on DoubleAsInt32
91         https://bugs.webkit.org/show_bug.cgi?id=125552
92
93         Reviewed by Oliver Hunt.
94         
95         Use SunSpider as a kind of spot-check for the
96         no-architecture-specific-optimization paths in the compiler.
97
98         * no-architecture-specific-optimizations.yaml: Added.
99
100 2013-10-08  Geoffrey Garen  <ggaren@apple.com>
101
102         Refined power management in SunSpider 1.0.2
103         https://bugs.webkit.org/show_bug.cgi?id=122525
104
105         Reviewed by Gavin Barraclough.
106
107         * hosted/sunspider.css:
108         (#frameparent):
109         (#countdown):
110         * resources/TEMPLATE.html: Provide an explicit countdown element in the
111         parent, instead of hacking it into each child frame.
112
113         * resources/driver-TEMPLATE.html: Be more explicit about how we respond
114         to power management. Just moving setTimeout(0) inside the loop is a
115         little too subtle, and may not work across browsers.
116
117         Also, leave a little extra time after changing the countdown graphic,
118         to give the browser some time to paint.
119
120 2013-10-02  Geoffrey Garen  <ggaren@apple.com>
121
122         Improved painting behavior in SunSpider 1.0.2
123         https://bugs.webkit.org/show_bug.cgi?id=122257
124
125         Reviewed by Oliver Hunt.
126
127         Individual test names with times was a good painting strategy years
128         back, when each benchmark took hundreds of milliseconds to run. Now,
129         the whole suite can finish in a a few hundres milliseconds, and things
130         look pretty flashy.
131
132         Instead, display a countdown that indicates which iteration of the
133         test we're on.
134
135         * resources/TEMPLATE.html: Work around a bug in Firefox by specifying
136         our style inside the .html file instead of in an external stylesheet.
137         Under heavy JavaScript load, Firefox neglects to apply external
138         stylesheets -- probably due to a starved timer.
139         
140         * resources/driver-TEMPLATE.html: Delay a little between runs to ensure
141         that our countdown has enough time to paint.
142
143 2013-09-26  Geoffrey Garen  <ggaren@apple.com>
144
145         Fixed a version number string in SunSpider 1.0.2
146         https://bugs.webkit.org/show_bug.cgi?id=122004
147
148         Reviewed by Gavin Barraclough.
149
150         * resources/driver-TEMPLATE.html:
151         * resources/results-TEMPLATE.html: 1.0.2!
152
153 2013-09-26  Geoffrey Garen  <ggaren@apple.com>
154
155         Tentative SunSpider 1.0.2
156         https://bugs.webkit.org/show_bug.cgi?id=122002
157
158         Reviewed by Gavin Barraclough.
159
160         Be even more aggressive about asking for zero delay between tests by
161         setting a timeout at the beginning of a test instead of the end. This is
162         another step toward reducing noise -- particularly noise caused by
163         power management -- just like the step down from 10ms to 0ms.
164
165         Browsers typically put a 4ms floor on timers. The sooner we set
166         our timer, the sooner it will fire.
167
168         * make-hosted:
169         * resources/driver-TEMPLATE.html: Changed the driver to set the timeout
170         earlier.
171         * tests/sunspider-1.0.2: Added. These tests are unchanged from 1.0.1.
172
173 2013-09-09  Filip Pizlo  <fpizlo@apple.com>
174
175         Stress tests should test the jsc profiler (-p)
176         https://bugs.webkit.org/show_bug.cgi?id=121043
177
178         Reviewed by Mark Hahnenberg.
179         
180         Add a jsc-stress-test that tries to profile SunSpider.
181
182         * profiler-test.yaml: Added.
183
184 2013-08-08  Cosmin Truta  <ctruta@blackberry.com>
185
186         SunSpider: Move the 3d-morph fix from version 1.0 to version 1.0.1
187         https://bugs.webkit.org/show_bug.cgi?id=117993
188
189         Reviewed by George Staikos.
190
191         Bump the SunSpider version to 1.0.1.
192
193         Unroll r149548 in order to restore version 1.0 to its original,
194         and add the 3d-morph fix to version 1.0.1 instead.
195
196         * hosted/sunspider.html: Updated to version 1.0.1.
197         * hosted/versions.html: Updated to version 1.0.1.
198         * make-hosted: Added support for version 1.0.1.
199         * resources/driver-TEMPLATE.html: Updated to version 1.0.1.
200         * resources/results-TEMPLATE.html: Updated to version 1.0.1.
201         * sunspider: Updated to version 1.0.1.
202         * sunspider-compare-results: Updated to version 1.0.1.
203         * tests/sunspider-1.0/3d-morph.js: Rolled back to the original version.
204         * tests/sunspider-1.0.1/LIST: Added.
205         * tests/sunspider-1.0.1/3d-cube.js: Added.
206         * tests/sunspider-1.0.1/3d-morph.js: Added.
207         * tests/sunspider-1.0.1/3d-raytrace.js: Added.
208         * tests/sunspider-1.0.1/access-binary-trees.js: Added.
209         * tests/sunspider-1.0.1/access-fannkuch.js: Added.
210         * tests/sunspider-1.0.1/access-nbody.js: Added.
211         * tests/sunspider-1.0.1/access-nsieve.js: Added.
212         * tests/sunspider-1.0.1/bitops-3bit-bits-in-byte.js: Added.
213         * tests/sunspider-1.0.1/bitops-bits-in-byte.js: Added.
214         * tests/sunspider-1.0.1/bitops-bitwise-and.js: Added.
215         * tests/sunspider-1.0.1/bitops-nsieve-bits.js: Added.
216         * tests/sunspider-1.0.1/controlflow-recursive.js: Added.
217         * tests/sunspider-1.0.1/crypto-aes.js: Added.
218         * tests/sunspider-1.0.1/crypto-md5.js: Added.
219         * tests/sunspider-1.0.1/crypto-sha1.js: Added.
220         * tests/sunspider-1.0.1/date-format-tofte.js: Added.
221         * tests/sunspider-1.0.1/date-format-xparb.js: Added.
222         * tests/sunspider-1.0.1/math-cordic.js: Added.
223         * tests/sunspider-1.0.1/math-partial-sums.js: Added.
224         * tests/sunspider-1.0.1/math-spectral-norm.js: Added.
225         * tests/sunspider-1.0.1/regexp-dna.js: Added.
226         * tests/sunspider-1.0.1/string-base64.js: Added.
227         * tests/sunspider-1.0.1/string-fasta.js: Added.
228         * tests/sunspider-1.0.1/string-tagcloud.js: Added.
229         * tests/sunspider-1.0.1/string-unpack-code.js: Added.
230         * tests/sunspider-1.0.1/string-validate-input.js: Added.
231
232 2013-05-03  Cosmin Truta  <ctruta@blackberry.com>
233
234         SunSpider 1.0: 3d-morph: use epsilon to check result
235         https://bugs.webkit.org/show_bug.cgi?id=115553
236
237         Reviewed by George Staikos.
238
239         SunSpider/3d-morph should be more lenient in validating its results, since they depend on Math.sin, which isn't formally specified.
240
241         * tests/sunspider-1.0/3d-morph.js:
242
243 2013-04-19  Filip Pizlo  <fpizlo@apple.com>
244
245         SunSpider/string-tagcloud should be more lenient in validating its results, since they depend on Math.log, which isn't formally specified
246         https://bugs.webkit.org/show_bug.cgi?id=114895
247
248         Reviewed by Michael Saboff.
249
250         * tests/sunspider-1.0/string-tagcloud.js:
251
252 2013-04-18  Filip Pizlo  <fpizlo@apple.com>
253
254         Whenever it is cheap and non-invasive, SunSpider tests should validate their results to ensure that the browser runs them correctly
255         https://bugs.webkit.org/show_bug.cgi?id=114852
256
257         Reviewed by Geoffrey Garen.
258         
259         This adds low-overhead checking of the results of each SunSpider tests. This is not
260         meant to be an exhaustive test that everything was executed correctly, but rather a
261         simple sanity check that will catch glaring mistakes. The philosophy here is that we're
262         not trying to prevent all forms of cheating, or that we're trying to prove the
263         browser's correctness. Moreover, these checks are meant to incur minimal overhead;
264         they currently clock in at <2% slow-down in SunSpider 1.0.
265         
266         The test expectations were generated using the LLInt on command-line, and I've verified
267         that Firefox 20, Chrome 26, IE 10, Safari 6, and ToT with all of the JITs agree. Note
268         that some tests cannot be precisely validated because they use Math functions, which
269         ECMAScript chooses to not formally specify - those functions like sin() and friends are
270         allowed to return implementation-dependent results. Also some tests cannot be validated
271         at all because their behavior is either intentionally random or is timezone-dependent.
272         But 23 out of 26 tests now have some kind of validation.
273         
274         I've updated the harnesses to show erroneous runs when displaying results.
275
276         * resources/TEMPLATE.html:
277         * resources/driver-TEMPLATE.html:
278         * resources/sunspider-analyze-results.js:
279         (formatResult):
280         (resultLine):
281         (printOutput):
282         * resources/sunspider-compare-results.js:
283         (.formatMean):
284         (.resultLine):
285         (.printOutput):
286         (sunspiderCompareResults):
287         * resources/sunspider-standalone-driver.js:
288         * tests/sunspider-1.0/3d-cube.js:
289         (Init):
290         * tests/sunspider-1.0/3d-morph.js:
291         * tests/sunspider-1.0/3d-raytrace.js:
292         * tests/sunspider-1.0/access-binary-trees.js:
293         * tests/sunspider-1.0/access-fannkuch.js:
294         * tests/sunspider-1.0/access-nbody.js:
295         * tests/sunspider-1.0/access-nsieve.js:
296         (sieve):
297         * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
298         * tests/sunspider-1.0/bitops-bits-in-byte.js:
299         * tests/sunspider-1.0/bitops-bitwise-and.js:
300         * tests/sunspider-1.0/bitops-nsieve-bits.js:
301         * tests/sunspider-1.0/controlflow-recursive.js:
302         * tests/sunspider-1.0/crypto-aes.js:
303         * tests/sunspider-1.0/crypto-md5.js:
304         * tests/sunspider-1.0/crypto-sha1.js:
305         * tests/sunspider-1.0/date-format-tofte.js:
306         * tests/sunspider-1.0/date-format-xparb.js:
307         * tests/sunspider-1.0/math-cordic.js:
308         * tests/sunspider-1.0/math-partial-sums.js:
309         (partial):
310         * tests/sunspider-1.0/math-spectral-norm.js:
311         * tests/sunspider-1.0/regexp-dna.js:
312         * tests/sunspider-1.0/string-base64.js:
313         * tests/sunspider-1.0/string-fasta.js:
314         (fastaRepeat):
315         (fastaRandom):
316         * tests/sunspider-1.0/string-tagcloud.js:
317         * tests/sunspider-1.0/string-unpack-code.js:
318         * tests/sunspider-1.0/string-validate-input.js:
319
320 2013-04-11  Filip Pizlo  <fpizlo@apple.com>
321
322         Get rid of the 10ms delay between test executions
323         https://bugs.webkit.org/show_bug.cgi?id=114458
324         <rdar://problem/13631112>
325
326         Reviewed by Darin Adler and Geoffrey Garen.
327         
328         More details on this in the bug. Long story short, I don't know why the 10ms delay is
329         there, and it appears to be the root of some noisyness and unreliability in the results.
330         Removing it makes results more consistent, which is a good thing.
331
332         * resources/driver-TEMPLATE.html:
333
334 2013-01-16  Eric Seidel  <eric@webkit.org>
335
336         Remove --shark* support from sunspider/run-sunspider now that Shark is dead
337         https://bugs.webkit.org/show_bug.cgi?id=99512
338
339         Reviewed by Darin Adler.
340
341         I tried to use --instruments, but that also seems broken (in the same way before/after this patch).
342
343         * sunspider:
344         (runTestsOnce):
345
346 2012-08-02  Filip Pizlo  <fpizlo@apple.com>
347
348         Unreviewed, revert debugging stuff I landed in http://trac.webkit.org/changeset/124557
349         by accident.
350
351         * tests/v8-v6/v8-crypto.js:
352         (am1):
353         (am2):
354         (am3):
355         (am4):
356         (bnpCopyTo):
357         (bnpFromString):
358         (bnpClamp):
359
360 2012-03-11  Maciej Stachowiak  <mjs@apple.com>
361
362         Add --instruments option to sunspider to profile with Instruments
363         https://bugs.webkit.org/show_bug.cgi?id=80783
364
365         Reviewed by Filip Pizlo.
366
367         * resources/TimeProfile20us.tracetemplate: Added.
368         * sunspider:
369         (runTestsOnce):
370
371 2011-11-15  Andy Wingo  <wingo@igalia.com>
372
373         Extend sunspider driver to be able to run kraken
374         https://bugs.webkit.org/show_bug.cgi?id=71799
375
376         Reviewed by Filip Pizlo.
377
378         * resources/sunspider-standalone-driver.js: Try to load a -data
379         file.  If that succeeds, we have a kraken-like test, so we time
380         the test using `load'.  Otherwise fall back to using `run'.
381
382 2011-08-11  Mark Rowe  <mrowe@apple.com>
383
384         Fix sunspider-compare-results to default to the same suite version as the sunspider script.
385
386         Rubber-stamped by Maciej Stachowiak.
387
388         * sunspider-compare-results:
389
390 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
391
392         Update SunSpider versioning to 1.0
393         https://bugs.webkit.org/show_bug.cgi?id=63870
394
395         Reviewed by Dan Bernstein.
396
397         * hosted/sunspider.html:
398         * hosted/versions.html:
399         * resources/driver-TEMPLATE.html:
400         * resources/results-TEMPLATE.html:
401         * sunspider:
402
403 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
404
405         Sunspider 0.9.1 harness doesn't actually close() its test documents
406         https://bugs.webkit.org/show_bug.cgi?id=47045
407
408         Reviewed by Daniel Bates.
409
410         * hosted/versions.html:
411         * resources/driver-TEMPLATE.html:
412
413 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
414
415         Not Reviewed. 
416         
417         Fix obviou typo in previous commit.
418
419         * tests/sunspider-1.0/bitops-bitwise-and.js:
420
421 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
422
423         Reviewed by Anders Carlsson.
424
425         SunSpider: all four bitops benchmarks can be replaced with NOP
426         https://bugs.webkit.org/show_bug.cgi?id=38446
427
428         * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
429         (TimeFunc): Save the result.
430         * tests/sunspider-1.0/bitops-bits-in-byte.js:
431         (TimeFunc): Save the result.
432         * tests/sunspider-1.0/bitops-bitwise-and.js: More explicitly
433         save the results.
434         * tests/sunspider-1.0/bitops-nsieve-bits.js:
435         (sieve): Save the result.
436
437 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
438
439         SunSpider: string-validate-input.js uses an undeclared variable named 'name', which is a DOM API
440         https://bugs.webkit.org/show_bug.cgi?id=60937
441
442         Reviewed by Dan Bernstein.
443
444         * tests/sunspider-1.0/string-validate-input.js:
445         (doTest): Rename the variable to username
446
447 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
448
449         Reviewed by Darin Adler.
450
451         SunSpider: The main function of math-cordic is dead code
452         https://bugs.webkit.org/show_bug.cgi?id=63863
453         
454         Try to prevent math-cordic from being eliminated as dead code or
455         compiled down to a constant.
456
457         * tests/sunspider-1.0/math-cordic.js:
458         (cordicsincos):
459         (cordic):
460
461 2011-06-30  Maciej Stachowiak  <mjs@apple.com>
462
463         Reviewed by Adam Barth.
464
465         Create sunspider-1.0 directory in preparation for fixing a bunch of bugs
466         https://bugs.webkit.org/show_bug.cgi?id=63782
467
468         * make-hosted: Learn about the new directory.
469         * tests/sunspider-1.0: Copied from PerformanceTests/SunSpider/tests/sunspider-0.9.1.
470
471 2010-11-29  Geoffrey Garen  <ggaren@apple.com>
472
473         Reviewed by Gavin Barraclough.
474
475         Improved accuracy of command-line SunSpider.
476
477         * resources/sunspider-standalone-driver.js: Use the returned number of
478         elapsed milliseconds from "run" and "checkSyntax" instead of measuring
479         ourselves, for slightly more accurate numbers.
480
481 2010-09-16  Gavin Barraclough  <barraclough@apple.com>
482
483         Reviewed by Mark Rowe.
484         https://bugs.webkit.org/show_bug.cgi?id=45924
485
486         Modify the SunSpider harness to allow a path to be passed to the --suite
487         argument, supress the normal prepend-"tests/" behaviour for suite names
488         containing a '/'.
489
490         Also wrap the code in an anonymous function to move variables out of global
491         scope (currently the harness cannot handle running tests the overwrite the
492         variable 'j').
493
494         * resources/sunspider-standalone-driver.js:
495         * sunspider:
496
497 2010-08-27  Michael Saboff  <msaboff@apple.com>
498
499         Reviewed by Stephanie Lewis.
500
501         Added version 5 and version 6 of V8 tests with modifications similar
502         to the version 4 tests so they'll work with the SunSpider test harness.
503         Note that these files are external source and therefore don't conform 
504         to WebKit coding standards.
505         https://bugs.webkit.org/show_bug.cgi?id=44638
506
507         * tests/v8-v5: Added.
508         * tests/v8-v5/LIST: Added.
509         * tests/v8-v5/v8-crypto.js: Added.
510         * tests/v8-v5/v8-deltablue.js: Added.
511         * tests/v8-v5/v8-earley-boyer.js: Added.
512         * tests/v8-v5/v8-raytrace.js: Added.
513         * tests/v8-v5/v8-regexp.js: Added.
514         * tests/v8-v5/v8-richards.js: Added.
515         * tests/v8-v5/v8-splay.js: Added.
516         * tests/v8-v6: Added.
517         * tests/v8-v6/LIST: Added.
518         * tests/v8-v6/v8-crypto.js: Added.
519         * tests/v8-v6/v8-deltablue.js: Added.
520         * tests/v8-v6/v8-earley-boyer.js: Added.
521         * tests/v8-v6/v8-raytrace.js: Added.
522         * tests/v8-v6/v8-regexp.js: Added.
523         * tests/v8-v6/v8-richards.js: Added.
524         * tests/v8-v6/v8-splay.js: Added.
525
526 2010-07-15  Stephanie Lewis  <slewis@apple.com>
527
528         Reviewed by Geoff Garen.
529
530         http://bugs.webkit.org/show_bug.cgi?id=42406
531         Output location of results file.
532
533         * sunspider:
534
535 2010-05-17  Maciej Stachowiak  <mjs@apple.com>
536
537         Reviewed by Mark Rowe.
538
539         Browser-hosted version of SunSpider runs tests twice as many times as advertised
540         https://bugs.webkit.org/show_bug.cgi?id=39201
541
542         * make-hosted: Make sure to reset test list after every test suite.
543
544 2010-05-07  Adam Roben  <aroben@apple.com>
545
546         Fix typo in sunspider-compare-results.js that caused incorrect
547         two-sample t statistics to be computed
548
549         Fixes <http://webkit.org/b/38768> SunSpider computes incorrect
550         two-sample t statistics
551
552         Reviewed by Darin Adler.
553
554         * resources/sunspider-compare-results.js:
555         (sunspiderCompareResults.resultLine): Fixed the calculation of the t
556         variable to match the definition of a two-sample t statistic. Looks
557         like this was just a simple typo.
558
559 2010-04-05  Darin Adler  <darin@apple.com>
560
561         Updated the ignore property for changes to the contents here.
562
563         * .: Modified property svn:ignore.
564
565 2010-02-20  Maciej Stachowiak  <mjs@apple.com>
566
567         Reviewed by Adam Barth.
568
569         Add sunspider-0.9.1 to the site in preparation for announcing it.
570         https://bugs.webkit.org/show_bug.cgi?id=35206
571
572         * hosted/sunspider.css: Remove tabs, which I noticed where present while
573         copying to the WebKitSite directory.
574
575 2010-01-11  Oliver Hunt  <oliver@apple.com>
576
577         Reviewed by Darin Adler.
578
579         Sunspider parse-only tests don't run
580         https://bugs.webkit.org/show_bug.cgi?id=33489
581
582         Fix parse-only test list
583
584         * tests/parse-only/LIST:
585
586 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
587
588         Reviewed by Adele Peterson.
589
590         Make SunSpider version more prominent in the title
591         https://bugs.webkit.org/show_bug.cgi?id=32574
592
593         * hosted/sunspider.html:
594         * hosted/versions.html:
595         * resources/driver-TEMPLATE.html:
596         * resources/results-TEMPLATE.html:
597
598 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
599
600         Reviewed by Darin Adler.
601
602         Make SunSpider refuse to compare results across content versions
603         https://bugs.webkit.org/show_bug.cgi?id=32573
604         
605         The results URL now includes a version indicator, and compare mode
606         will refuse to compare between versions.
607
608         * resources/driver-TEMPLATE.html:
609         * resources/results-TEMPLATE.html:
610
611 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
612
613         Reviewed by Darin Adler.
614
615         Some Browser-hosted SunSpider files are not valid HTML5
616         https://bugs.webkit.org/show_bug.cgi?id=32536
617         
618         Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
619         close tags and adding a meta charset declaration).
620
621         * hosted/sunspider.html:
622         * hosted/versions.html:
623         * resources/TEMPLATE.html:
624         * resources/driver-TEMPLATE.html:
625         * resources/results-TEMPLATE.html:
626
627 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
628
629         Reviewed by Eric Seidel.
630
631         Make sunspider-0.9.1 the default content set (both command-line and hosted)
632         https://bugs.webkit.org/show_bug.cgi?id=32537
633
634         * hosted/sunspider.html: Change default suite link.
635         * sunspider: Change default suite.
636         * sunspider-compare-results: Likewise.
637
638 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
639
640         Reviewed by Sam Weinig.
641
642         in-browser SunSpider suffers excessive penalty under power management
643         https://bugs.webkit.org/show_bug.cgi?id=32505
644         
645         I have made a few changes to address this:
646         
647         1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
648         document.write to reduce triggering of progress UI.
649         2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
650         stepped all the way down the whole time.
651         3) Run the test cycle an extra time for warmup before the runs that count.
652
653         * hosted/sunspider-record-result.js: Removed.
654         * make-hosted:
655         * resources/TEMPLATE.html:
656         * resources/driver-TEMPLATE.html:
657
658 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
659
660         Reviewed by Gavin Barraclough.
661
662         SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
663         https://bugs.webkit.org/show_bug.cgi?id=16806
664
665         Based on a patch by Eric Seidel.
666         
667         Fix the base64 computation to actually compute correct results. The impact on runtime of
668         the test is pretty small, but noticeable for some browsers. But at least it's not
669         doing a wrong and meaningless computation any more.
670         
671         * tests/sunspider-0.9.1/string-base64.js:
672         ():
673         (base64ToString):
674
675 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
676
677         Fixing commit error...
678         
679         I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
680
681         * resources/driver-TEMPLATE.html:
682
683 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
684
685         Reviewed by Sam Weinig.
686
687         Use JSON.parse instead of eval for Web-hosted SunSpider results processing
688         https://bugs.webkit.org/show_bug.cgi?id=32490
689
690         * hosted/json2.js: Added. Incorporated from json.org
691         * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
692         * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
693         results.
694
695 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
696
697         Reviewed by Oliver Hunt.
698
699         Enable Web-hosted version of SunSpider to handle multiple versions
700         https://bugs.webkit.org/show_bug.cgi?id=32478
701
702         * make-hosted: Now generate the hosted version to be able to run both the
703         0.9 and the 0.9.1 test suites.
704         * hosted: Modified property svn:ignore.
705         * hosted/sunspider-driver.html: Removed.
706         * hosted/sunspider-results.html: Removed.
707         * hosted/sunspider.html:
708         * hosted/versions.html: Added.
709         * resources/TEMPLATE.html:
710         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
711         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
712
713 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
714
715         Reviewed by Oliver Hunt.
716
717         Give command-line sunspider the ability to handle multiple suites and versions
718         https://bugs.webkit.org/show_bug.cgi?id=32477
719         
720         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
721         enumerating the tests. Current suites are:
722         
723         sunspider-0.9 - riginal SunSpider content 
724         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
725         ubench - "ubench" microbenchmark suite
726         v8-v4 - version 4 of the v8 benchmark
727         parse-only - some parser tests
728         
729         Sunspider can now be run with the --suite= parameter to select a
730         suite. The default is --suite=sunspider-0.9.
731
732         * .: Modified property svn:ignore to ignore new results directories.
733         * make-hosted: Generate based on sunspider-0.9 suite.
734         * resources/TEMPLATE.html: Fix an HTML compliance problem.
735         * resources/sunspider-standalone-driver.js: Become aware of suite names.
736         * sunspider: Modified to handle multiple suites as described above.
737         * sunspider-compare-results: ditto
738         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
739         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
740         * tests/LIST-UBENCH: Removed.
741         * tests/LIST-V8: Removed.
742         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
743         * tests/3d-morph.js: ditto
744         * tests/3d-raytrace.js: ditto
745         * tests/access-binary-trees.js: ditto
746         * tests/access-fannkuch.js: ditto
747         * tests/access-nbody.js: ditto
748         * tests/access-nsieve.js: ditto
749         * tests/bitops-3bit-bits-in-byte.js: ditto
750         * tests/bitops-bits-in-byte.js: ditto
751         * tests/bitops-bitwise-and.js: ditto
752         * tests/bitops-nsieve-bits.js: ditto
753         * tests/controlflow-recursive.js: ditto
754         * tests/crypto-aes.js: ditto
755         * tests/crypto-md5.js: ditto
756         * tests/crypto-sha1.js: ditto
757         * tests/date-format-tofte.js: ditto
758         * tests/date-format-xparb.js: ditto
759         * tests/math-cordic.js: ditto
760         * tests/math-partial-sums.js: ditto
761         * tests/math-spectral-norm.js: ditto
762         * tests/regexp-dna.js: ditto
763         * tests/string-base64.js: ditto
764         * tests/string-fasta.js: ditto
765         * tests/string-tagcloud.js: ditto
766         * tests/string-unpack-code.js: ditto
767         * tests/string-validate-input.js: ditto
768         * tests/sunspider-0.9/: Added
769         * tests/sunspider-0.9/*.js: Moved from one level up
770         * tests/sunspider-0.9.1: Added.
771         * tests/sunspider-0.9.1/*.js: Moved from one level up.
772         * tests/v8-crypto.js: Moved to v8-v4/ directory.
773         * tests/v8-deltablue.js: ditto
774         * tests/v8-earley-boyer.js: ditto
775         * tests/v8-raytrace.js: ditto
776         * tests/v8-regexp.js: ditto
777         * tests/v8-richards.js: ditto
778         * tests/v8-splay.js: ditto
779         * tests/v8-v4: Added.
780
781 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
782
783         Reviewed by Darin Adler.
784
785         Platform specific null device replaced with a platform independent.
786         https://bugs.webkit.org/show_bug.cgi?id=29544
787
788         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
789
790 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
791
792         Reviewed by Maciej Stachowiak.
793
794         [Qt] Make sunspider script work on Windows platform.
795         https://bugs.webkit.org/show_bug.cgi?id=29544
796
797         * sunspider:
798         Using platform specific null device instead of /dev/null.
799
800 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
801
802         Reviewed by Oliver Hunt.
803
804         https://bugs.webkit.org/show_bug.cgi?id=26540
805         Currently the SunSpider test driver lacks an option to run a test suite that
806         will test JavaScriptCore parsing performance only.  This patch adds just such
807         a test suite and option to SunSpider as well as the jsc test shell.  I've included
808         three large javascript source files found in the wild: jquery, mootools and prototype.
809         Combined with the concatenation of all three, these form a new testsuite to measure
810         and test pure JavaScriptCore parsing performance.
811
812         * resources/sunspider-standalone-driver.js:
813           Load the js with parse-only flag if it is part of the parse-only suite.
814         * sunspider:
815           Add the parse-only suite to the list of options.
816         * sunspider-compare-results:
817           Add the parse-only suite to the list of options.
818         * tests/LIST-PARSE-ONLY: Added.
819         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
820           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
821         * tests/parse-only/jquery-1.3.2.js: Added.
822           Latest version from jquery.com.
823         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
824           Latest version from mootools.net.
825         * tests/parse-only/prototype-1.6.0.3.js: Added.
826           Latest version from prototypejs.org.
827
828 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
829
830         Update V8 benchmark to version 4.
831         
832         Reviewed by Cameron Zwarich.
833
834         * tests/LIST-V8:
835         * tests/v8-crypto.js:
836         * tests/v8-deltablue.js:
837         * tests/v8-earley-boyer.js:
838         * tests/v8-raytrace.js:
839         * tests/v8-regexp.js: Added.
840         * tests/v8-richards.js:
841         * tests/v8-splay.js: Added.
842
843 2009-05-26  Olivier DOLE  <odole@pleyo.com>
844
845         Reviewed by Maciej.
846
847         Store sunspider test suite results in a specific directory for each
848         different type of benchmark (sunspider, V8 or ubench).
849
850         * sunspider:
851         * sunspider-compare-results:
852
853 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
854
855         Build fix.
856
857         * SunSpider.make:
858
859 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
860
861         Include SunSpider tests in production builds.
862         
863         Reviewed by Adam Roben.
864
865         * SunSpider.make: Added.
866         * xcopy.excludes: Added.
867
868 2009-03-04  Oliver Hunt  <oliver@apple.com>
869
870         Added a few more tests to ubench
871
872         RS=Gavin Barraclough
873
874         Adding a couple of test to cover too few and too many arguments.  This
875         is needed to prevent regressions in all call paths from my current refactoring.
876
877         * tests/LIST-UBENCH:
878         * tests/ubench/function-correct-args.js: Added.
879         (f):
880         * tests/ubench/function-excess-args.js: Added.
881         (f):
882
883 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
884
885         Fix a bug where when V8's or UBENCH's test suite list file is missing
886         the error message is suggesting that SunSpider's list file is missing.
887
888         Reviewed by Sam Weinig.
889
890         * sunspider:
891         (loadTestsList): Fix the error message to use the right list name.
892
893 2008-10-07  David Hyatt  <hyatt@apple.com>
894
895         https://bugs.webkit.org/show_bug.cgi?id=18989
896
897         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
898         we want all browsers to be doing the same thing on the test.
899
900         Reviewed by Maciej
901
902         * tests/regexp-dna.js:
903
904 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
905
906         Rubber-stamped by Maciej Stachowiak.
907
908         The bleeding edge branch of V8 now supports passing fileanmes after
909         '-f', so the SunSpider script should be changed to do this again.
910         This allows all 3 major JavaScript engines to be used with the
911         unmodified SunSpider script.
912
913         * sunspider:
914
915 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
916
917         Reviewed by Maciej Stachowiak.
918
919         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
920         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
921         correct the punctuation in its documentation.
922
923         * sunspider:
924
925 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
926
927         Reviewed by Oliver Hunt.
928
929         Bug 20616: Incorporate V8 benchmarks in testing
930         <https://bugs.webkit.org/show_bug.cgi?id=20616>
931
932         This is a first attempt at running the V8 benchmarks within SunSpider,
933         with the --v8 option. The number of iterations is fixed for each
934         benchmark, and was chosen to make each of the benchmarks run between
935         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
936
937         * sunspider:
938         * tests/LIST-V8: Added.
939         * tests/v8-crypto.js: Added.
940         * tests/v8-deltablue.js: Added.
941         * tests/v8-earley-boyer.js: Added.
942         * tests/v8-raytrace.js: Added.
943         * tests/v8-richards.js: Added.
944
945 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
946
947         Not reviewed.
948
949         Remove a local change that was accidentally committed in r36034.
950
951         * resources/sunspider-standalone-driver.js:
952
953 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
954
955         Rubber-stamped by Oliver Hunt.
956
957         Make SunSpider work with JavaScript shells that don't take filenames
958         after a '-f' argument.
959
960         * sunspider:
961
962 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
963
964         Reviewed by Timothy.
965
966         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
967
968         * sunspider:
969         * tests/LIST-SQUIRRELFISH: Removed.
970
971 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
972
973         Rubber-stamped by Mark Rowe.
974
975         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
976
977         * sunspider:
978
979 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
980
981         Rubber stamped by Oliver.
982         
983         - add newly working tests to squirrelfish list
984
985         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
986
987 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
988
989         Rubber stamped by Oliver.
990         
991         - move ubench tests properly.
992
993         * tests/ubench: Added.
994         * tests/ubench/function-closure.js: Added.
995         * tests/ubench/function-empty.js: Added.
996         * tests/ubench/function-missing-args.js: Added.
997         * tests/ubench/function-sum.js: Added.
998         * tests/ubench/loop-empty-resolve.js: Added.
999         * tests/ubench/loop-empty.js: Added.
1000         * tests/ubench/loop-sum.js: Added.
1001
1002 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
1003
1004         Rubber stamped by Oliver.
1005
1006         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
1007
1008 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
1009
1010         Reviewed by Oliver.
1011
1012         - make --squirrelfish mode use the real harness, but a reduced set of tests
1013
1014         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
1015
1016         * sunspider:
1017         * tests/LIST-SQUIRRELFISH:
1018         * tests/LIST-UBENCH: Added.
1019         * tests/squirrelfish: Moved to tests/ubench
1020
1021 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
1022
1023         Reviewed by Maciej Stachowiak.
1024         
1025         Removed an unused test.
1026
1027         * tests/squirrelfish/loop-resolve.js: Removed.
1028
1029 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
1030
1031         Reviewed by Oliver.
1032         
1033         Add newly runnable tests to --squirrelfish mode.
1034
1035         * tests/LIST-SQUIRRELFISH:
1036
1037 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
1038
1039         Reviewed by Maciej Stachowiak.
1040         
1041         A little more love for --squirrelfish mode:
1042         
1043         Fixed a misplaced sort. Results properly sort now.
1044         
1045         Added a test list just for squirrelfish, and updated it to include
1046         all currently passing SunSpider tests.
1047         
1048         Fixed the pruning regexp to match 3d-morph.js.
1049
1050         * sunspider:
1051
1052 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
1053
1054         Reviewed by Oliver Hunt.
1055         
1056         --squirrelfish mode: pared down tests for squirrelfish to chew on.
1057
1058         * sunspider:
1059         * tests/LIST:
1060         * tests/squirrelfish: Added.
1061         * tests/squirrelfish/function-closure.js: Added.
1062         * tests/squirrelfish/function-empty.js: Added.
1063         * tests/squirrelfish/function-missing-args.js: Added.
1064         * tests/squirrelfish/function-sum.js: Added.
1065         * tests/squirrelfish/loop-empty-resolve.js: Added.
1066         * tests/squirrelfish/loop-empty.js: Added.
1067         * tests/squirrelfish/loop-resolve.js: Added.
1068         * tests/squirrelfish/loop-sum.js: Added.
1069
1070 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
1071
1072         Reviewed by Geoff.
1073
1074         * sunspider: Pause update daemon on Darwin to make results reliable
1075         again (workaround for <rdar://problem/5811127>).
1076
1077 2008-02-07  Eric Seidel  <eric@webkit.org>
1078
1079         Reviewed by Sam.
1080
1081         * sunspider: add an error message when ./tests/LIST is not found
1082
1083 2007-12-29  Alp Toker  <alp@atoker.com>
1084
1085         Fix typo.
1086
1087         * hosted/sunspider.html:
1088
1089 2007-12-29  Alp Toker  <alp@atoker.com>
1090
1091         Reviewed by Maciej.
1092
1093         Provide the current URL in the SunSpider results page. Useful for
1094         copying benchmark results out of browsers without a location bar or
1095         where the location bar can't handle long URLs.
1096
1097         * hosted/sunspider-results.html:
1098
1099 2007-12-20  Eric Seidel  <eric@webkit.org>
1100
1101         Reviewed by mjs.
1102
1103         * sunspider: increase default number of runs for more accurate numbers
1104
1105 2007-12-18  Mark Rowe  <mrowe@apple.com>
1106
1107         Fix typo.
1108
1109         * hosted/sunspider.html:
1110
1111 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
1112
1113         Reviewed by Eric.
1114         
1115         - give an extra digit of precision for small differences.
1116
1117         * resources/sunspider-compare-results.js:
1118
1119 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
1120
1121         Reviewed by Eric.
1122         
1123         - work around IE layout bugs
1124
1125         * hosted/sunspider.css:
1126
1127 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
1128
1129         Reviewed by Mark.
1130         
1131         - fix compare feature
1132
1133         * hosted/sunspider-results.html:
1134
1135 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
1136
1137         Reviewed by Mark.
1138         
1139         - Change the format of the output URL to be shorter, to meet IE constraints
1140
1141         * hosted/sunspider-driver.html:
1142         * hosted/sunspider-results.html:
1143
1144 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
1145
1146         Reviewed by Oliver.
1147         
1148         - Add some explanatory test and a dumb logo to SunSpider web hosted version
1149
1150         * hosted/sunspider-driver.html:
1151         * hosted/sunspider-results.html:
1152         * hosted/sunspider.css:
1153         * hosted/sunspider.html:
1154
1155 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
1156
1157         Reviewed by Oliver.
1158         
1159         - Make the web-hosted version of SunSpider look a little bit less drab.
1160
1161         * hosted/sunspider-driver.html:
1162         * hosted/sunspider-results.html:
1163         * hosted/sunspider.css: Added.
1164         * hosted/sunspider.html:
1165         * resources/TEMPLATE.html:
1166
1167 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
1168
1169         Reviewed by Sam.
1170         
1171         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
1172         
1173         A sample of what new-style results would look like:
1174         
1175         TEST                   COMPARISON            FROM                 TO             DETAILS
1176         ...
1177         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
1178         ...
1179             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
1180         
1181         * resources/sunspider-compare-results.js:
1182
1183 2007-11-14  Eric Seidel  <eric@webkit.org>
1184
1185         Reviewed by Sam.
1186
1187         * sunspider: add --shark-cache for L2 Cache Miss profiling
1188
1189 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
1190
1191         Reviewed by Darin Adler.
1192
1193         http://bugs.webkit.org/show_bug.cgi?id=15925
1194         SunSpider should check for Shark being installed
1195
1196         * sunspider: Explicitly test that shark command line tool is installed.
1197
1198 2007-11-04  Darin Adler  <darin@apple.com>
1199
1200         * hosted/sunspider.html: Tweaked formatting a bit.
1201
1202 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
1203
1204         Reviewed by Darin Adler.
1205
1206         - Add compare mode and a description of the benchmark to the hosted version.
1207
1208         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
1209         * hosted/sunspider.html: Added intro text.
1210         * make-hosted: Copy new file.
1211         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
1212         a function.
1213         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
1214         expects.
1215         * sunspider-compare-results: Use sunspider-standalone-compare.js.
1216
1217 2007-11-02  Darin Adler  <darin@apple.com>
1218
1219         * sunspider: Changed "--base" to "--set-baseline".
1220
1221 2007-11-02  Darin Adler  <darin@apple.com>
1222
1223         Reviewed by Maciej.
1224
1225         * sunspider: Added a "--base" option that records the current test as a baseline.
1226         Moved the code to find the newest .mshark file into a function (since I needed to use
1227         it in the other script and I prefer to copy/paste a function).
1228
1229         * sunspider-compare-results: Added logic to compare the baseline with the last
1230         run when you don't pass any parameters.
1231
1232 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
1233
1234         Reviewed by Eric.
1235         
1236         - Added list of ECMAScript3 features that SunSpider does not cover currently.
1237
1238         * UNCOVERED: Added.
1239
1240 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
1241
1242         Reviewed by Oliver.
1243         
1244         - Give better error messages when sunspider-compare-results fails
1245
1246         * sunspider-compare-results:
1247
1248 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
1249
1250         Reviewed by Darin Adler.
1251         
1252         - fix Date tests to work in command-line SpiderMonkey
1253
1254         * tests/date-format-tofte.js:
1255         * tests/date-format-xparb.js:
1256
1257 2007-10-29  Darin Adler  <darin@apple.com>
1258
1259         Reviewed by Maciej.
1260
1261         * sunspider: Added --shark20 option, to run Shark at its highest sample
1262         resolution instead of its default.
1263
1264 2007-10-28  Adam Roben  <aroben@apple.com>
1265
1266         Fix SunSpider on Windows
1267
1268         Reviewed by Eric.
1269
1270         * sunspider: Don't use colons in filenames.
1271
1272 2007-10-28  Darin Adler  <darin@apple.com>
1273
1274         Reviewed by Adam.
1275
1276         * resources/sunspider-analyze-results.js: Tweak the output format a little.
1277         Change so that when there's only one run we don't write out confidence
1278         intervals at all rather than writing "NaN" over and over again.
1279
1280         * sunspider: Use the time and date as a suffix on the results file, that way
1281         all the results are left behind in the tmp directory. This will make it easy
1282         for us to add features that compare past results in the future. Also rename
1283         the .mshark files using the same scheme.
1284
1285         * sunspider-compare-results: Relax the parsing rules so we can parse the
1286         results file exactly as it's emitted from sunspider.
1287
1288 2007-10-25  Adam Roben  <aroben@apple.com>
1289
1290         Strip carriage returns from results
1291
1292         Carriage returns were screwing up sunspider-compare-results on
1293         Windows.
1294
1295         Rubberstamped by Oliver.
1296
1297         * sunspider:
1298
1299 2007-10-24  Darin Adler  <darin@apple.com>
1300
1301         Reviewed by Eric.
1302
1303         * sunspider: Open the profile in shark after completing a run.
1304         * sunspider-compare-results: Tiny perl tweaks.
1305
1306 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
1307
1308         Reviewed by Darin Adler.
1309
1310         - Don't hardcode my path to testkjs
1311         
1312         * sunspider-compare-results:
1313
1314 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
1315
1316         Reviewed by Darin Adler.
1317         
1318         - Add a compare script to compare results, and improve formatting of standard results a bit.
1319
1320         * resources/sunspider-analyze-results.js:
1321         * resources/sunspider-compare-results.js: Added.
1322         * sunspider-compare-results: Added.
1323
1324 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
1325
1326         Reviewed by Darin Adler.
1327
1328         * resources/sunspider-analyze-results.js: Do the error range
1329         properly, using the t-distribution instead of 1.96 (which was
1330         based on the normal distribution).
1331         
1332         * sunspider: Print results in a way that will be friendlier to the
1333         soon-to-come compare mode.
1334
1335 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
1336
1337         Reviewed by Review.
1338         
1339         - add more tests (probably the final set for now)
1340         
1341         Already balanced. Added date, regexp, control flow, and a few
1342         more string and object/array access tests.
1343         
1344         * tests/LIST:
1345         * tests/access-fannkuch.js: Added.
1346         * tests/access-nbody.js: Added.
1347         * tests/controlflow-recursive.js: Added.
1348         * tests/date-format-tofte.js: Added.
1349         * tests/date-format-xparb.js: Added.
1350         * tests/regexp-dna.js: Added.
1351         * tests/string-validate-input.js: Added.
1352
1353 2007-10-21  Eric Seidel  <eric@webkit.org>
1354
1355         Reviewed by mjs.
1356
1357         * sunspider: make --shark only sample from testkjs
1358
1359 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
1360
1361         Rubber stamped by Eric.
1362         
1363         - added crypto tests (already balanced)
1364
1365         * tests/LIST:
1366         * tests/crypto-aes.js: Added.
1367         * tests/crypto-md5.js: Added.
1368         * tests/crypto-sha1.js: Added.
1369
1370 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
1371
1372         Reviewed by Adam.
1373         
1374         - rebalance new tests
1375         
1376         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
1377         
1378         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1379
1380         This balances them with the existing tests.
1381
1382         * tests/access-binary-trees.js:
1383         * tests/access-nsieve.js:
1384         * tests/bitops-nsieve-bits.js:
1385         * tests/math-partial-sums.js:
1386         * tests/math-spectral-norm.js:
1387         * tests/string-fasta.js:
1388
1389 2007-10-20  Eric Seidel  <eric@webkit.org>
1390
1391         Reviewed by darin.
1392         
1393         Add improved argument handling to sunspider, including
1394         --runs=<number>, --tests=<pattern, --shark, and --help
1395         Also re-factor code into subroutines
1396
1397         * sunspider: improved argument handling
1398
1399 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1400
1401         Reviewed by Mark.
1402
1403         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
1404         
1405         * TODO:
1406         * tests/LIST:
1407         * tests/access-binary-trees.js: Added.
1408         * tests/access-nsieve.js: Added.
1409         * tests/bitops-nsieve-bits.js: Added.
1410         * tests/math-partial-sums.js: Added.
1411         * tests/math-spectral-norm.js: Added.
1412         * tests/string-fasta.js: Added.
1413
1414 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1415
1416         Reviewed by Darin Adler.
1417
1418         * TODO: Updated to note areas that are now well-covered.
1419         
1420         - Change the tests and the driver to avoid leaving large object
1421         graphs hanging around, since that throws off the subsequent tests.
1422         
1423         * resources/sunspider-standalone-driver.js:
1424         * tests/3d-cube.js:
1425         * tests/3d-morph.js:
1426         * tests/3d-raytrace.js:
1427         * tests/bitops-bitwise-and.js:
1428         * tests/string-base64.js:
1429         * tests/string-tagcloud.js:
1430
1431 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1432
1433         Reviewed by Nikolas.
1434         
1435         - Rebalance test complexity.
1436         
1437         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
1438         
1439         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1440
1441         However, for some tests, IE7 was a huge outlier compared to other
1442         major browsers. For those tests, I used Firefox 2.0.0.8 on the
1443         same OS and hardware as a normalization basis.
1444         
1445         * tests/3d-morph.js:
1446         * tests/3d-raytrace.js:
1447         * tests/bitops-3bit-bits-in-byte.js:
1448         * tests/bitops-bits-in-byte.js:
1449         * tests/bitops-bitwise-and.js:
1450         * tests/math-cordic.js:
1451         * tests/string-base64.js:
1452         * tests/string-tagcloud.js:
1453         * tests/string-unpack-code.js:
1454
1455 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1456
1457         Reviewed by Oliver.
1458         
1459         - Fix the browser-hosted driver to be compatible with Safari 2
1460
1461         * hosted/sunspider-driver.html:
1462
1463 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1464
1465         Rubber stamped by Hyatt.
1466         
1467         - Add BSD license notice to cordic test, the original author said to do so.
1468
1469         * tests/math-cordic.js:
1470
1471 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1472
1473         Reviewed by Eric.
1474         
1475         - Support Windows and Windows browsers.
1476
1477         * hosted/sunspider-results.html: Display results correctly in IE.
1478         * sunspider: tweaks to work on Windows, and with spaces in the path.
1479
1480 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1481
1482         Rubber stamped by Oliver.
1483
1484         * tests/string-tagcloud.js: Remove stray print()
1485
1486 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
1487
1488         Reviewed by Oliver.
1489         
1490         Add some string test cases.
1491
1492         * tests/string-base64.js: Added. Does base64 encoding and decoding.
1493         * tests/string-tagcloud.js: Added. Parses a JSON string and
1494         generates tagcloud HTML markup based on the data.        
1495         * tests/string-unpack-code.js: Added. Unpack compressed versions of
1496         four of the most popular JavaScript libraries.
1497         * tests/LIST: Added the new tests.
1498
1499 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
1500
1501         Reviewed by Anders.
1502
1503         * TODO: Updated with info on how to do the stats properly.
1504
1505         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
1506         avoid overly slowing down the benchmark.
1507
1508         Added some 3D-related test cases.
1509         
1510         * tests/3d-cube.js: Added.
1511         * tests/3d-morph.js: Added.
1512         * tests/3d-raytrace.js: Added.
1513         * tests/LIST:
1514
1515 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
1516
1517         Reviewed by Darin Adler.
1518
1519         - New JavaScript benchmark
1520         http://bugs.webkit.org/show_bug.cgi?id=15515
1521         
1522         A start on a new JavaScript benchmark, with standalone and
1523         browser-hosted drivers.
1524         
1525         The test content is still incomplete and the drivers could use
1526         some more features but there is enough here to make a good start.
1527         
1528         The drivers compute a 95% confidence interval on the mean for the
1529         whole test, each category, and each individual test to make it
1530         easier to tell whether differences are statistically
1531         significant. The confidence interval can be narrowed by running
1532         the test more times.
1533         
1534         * TODO: Added. Remaining things that need to be done.
1535         * sunspider: Added. Perl script that acts as the standalone test driver.
1536         * resources/sunspider-standalone-driver.js: Added. JavaScript part
1537         of standalone test driver.
1538         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
1539         analysis code.
1540         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
1541         * make-hosted: Added. Script to generate browser-hosted tests.
1542         * hosted/sunspider.html: Added. Start page for browser-hosted test.
1543         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
1544         * hosted/sunspider-results.html: Added. Results page for browser-hosted
1545         tests.
1546         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
1547         tests.
1548         * tests/LIST: Added. List of tests to use.
1549         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
1550         * tests/bitops-bits-in-byte.js: Added. Ditto.
1551         * tests/bitops-bitwise-and.js: Added. Ditto.
1552         * tests/math-cordic.js: Added. Ditto.
1553