Failure when building WebKit for appletvsimulator.
[WebKit-https.git] / PerformanceTests / SunSpider / ChangeLog
1 2015-03-30  Michael Saboff  <msaboff@apple.com>
2
3         Fix failing v8-deltablue.js for ARM
4         https://bugs.webkit.org/show_bug.cgi?id=138392
5
6         Rubber-stamped by Mark Lam.
7
8         Re-enable this test for ARM32 on iOS as the tests now run without issues.
9
10         * tests/v8-v6/v8-deltablue.js:
11
12 2015-02-25  Benjamin Poulain  <bpoulain@apple.com>
13
14         CodeBlock crashes when dumping op_push_name_scope
15         https://bugs.webkit.org/show_bug.cgi?id=141953
16
17         Reviewed by Filip Pizlo.
18
19         * profiler-test.yaml:
20
21 2015-01-27  Csaba Osztrogon√°c  <ossy@webkit.org>
22
23         [Win] JSC profiler tests asserts in debug mode
24         https://bugs.webkit.org/show_bug.cgi?id=140931
25
26         Unreviewed gardening, skip failing tests.
27
28         * profiler-test.yaml:
29
30 2014-11-04  Michael Saboff  <msaboff@apple.com>
31
32         Disable failing v8-deltablue.js for ARM
33         https://bugs.webkit.org/show_bug.cgi?id=138382
34
35         Reviewed by Geoffrey Garen.
36
37         This test is failing on ARM 32 bit.
38
39         * tests/v8-v6/v8-deltablue.js:
40
41 2014-07-08  Gergo Balogh  <gbalogh.u-szeged@partner.samsung.com>
42
43         Add option to specify output file location in sunspider.
44         https://bugs.webkit.org/show_bug.cgi?id=134339
45
46         Reviewed by Darin Adler.
47
48         * sunspider:
49
50 2014-06-03  Eva Balazsfalvi  <evab.u-szeged@partner.samsung.com>
51
52         REGRESSION(r169092 and r169102): Skip failing JSC tests poperly on non-x86 Darwin platforms
53         https://bugs.webkit.org/show_bug.cgi?id=133149
54
55         Reviewed by Csaba Osztrogon√°c.
56
57         * profiler-test.yaml: Skip profiler tests only if the architecture isn't x86 and the host is Darwin.
58
59 2014-05-21  Filip Pizlo  <fpizlo@apple.com>
60
61         Unreviewed, roll out http://trac.webkit.org/changeset/169159.
62         
63         This was a unilateral change and wasn't properly reviewed.
64
65         * profiler-test.yaml:
66
67 2014-05-19  Filip Pizlo  <fpizlo@apple.com>
68
69         Take care of some ARM64 test failures
70         https://bugs.webkit.org/show_bug.cgi?id=133090
71
72         Reviewed by Geoffrey Garen.
73
74         * profiler-test.yaml: Skip these on not-x86.
75
76 2014-03-14  Maciej Stachowiak  <mjs@apple.com>
77
78         Replace "Apple Computer, Inc." with "Apple Inc." in copyright headers
79         https://bugs.webkit.org/show_bug.cgi?id=130276
80         <rdar://problem/16266927>
81
82         Reviewed by Simon Fraser.
83
84         * hosted/sunspider.html:
85         * hosted/versions.html:
86         * make-hosted:
87         * resources/TEMPLATE.html:
88         * resources/driver-TEMPLATE.html:
89         * resources/results-TEMPLATE.html:
90         * resources/sunspider-analyze-results.js:
91         * resources/sunspider-compare-results.js:
92         * resources/sunspider-standalone-compare.js:
93         * resources/sunspider-standalone-driver.js:
94         * sunspider:
95         * sunspider-compare-results:
96         * tests/sunspider-0.9.1/3d-morph.js:
97         * tests/sunspider-0.9.1/3d-raytrace.js:
98         * tests/sunspider-0.9.1/bitops-bitwise-and.js:
99         * tests/sunspider-0.9.1/math-cordic.js:
100         * tests/sunspider-0.9.1/string-tagcloud.js:
101         * tests/sunspider-0.9/3d-morph.js:
102         * tests/sunspider-0.9/3d-raytrace.js:
103         * tests/sunspider-0.9/bitops-bitwise-and.js:
104         * tests/sunspider-0.9/math-cordic.js:
105         * tests/sunspider-0.9/string-tagcloud.js:
106         * tests/sunspider-1.0.1/3d-morph.js:
107         * tests/sunspider-1.0.1/3d-raytrace.js:
108         * tests/sunspider-1.0.1/bitops-bitwise-and.js:
109         * tests/sunspider-1.0.1/math-cordic.js:
110         * tests/sunspider-1.0.1/string-tagcloud.js:
111         * tests/sunspider-1.0.2/3d-morph.js:
112         * tests/sunspider-1.0.2/3d-raytrace.js:
113         * tests/sunspider-1.0.2/bitops-bitwise-and.js:
114         * tests/sunspider-1.0.2/math-cordic.js:
115         * tests/sunspider-1.0.2/string-tagcloud.js:
116         * tests/sunspider-1.0/3d-morph.js:
117         * tests/sunspider-1.0/3d-raytrace.js:
118         * tests/sunspider-1.0/bitops-bitwise-and.js:
119         * tests/sunspider-1.0/math-cordic.js:
120         * tests/sunspider-1.0/string-tagcloud.js:
121
122 2014-02-10  Filip Pizlo  <fpizlo@apple.com>
123
124         Unreviewed, revert the accidental change from r163789.
125
126         * tests/v8-v6/v8-deltablue.js:
127
128 2013-12-10  Filip Pizlo  <fpizlo@apple.com>
129
130         Get rid of forward exit on DoubleAsInt32
131         https://bugs.webkit.org/show_bug.cgi?id=125552
132
133         Reviewed by Oliver Hunt.
134         
135         Use SunSpider as a kind of spot-check for the
136         no-architecture-specific-optimization paths in the compiler.
137
138         * no-architecture-specific-optimizations.yaml: Added.
139
140 2013-10-08  Geoffrey Garen  <ggaren@apple.com>
141
142         Refined power management in SunSpider 1.0.2
143         https://bugs.webkit.org/show_bug.cgi?id=122525
144
145         Reviewed by Gavin Barraclough.
146
147         * hosted/sunspider.css:
148         (#frameparent):
149         (#countdown):
150         * resources/TEMPLATE.html: Provide an explicit countdown element in the
151         parent, instead of hacking it into each child frame.
152
153         * resources/driver-TEMPLATE.html: Be more explicit about how we respond
154         to power management. Just moving setTimeout(0) inside the loop is a
155         little too subtle, and may not work across browsers.
156
157         Also, leave a little extra time after changing the countdown graphic,
158         to give the browser some time to paint.
159
160 2013-10-02  Geoffrey Garen  <ggaren@apple.com>
161
162         Improved painting behavior in SunSpider 1.0.2
163         https://bugs.webkit.org/show_bug.cgi?id=122257
164
165         Reviewed by Oliver Hunt.
166
167         Individual test names with times was a good painting strategy years
168         back, when each benchmark took hundreds of milliseconds to run. Now,
169         the whole suite can finish in a a few hundres milliseconds, and things
170         look pretty flashy.
171
172         Instead, display a countdown that indicates which iteration of the
173         test we're on.
174
175         * resources/TEMPLATE.html: Work around a bug in Firefox by specifying
176         our style inside the .html file instead of in an external stylesheet.
177         Under heavy JavaScript load, Firefox neglects to apply external
178         stylesheets -- probably due to a starved timer.
179         
180         * resources/driver-TEMPLATE.html: Delay a little between runs to ensure
181         that our countdown has enough time to paint.
182
183 2013-09-26  Geoffrey Garen  <ggaren@apple.com>
184
185         Fixed a version number string in SunSpider 1.0.2
186         https://bugs.webkit.org/show_bug.cgi?id=122004
187
188         Reviewed by Gavin Barraclough.
189
190         * resources/driver-TEMPLATE.html:
191         * resources/results-TEMPLATE.html: 1.0.2!
192
193 2013-09-26  Geoffrey Garen  <ggaren@apple.com>
194
195         Tentative SunSpider 1.0.2
196         https://bugs.webkit.org/show_bug.cgi?id=122002
197
198         Reviewed by Gavin Barraclough.
199
200         Be even more aggressive about asking for zero delay between tests by
201         setting a timeout at the beginning of a test instead of the end. This is
202         another step toward reducing noise -- particularly noise caused by
203         power management -- just like the step down from 10ms to 0ms.
204
205         Browsers typically put a 4ms floor on timers. The sooner we set
206         our timer, the sooner it will fire.
207
208         * make-hosted:
209         * resources/driver-TEMPLATE.html: Changed the driver to set the timeout
210         earlier.
211         * tests/sunspider-1.0.2: Added. These tests are unchanged from 1.0.1.
212
213 2013-09-09  Filip Pizlo  <fpizlo@apple.com>
214
215         Stress tests should test the jsc profiler (-p)
216         https://bugs.webkit.org/show_bug.cgi?id=121043
217
218         Reviewed by Mark Hahnenberg.
219         
220         Add a jsc-stress-test that tries to profile SunSpider.
221
222         * profiler-test.yaml: Added.
223
224 2013-08-08  Cosmin Truta  <ctruta@blackberry.com>
225
226         SunSpider: Move the 3d-morph fix from version 1.0 to version 1.0.1
227         https://bugs.webkit.org/show_bug.cgi?id=117993
228
229         Reviewed by George Staikos.
230
231         Bump the SunSpider version to 1.0.1.
232
233         Unroll r149548 in order to restore version 1.0 to its original,
234         and add the 3d-morph fix to version 1.0.1 instead.
235
236         * hosted/sunspider.html: Updated to version 1.0.1.
237         * hosted/versions.html: Updated to version 1.0.1.
238         * make-hosted: Added support for version 1.0.1.
239         * resources/driver-TEMPLATE.html: Updated to version 1.0.1.
240         * resources/results-TEMPLATE.html: Updated to version 1.0.1.
241         * sunspider: Updated to version 1.0.1.
242         * sunspider-compare-results: Updated to version 1.0.1.
243         * tests/sunspider-1.0/3d-morph.js: Rolled back to the original version.
244         * tests/sunspider-1.0.1/LIST: Added.
245         * tests/sunspider-1.0.1/3d-cube.js: Added.
246         * tests/sunspider-1.0.1/3d-morph.js: Added.
247         * tests/sunspider-1.0.1/3d-raytrace.js: Added.
248         * tests/sunspider-1.0.1/access-binary-trees.js: Added.
249         * tests/sunspider-1.0.1/access-fannkuch.js: Added.
250         * tests/sunspider-1.0.1/access-nbody.js: Added.
251         * tests/sunspider-1.0.1/access-nsieve.js: Added.
252         * tests/sunspider-1.0.1/bitops-3bit-bits-in-byte.js: Added.
253         * tests/sunspider-1.0.1/bitops-bits-in-byte.js: Added.
254         * tests/sunspider-1.0.1/bitops-bitwise-and.js: Added.
255         * tests/sunspider-1.0.1/bitops-nsieve-bits.js: Added.
256         * tests/sunspider-1.0.1/controlflow-recursive.js: Added.
257         * tests/sunspider-1.0.1/crypto-aes.js: Added.
258         * tests/sunspider-1.0.1/crypto-md5.js: Added.
259         * tests/sunspider-1.0.1/crypto-sha1.js: Added.
260         * tests/sunspider-1.0.1/date-format-tofte.js: Added.
261         * tests/sunspider-1.0.1/date-format-xparb.js: Added.
262         * tests/sunspider-1.0.1/math-cordic.js: Added.
263         * tests/sunspider-1.0.1/math-partial-sums.js: Added.
264         * tests/sunspider-1.0.1/math-spectral-norm.js: Added.
265         * tests/sunspider-1.0.1/regexp-dna.js: Added.
266         * tests/sunspider-1.0.1/string-base64.js: Added.
267         * tests/sunspider-1.0.1/string-fasta.js: Added.
268         * tests/sunspider-1.0.1/string-tagcloud.js: Added.
269         * tests/sunspider-1.0.1/string-unpack-code.js: Added.
270         * tests/sunspider-1.0.1/string-validate-input.js: Added.
271
272 2013-05-03  Cosmin Truta  <ctruta@blackberry.com>
273
274         SunSpider 1.0: 3d-morph: use epsilon to check result
275         https://bugs.webkit.org/show_bug.cgi?id=115553
276
277         Reviewed by George Staikos.
278
279         SunSpider/3d-morph should be more lenient in validating its results, since they depend on Math.sin, which isn't formally specified.
280
281         * tests/sunspider-1.0/3d-morph.js:
282
283 2013-04-19  Filip Pizlo  <fpizlo@apple.com>
284
285         SunSpider/string-tagcloud should be more lenient in validating its results, since they depend on Math.log, which isn't formally specified
286         https://bugs.webkit.org/show_bug.cgi?id=114895
287
288         Reviewed by Michael Saboff.
289
290         * tests/sunspider-1.0/string-tagcloud.js:
291
292 2013-04-18  Filip Pizlo  <fpizlo@apple.com>
293
294         Whenever it is cheap and non-invasive, SunSpider tests should validate their results to ensure that the browser runs them correctly
295         https://bugs.webkit.org/show_bug.cgi?id=114852
296
297         Reviewed by Geoffrey Garen.
298         
299         This adds low-overhead checking of the results of each SunSpider tests. This is not
300         meant to be an exhaustive test that everything was executed correctly, but rather a
301         simple sanity check that will catch glaring mistakes. The philosophy here is that we're
302         not trying to prevent all forms of cheating, or that we're trying to prove the
303         browser's correctness. Moreover, these checks are meant to incur minimal overhead;
304         they currently clock in at <2% slow-down in SunSpider 1.0.
305         
306         The test expectations were generated using the LLInt on command-line, and I've verified
307         that Firefox 20, Chrome 26, IE 10, Safari 6, and ToT with all of the JITs agree. Note
308         that some tests cannot be precisely validated because they use Math functions, which
309         ECMAScript chooses to not formally specify - those functions like sin() and friends are
310         allowed to return implementation-dependent results. Also some tests cannot be validated
311         at all because their behavior is either intentionally random or is timezone-dependent.
312         But 23 out of 26 tests now have some kind of validation.
313         
314         I've updated the harnesses to show erroneous runs when displaying results.
315
316         * resources/TEMPLATE.html:
317         * resources/driver-TEMPLATE.html:
318         * resources/sunspider-analyze-results.js:
319         (formatResult):
320         (resultLine):
321         (printOutput):
322         * resources/sunspider-compare-results.js:
323         (.formatMean):
324         (.resultLine):
325         (.printOutput):
326         (sunspiderCompareResults):
327         * resources/sunspider-standalone-driver.js:
328         * tests/sunspider-1.0/3d-cube.js:
329         (Init):
330         * tests/sunspider-1.0/3d-morph.js:
331         * tests/sunspider-1.0/3d-raytrace.js:
332         * tests/sunspider-1.0/access-binary-trees.js:
333         * tests/sunspider-1.0/access-fannkuch.js:
334         * tests/sunspider-1.0/access-nbody.js:
335         * tests/sunspider-1.0/access-nsieve.js:
336         (sieve):
337         * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
338         * tests/sunspider-1.0/bitops-bits-in-byte.js:
339         * tests/sunspider-1.0/bitops-bitwise-and.js:
340         * tests/sunspider-1.0/bitops-nsieve-bits.js:
341         * tests/sunspider-1.0/controlflow-recursive.js:
342         * tests/sunspider-1.0/crypto-aes.js:
343         * tests/sunspider-1.0/crypto-md5.js:
344         * tests/sunspider-1.0/crypto-sha1.js:
345         * tests/sunspider-1.0/date-format-tofte.js:
346         * tests/sunspider-1.0/date-format-xparb.js:
347         * tests/sunspider-1.0/math-cordic.js:
348         * tests/sunspider-1.0/math-partial-sums.js:
349         (partial):
350         * tests/sunspider-1.0/math-spectral-norm.js:
351         * tests/sunspider-1.0/regexp-dna.js:
352         * tests/sunspider-1.0/string-base64.js:
353         * tests/sunspider-1.0/string-fasta.js:
354         (fastaRepeat):
355         (fastaRandom):
356         * tests/sunspider-1.0/string-tagcloud.js:
357         * tests/sunspider-1.0/string-unpack-code.js:
358         * tests/sunspider-1.0/string-validate-input.js:
359
360 2013-04-11  Filip Pizlo  <fpizlo@apple.com>
361
362         Get rid of the 10ms delay between test executions
363         https://bugs.webkit.org/show_bug.cgi?id=114458
364         <rdar://problem/13631112>
365
366         Reviewed by Darin Adler and Geoffrey Garen.
367         
368         More details on this in the bug. Long story short, I don't know why the 10ms delay is
369         there, and it appears to be the root of some noisyness and unreliability in the results.
370         Removing it makes results more consistent, which is a good thing.
371
372         * resources/driver-TEMPLATE.html:
373
374 2013-01-16  Eric Seidel  <eric@webkit.org>
375
376         Remove --shark* support from sunspider/run-sunspider now that Shark is dead
377         https://bugs.webkit.org/show_bug.cgi?id=99512
378
379         Reviewed by Darin Adler.
380
381         I tried to use --instruments, but that also seems broken (in the same way before/after this patch).
382
383         * sunspider:
384         (runTestsOnce):
385
386 2012-08-02  Filip Pizlo  <fpizlo@apple.com>
387
388         Unreviewed, revert debugging stuff I landed in http://trac.webkit.org/changeset/124557
389         by accident.
390
391         * tests/v8-v6/v8-crypto.js:
392         (am1):
393         (am2):
394         (am3):
395         (am4):
396         (bnpCopyTo):
397         (bnpFromString):
398         (bnpClamp):
399
400 2012-03-11  Maciej Stachowiak  <mjs@apple.com>
401
402         Add --instruments option to sunspider to profile with Instruments
403         https://bugs.webkit.org/show_bug.cgi?id=80783
404
405         Reviewed by Filip Pizlo.
406
407         * resources/TimeProfile20us.tracetemplate: Added.
408         * sunspider:
409         (runTestsOnce):
410
411 2011-11-15  Andy Wingo  <wingo@igalia.com>
412
413         Extend sunspider driver to be able to run kraken
414         https://bugs.webkit.org/show_bug.cgi?id=71799
415
416         Reviewed by Filip Pizlo.
417
418         * resources/sunspider-standalone-driver.js: Try to load a -data
419         file.  If that succeeds, we have a kraken-like test, so we time
420         the test using `load'.  Otherwise fall back to using `run'.
421
422 2011-08-11  Mark Rowe  <mrowe@apple.com>
423
424         Fix sunspider-compare-results to default to the same suite version as the sunspider script.
425
426         Rubber-stamped by Maciej Stachowiak.
427
428         * sunspider-compare-results:
429
430 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
431
432         Update SunSpider versioning to 1.0
433         https://bugs.webkit.org/show_bug.cgi?id=63870
434
435         Reviewed by Dan Bernstein.
436
437         * hosted/sunspider.html:
438         * hosted/versions.html:
439         * resources/driver-TEMPLATE.html:
440         * resources/results-TEMPLATE.html:
441         * sunspider:
442
443 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
444
445         Sunspider 0.9.1 harness doesn't actually close() its test documents
446         https://bugs.webkit.org/show_bug.cgi?id=47045
447
448         Reviewed by Daniel Bates.
449
450         * hosted/versions.html:
451         * resources/driver-TEMPLATE.html:
452
453 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
454
455         Not Reviewed. 
456         
457         Fix obviou typo in previous commit.
458
459         * tests/sunspider-1.0/bitops-bitwise-and.js:
460
461 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
462
463         Reviewed by Anders Carlsson.
464
465         SunSpider: all four bitops benchmarks can be replaced with NOP
466         https://bugs.webkit.org/show_bug.cgi?id=38446
467
468         * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
469         (TimeFunc): Save the result.
470         * tests/sunspider-1.0/bitops-bits-in-byte.js:
471         (TimeFunc): Save the result.
472         * tests/sunspider-1.0/bitops-bitwise-and.js: More explicitly
473         save the results.
474         * tests/sunspider-1.0/bitops-nsieve-bits.js:
475         (sieve): Save the result.
476
477 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
478
479         SunSpider: string-validate-input.js uses an undeclared variable named 'name', which is a DOM API
480         https://bugs.webkit.org/show_bug.cgi?id=60937
481
482         Reviewed by Dan Bernstein.
483
484         * tests/sunspider-1.0/string-validate-input.js:
485         (doTest): Rename the variable to username
486
487 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
488
489         Reviewed by Darin Adler.
490
491         SunSpider: The main function of math-cordic is dead code
492         https://bugs.webkit.org/show_bug.cgi?id=63863
493         
494         Try to prevent math-cordic from being eliminated as dead code or
495         compiled down to a constant.
496
497         * tests/sunspider-1.0/math-cordic.js:
498         (cordicsincos):
499         (cordic):
500
501 2011-06-30  Maciej Stachowiak  <mjs@apple.com>
502
503         Reviewed by Adam Barth.
504
505         Create sunspider-1.0 directory in preparation for fixing a bunch of bugs
506         https://bugs.webkit.org/show_bug.cgi?id=63782
507
508         * make-hosted: Learn about the new directory.
509         * tests/sunspider-1.0: Copied from PerformanceTests/SunSpider/tests/sunspider-0.9.1.
510
511 2010-11-29  Geoffrey Garen  <ggaren@apple.com>
512
513         Reviewed by Gavin Barraclough.
514
515         Improved accuracy of command-line SunSpider.
516
517         * resources/sunspider-standalone-driver.js: Use the returned number of
518         elapsed milliseconds from "run" and "checkSyntax" instead of measuring
519         ourselves, for slightly more accurate numbers.
520
521 2010-09-16  Gavin Barraclough  <barraclough@apple.com>
522
523         Reviewed by Mark Rowe.
524         https://bugs.webkit.org/show_bug.cgi?id=45924
525
526         Modify the SunSpider harness to allow a path to be passed to the --suite
527         argument, supress the normal prepend-"tests/" behaviour for suite names
528         containing a '/'.
529
530         Also wrap the code in an anonymous function to move variables out of global
531         scope (currently the harness cannot handle running tests the overwrite the
532         variable 'j').
533
534         * resources/sunspider-standalone-driver.js:
535         * sunspider:
536
537 2010-08-27  Michael Saboff  <msaboff@apple.com>
538
539         Reviewed by Stephanie Lewis.
540
541         Added version 5 and version 6 of V8 tests with modifications similar
542         to the version 4 tests so they'll work with the SunSpider test harness.
543         Note that these files are external source and therefore don't conform 
544         to WebKit coding standards.
545         https://bugs.webkit.org/show_bug.cgi?id=44638
546
547         * tests/v8-v5: Added.
548         * tests/v8-v5/LIST: Added.
549         * tests/v8-v5/v8-crypto.js: Added.
550         * tests/v8-v5/v8-deltablue.js: Added.
551         * tests/v8-v5/v8-earley-boyer.js: Added.
552         * tests/v8-v5/v8-raytrace.js: Added.
553         * tests/v8-v5/v8-regexp.js: Added.
554         * tests/v8-v5/v8-richards.js: Added.
555         * tests/v8-v5/v8-splay.js: Added.
556         * tests/v8-v6: Added.
557         * tests/v8-v6/LIST: Added.
558         * tests/v8-v6/v8-crypto.js: Added.
559         * tests/v8-v6/v8-deltablue.js: Added.
560         * tests/v8-v6/v8-earley-boyer.js: Added.
561         * tests/v8-v6/v8-raytrace.js: Added.
562         * tests/v8-v6/v8-regexp.js: Added.
563         * tests/v8-v6/v8-richards.js: Added.
564         * tests/v8-v6/v8-splay.js: Added.
565
566 2010-07-15  Stephanie Lewis  <slewis@apple.com>
567
568         Reviewed by Geoff Garen.
569
570         http://bugs.webkit.org/show_bug.cgi?id=42406
571         Output location of results file.
572
573         * sunspider:
574
575 2010-05-17  Maciej Stachowiak  <mjs@apple.com>
576
577         Reviewed by Mark Rowe.
578
579         Browser-hosted version of SunSpider runs tests twice as many times as advertised
580         https://bugs.webkit.org/show_bug.cgi?id=39201
581
582         * make-hosted: Make sure to reset test list after every test suite.
583
584 2010-05-07  Adam Roben  <aroben@apple.com>
585
586         Fix typo in sunspider-compare-results.js that caused incorrect
587         two-sample t statistics to be computed
588
589         Fixes <http://webkit.org/b/38768> SunSpider computes incorrect
590         two-sample t statistics
591
592         Reviewed by Darin Adler.
593
594         * resources/sunspider-compare-results.js:
595         (sunspiderCompareResults.resultLine): Fixed the calculation of the t
596         variable to match the definition of a two-sample t statistic. Looks
597         like this was just a simple typo.
598
599 2010-04-05  Darin Adler  <darin@apple.com>
600
601         Updated the ignore property for changes to the contents here.
602
603         * .: Modified property svn:ignore.
604
605 2010-02-20  Maciej Stachowiak  <mjs@apple.com>
606
607         Reviewed by Adam Barth.
608
609         Add sunspider-0.9.1 to the site in preparation for announcing it.
610         https://bugs.webkit.org/show_bug.cgi?id=35206
611
612         * hosted/sunspider.css: Remove tabs, which I noticed where present while
613         copying to the WebKitSite directory.
614
615 2010-01-11  Oliver Hunt  <oliver@apple.com>
616
617         Reviewed by Darin Adler.
618
619         Sunspider parse-only tests don't run
620         https://bugs.webkit.org/show_bug.cgi?id=33489
621
622         Fix parse-only test list
623
624         * tests/parse-only/LIST:
625
626 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
627
628         Reviewed by Adele Peterson.
629
630         Make SunSpider version more prominent in the title
631         https://bugs.webkit.org/show_bug.cgi?id=32574
632
633         * hosted/sunspider.html:
634         * hosted/versions.html:
635         * resources/driver-TEMPLATE.html:
636         * resources/results-TEMPLATE.html:
637
638 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
639
640         Reviewed by Darin Adler.
641
642         Make SunSpider refuse to compare results across content versions
643         https://bugs.webkit.org/show_bug.cgi?id=32573
644         
645         The results URL now includes a version indicator, and compare mode
646         will refuse to compare between versions.
647
648         * resources/driver-TEMPLATE.html:
649         * resources/results-TEMPLATE.html:
650
651 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
652
653         Reviewed by Darin Adler.
654
655         Some Browser-hosted SunSpider files are not valid HTML5
656         https://bugs.webkit.org/show_bug.cgi?id=32536
657         
658         Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
659         close tags and adding a meta charset declaration).
660
661         * hosted/sunspider.html:
662         * hosted/versions.html:
663         * resources/TEMPLATE.html:
664         * resources/driver-TEMPLATE.html:
665         * resources/results-TEMPLATE.html:
666
667 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
668
669         Reviewed by Eric Seidel.
670
671         Make sunspider-0.9.1 the default content set (both command-line and hosted)
672         https://bugs.webkit.org/show_bug.cgi?id=32537
673
674         * hosted/sunspider.html: Change default suite link.
675         * sunspider: Change default suite.
676         * sunspider-compare-results: Likewise.
677
678 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
679
680         Reviewed by Sam Weinig.
681
682         in-browser SunSpider suffers excessive penalty under power management
683         https://bugs.webkit.org/show_bug.cgi?id=32505
684         
685         I have made a few changes to address this:
686         
687         1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
688         document.write to reduce triggering of progress UI.
689         2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
690         stepped all the way down the whole time.
691         3) Run the test cycle an extra time for warmup before the runs that count.
692
693         * hosted/sunspider-record-result.js: Removed.
694         * make-hosted:
695         * resources/TEMPLATE.html:
696         * resources/driver-TEMPLATE.html:
697
698 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
699
700         Reviewed by Gavin Barraclough.
701
702         SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
703         https://bugs.webkit.org/show_bug.cgi?id=16806
704
705         Based on a patch by Eric Seidel.
706         
707         Fix the base64 computation to actually compute correct results. The impact on runtime of
708         the test is pretty small, but noticeable for some browsers. But at least it's not
709         doing a wrong and meaningless computation any more.
710         
711         * tests/sunspider-0.9.1/string-base64.js:
712         ():
713         (base64ToString):
714
715 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
716
717         Fixing commit error...
718         
719         I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
720
721         * resources/driver-TEMPLATE.html:
722
723 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
724
725         Reviewed by Sam Weinig.
726
727         Use JSON.parse instead of eval for Web-hosted SunSpider results processing
728         https://bugs.webkit.org/show_bug.cgi?id=32490
729
730         * hosted/json2.js: Added. Incorporated from json.org
731         * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
732         * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
733         results.
734
735 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
736
737         Reviewed by Oliver Hunt.
738
739         Enable Web-hosted version of SunSpider to handle multiple versions
740         https://bugs.webkit.org/show_bug.cgi?id=32478
741
742         * make-hosted: Now generate the hosted version to be able to run both the
743         0.9 and the 0.9.1 test suites.
744         * hosted: Modified property svn:ignore.
745         * hosted/sunspider-driver.html: Removed.
746         * hosted/sunspider-results.html: Removed.
747         * hosted/sunspider.html:
748         * hosted/versions.html: Added.
749         * resources/TEMPLATE.html:
750         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
751         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
752
753 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
754
755         Reviewed by Oliver Hunt.
756
757         Give command-line sunspider the ability to handle multiple suites and versions
758         https://bugs.webkit.org/show_bug.cgi?id=32477
759         
760         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
761         enumerating the tests. Current suites are:
762         
763         sunspider-0.9 - riginal SunSpider content 
764         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
765         ubench - "ubench" microbenchmark suite
766         v8-v4 - version 4 of the v8 benchmark
767         parse-only - some parser tests
768         
769         Sunspider can now be run with the --suite= parameter to select a
770         suite. The default is --suite=sunspider-0.9.
771
772         * .: Modified property svn:ignore to ignore new results directories.
773         * make-hosted: Generate based on sunspider-0.9 suite.
774         * resources/TEMPLATE.html: Fix an HTML compliance problem.
775         * resources/sunspider-standalone-driver.js: Become aware of suite names.
776         * sunspider: Modified to handle multiple suites as described above.
777         * sunspider-compare-results: ditto
778         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
779         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
780         * tests/LIST-UBENCH: Removed.
781         * tests/LIST-V8: Removed.
782         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
783         * tests/3d-morph.js: ditto
784         * tests/3d-raytrace.js: ditto
785         * tests/access-binary-trees.js: ditto
786         * tests/access-fannkuch.js: ditto
787         * tests/access-nbody.js: ditto
788         * tests/access-nsieve.js: ditto
789         * tests/bitops-3bit-bits-in-byte.js: ditto
790         * tests/bitops-bits-in-byte.js: ditto
791         * tests/bitops-bitwise-and.js: ditto
792         * tests/bitops-nsieve-bits.js: ditto
793         * tests/controlflow-recursive.js: ditto
794         * tests/crypto-aes.js: ditto
795         * tests/crypto-md5.js: ditto
796         * tests/crypto-sha1.js: ditto
797         * tests/date-format-tofte.js: ditto
798         * tests/date-format-xparb.js: ditto
799         * tests/math-cordic.js: ditto
800         * tests/math-partial-sums.js: ditto
801         * tests/math-spectral-norm.js: ditto
802         * tests/regexp-dna.js: ditto
803         * tests/string-base64.js: ditto
804         * tests/string-fasta.js: ditto
805         * tests/string-tagcloud.js: ditto
806         * tests/string-unpack-code.js: ditto
807         * tests/string-validate-input.js: ditto
808         * tests/sunspider-0.9/: Added
809         * tests/sunspider-0.9/*.js: Moved from one level up
810         * tests/sunspider-0.9.1: Added.
811         * tests/sunspider-0.9.1/*.js: Moved from one level up.
812         * tests/v8-crypto.js: Moved to v8-v4/ directory.
813         * tests/v8-deltablue.js: ditto
814         * tests/v8-earley-boyer.js: ditto
815         * tests/v8-raytrace.js: ditto
816         * tests/v8-regexp.js: ditto
817         * tests/v8-richards.js: ditto
818         * tests/v8-splay.js: ditto
819         * tests/v8-v4: Added.
820
821 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
822
823         Reviewed by Darin Adler.
824
825         Platform specific null device replaced with a platform independent.
826         https://bugs.webkit.org/show_bug.cgi?id=29544
827
828         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
829
830 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
831
832         Reviewed by Maciej Stachowiak.
833
834         [Qt] Make sunspider script work on Windows platform.
835         https://bugs.webkit.org/show_bug.cgi?id=29544
836
837         * sunspider:
838         Using platform specific null device instead of /dev/null.
839
840 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
841
842         Reviewed by Oliver Hunt.
843
844         https://bugs.webkit.org/show_bug.cgi?id=26540
845         Currently the SunSpider test driver lacks an option to run a test suite that
846         will test JavaScriptCore parsing performance only.  This patch adds just such
847         a test suite and option to SunSpider as well as the jsc test shell.  I've included
848         three large javascript source files found in the wild: jquery, mootools and prototype.
849         Combined with the concatenation of all three, these form a new testsuite to measure
850         and test pure JavaScriptCore parsing performance.
851
852         * resources/sunspider-standalone-driver.js:
853           Load the js with parse-only flag if it is part of the parse-only suite.
854         * sunspider:
855           Add the parse-only suite to the list of options.
856         * sunspider-compare-results:
857           Add the parse-only suite to the list of options.
858         * tests/LIST-PARSE-ONLY: Added.
859         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
860           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
861         * tests/parse-only/jquery-1.3.2.js: Added.
862           Latest version from jquery.com.
863         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
864           Latest version from mootools.net.
865         * tests/parse-only/prototype-1.6.0.3.js: Added.
866           Latest version from prototypejs.org.
867
868 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
869
870         Update V8 benchmark to version 4.
871         
872         Reviewed by Cameron Zwarich.
873
874         * tests/LIST-V8:
875         * tests/v8-crypto.js:
876         * tests/v8-deltablue.js:
877         * tests/v8-earley-boyer.js:
878         * tests/v8-raytrace.js:
879         * tests/v8-regexp.js: Added.
880         * tests/v8-richards.js:
881         * tests/v8-splay.js: Added.
882
883 2009-05-26  Olivier DOLE  <odole@pleyo.com>
884
885         Reviewed by Maciej.
886
887         Store sunspider test suite results in a specific directory for each
888         different type of benchmark (sunspider, V8 or ubench).
889
890         * sunspider:
891         * sunspider-compare-results:
892
893 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
894
895         Build fix.
896
897         * SunSpider.make:
898
899 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
900
901         Include SunSpider tests in production builds.
902         
903         Reviewed by Adam Roben.
904
905         * SunSpider.make: Added.
906         * xcopy.excludes: Added.
907
908 2009-03-04  Oliver Hunt  <oliver@apple.com>
909
910         Added a few more tests to ubench
911
912         RS=Gavin Barraclough
913
914         Adding a couple of test to cover too few and too many arguments.  This
915         is needed to prevent regressions in all call paths from my current refactoring.
916
917         * tests/LIST-UBENCH:
918         * tests/ubench/function-correct-args.js: Added.
919         (f):
920         * tests/ubench/function-excess-args.js: Added.
921         (f):
922
923 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
924
925         Fix a bug where when V8's or UBENCH's test suite list file is missing
926         the error message is suggesting that SunSpider's list file is missing.
927
928         Reviewed by Sam Weinig.
929
930         * sunspider:
931         (loadTestsList): Fix the error message to use the right list name.
932
933 2008-10-07  David Hyatt  <hyatt@apple.com>
934
935         https://bugs.webkit.org/show_bug.cgi?id=18989
936
937         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
938         we want all browsers to be doing the same thing on the test.
939
940         Reviewed by Maciej
941
942         * tests/regexp-dna.js:
943
944 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
945
946         Rubber-stamped by Maciej Stachowiak.
947
948         The bleeding edge branch of V8 now supports passing fileanmes after
949         '-f', so the SunSpider script should be changed to do this again.
950         This allows all 3 major JavaScript engines to be used with the
951         unmodified SunSpider script.
952
953         * sunspider:
954
955 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
956
957         Reviewed by Maciej Stachowiak.
958
959         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
960         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
961         correct the punctuation in its documentation.
962
963         * sunspider:
964
965 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
966
967         Reviewed by Oliver Hunt.
968
969         Bug 20616: Incorporate V8 benchmarks in testing
970         <https://bugs.webkit.org/show_bug.cgi?id=20616>
971
972         This is a first attempt at running the V8 benchmarks within SunSpider,
973         with the --v8 option. The number of iterations is fixed for each
974         benchmark, and was chosen to make each of the benchmarks run between
975         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
976
977         * sunspider:
978         * tests/LIST-V8: Added.
979         * tests/v8-crypto.js: Added.
980         * tests/v8-deltablue.js: Added.
981         * tests/v8-earley-boyer.js: Added.
982         * tests/v8-raytrace.js: Added.
983         * tests/v8-richards.js: Added.
984
985 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
986
987         Not reviewed.
988
989         Remove a local change that was accidentally committed in r36034.
990
991         * resources/sunspider-standalone-driver.js:
992
993 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
994
995         Rubber-stamped by Oliver Hunt.
996
997         Make SunSpider work with JavaScript shells that don't take filenames
998         after a '-f' argument.
999
1000         * sunspider:
1001
1002 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
1003
1004         Reviewed by Timothy.
1005
1006         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
1007
1008         * sunspider:
1009         * tests/LIST-SQUIRRELFISH: Removed.
1010
1011 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
1012
1013         Rubber-stamped by Mark Rowe.
1014
1015         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
1016
1017         * sunspider:
1018
1019 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
1020
1021         Rubber stamped by Oliver.
1022         
1023         - add newly working tests to squirrelfish list
1024
1025         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
1026
1027 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
1028
1029         Rubber stamped by Oliver.
1030         
1031         - move ubench tests properly.
1032
1033         * tests/ubench: Added.
1034         * tests/ubench/function-closure.js: Added.
1035         * tests/ubench/function-empty.js: Added.
1036         * tests/ubench/function-missing-args.js: Added.
1037         * tests/ubench/function-sum.js: Added.
1038         * tests/ubench/loop-empty-resolve.js: Added.
1039         * tests/ubench/loop-empty.js: Added.
1040         * tests/ubench/loop-sum.js: Added.
1041
1042 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
1043
1044         Rubber stamped by Oliver.
1045
1046         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
1047
1048 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
1049
1050         Reviewed by Oliver.
1051
1052         - make --squirrelfish mode use the real harness, but a reduced set of tests
1053
1054         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
1055
1056         * sunspider:
1057         * tests/LIST-SQUIRRELFISH:
1058         * tests/LIST-UBENCH: Added.
1059         * tests/squirrelfish: Moved to tests/ubench
1060
1061 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
1062
1063         Reviewed by Maciej Stachowiak.
1064         
1065         Removed an unused test.
1066
1067         * tests/squirrelfish/loop-resolve.js: Removed.
1068
1069 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
1070
1071         Reviewed by Oliver.
1072         
1073         Add newly runnable tests to --squirrelfish mode.
1074
1075         * tests/LIST-SQUIRRELFISH:
1076
1077 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
1078
1079         Reviewed by Maciej Stachowiak.
1080         
1081         A little more love for --squirrelfish mode:
1082         
1083         Fixed a misplaced sort. Results properly sort now.
1084         
1085         Added a test list just for squirrelfish, and updated it to include
1086         all currently passing SunSpider tests.
1087         
1088         Fixed the pruning regexp to match 3d-morph.js.
1089
1090         * sunspider:
1091
1092 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
1093
1094         Reviewed by Oliver Hunt.
1095         
1096         --squirrelfish mode: pared down tests for squirrelfish to chew on.
1097
1098         * sunspider:
1099         * tests/LIST:
1100         * tests/squirrelfish: Added.
1101         * tests/squirrelfish/function-closure.js: Added.
1102         * tests/squirrelfish/function-empty.js: Added.
1103         * tests/squirrelfish/function-missing-args.js: Added.
1104         * tests/squirrelfish/function-sum.js: Added.
1105         * tests/squirrelfish/loop-empty-resolve.js: Added.
1106         * tests/squirrelfish/loop-empty.js: Added.
1107         * tests/squirrelfish/loop-resolve.js: Added.
1108         * tests/squirrelfish/loop-sum.js: Added.
1109
1110 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
1111
1112         Reviewed by Geoff.
1113
1114         * sunspider: Pause update daemon on Darwin to make results reliable
1115         again (workaround for <rdar://problem/5811127>).
1116
1117 2008-02-07  Eric Seidel  <eric@webkit.org>
1118
1119         Reviewed by Sam.
1120
1121         * sunspider: add an error message when ./tests/LIST is not found
1122
1123 2007-12-29  Alp Toker  <alp@atoker.com>
1124
1125         Fix typo.
1126
1127         * hosted/sunspider.html:
1128
1129 2007-12-29  Alp Toker  <alp@atoker.com>
1130
1131         Reviewed by Maciej.
1132
1133         Provide the current URL in the SunSpider results page. Useful for
1134         copying benchmark results out of browsers without a location bar or
1135         where the location bar can't handle long URLs.
1136
1137         * hosted/sunspider-results.html:
1138
1139 2007-12-20  Eric Seidel  <eric@webkit.org>
1140
1141         Reviewed by mjs.
1142
1143         * sunspider: increase default number of runs for more accurate numbers
1144
1145 2007-12-18  Mark Rowe  <mrowe@apple.com>
1146
1147         Fix typo.
1148
1149         * hosted/sunspider.html:
1150
1151 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
1152
1153         Reviewed by Eric.
1154         
1155         - give an extra digit of precision for small differences.
1156
1157         * resources/sunspider-compare-results.js:
1158
1159 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
1160
1161         Reviewed by Eric.
1162         
1163         - work around IE layout bugs
1164
1165         * hosted/sunspider.css:
1166
1167 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
1168
1169         Reviewed by Mark.
1170         
1171         - fix compare feature
1172
1173         * hosted/sunspider-results.html:
1174
1175 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
1176
1177         Reviewed by Mark.
1178         
1179         - Change the format of the output URL to be shorter, to meet IE constraints
1180
1181         * hosted/sunspider-driver.html:
1182         * hosted/sunspider-results.html:
1183
1184 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
1185
1186         Reviewed by Oliver.
1187         
1188         - Add some explanatory test and a dumb logo to SunSpider web hosted version
1189
1190         * hosted/sunspider-driver.html:
1191         * hosted/sunspider-results.html:
1192         * hosted/sunspider.css:
1193         * hosted/sunspider.html:
1194
1195 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
1196
1197         Reviewed by Oliver.
1198         
1199         - Make the web-hosted version of SunSpider look a little bit less drab.
1200
1201         * hosted/sunspider-driver.html:
1202         * hosted/sunspider-results.html:
1203         * hosted/sunspider.css: Added.
1204         * hosted/sunspider.html:
1205         * resources/TEMPLATE.html:
1206
1207 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
1208
1209         Reviewed by Sam.
1210         
1211         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
1212         
1213         A sample of what new-style results would look like:
1214         
1215         TEST                   COMPARISON            FROM                 TO             DETAILS
1216         ...
1217         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
1218         ...
1219             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
1220         
1221         * resources/sunspider-compare-results.js:
1222
1223 2007-11-14  Eric Seidel  <eric@webkit.org>
1224
1225         Reviewed by Sam.
1226
1227         * sunspider: add --shark-cache for L2 Cache Miss profiling
1228
1229 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
1230
1231         Reviewed by Darin Adler.
1232
1233         http://bugs.webkit.org/show_bug.cgi?id=15925
1234         SunSpider should check for Shark being installed
1235
1236         * sunspider: Explicitly test that shark command line tool is installed.
1237
1238 2007-11-04  Darin Adler  <darin@apple.com>
1239
1240         * hosted/sunspider.html: Tweaked formatting a bit.
1241
1242 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
1243
1244         Reviewed by Darin Adler.
1245
1246         - Add compare mode and a description of the benchmark to the hosted version.
1247
1248         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
1249         * hosted/sunspider.html: Added intro text.
1250         * make-hosted: Copy new file.
1251         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
1252         a function.
1253         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
1254         expects.
1255         * sunspider-compare-results: Use sunspider-standalone-compare.js.
1256
1257 2007-11-02  Darin Adler  <darin@apple.com>
1258
1259         * sunspider: Changed "--base" to "--set-baseline".
1260
1261 2007-11-02  Darin Adler  <darin@apple.com>
1262
1263         Reviewed by Maciej.
1264
1265         * sunspider: Added a "--base" option that records the current test as a baseline.
1266         Moved the code to find the newest .mshark file into a function (since I needed to use
1267         it in the other script and I prefer to copy/paste a function).
1268
1269         * sunspider-compare-results: Added logic to compare the baseline with the last
1270         run when you don't pass any parameters.
1271
1272 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
1273
1274         Reviewed by Eric.
1275         
1276         - Added list of ECMAScript3 features that SunSpider does not cover currently.
1277
1278         * UNCOVERED: Added.
1279
1280 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
1281
1282         Reviewed by Oliver.
1283         
1284         - Give better error messages when sunspider-compare-results fails
1285
1286         * sunspider-compare-results:
1287
1288 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
1289
1290         Reviewed by Darin Adler.
1291         
1292         - fix Date tests to work in command-line SpiderMonkey
1293
1294         * tests/date-format-tofte.js:
1295         * tests/date-format-xparb.js:
1296
1297 2007-10-29  Darin Adler  <darin@apple.com>
1298
1299         Reviewed by Maciej.
1300
1301         * sunspider: Added --shark20 option, to run Shark at its highest sample
1302         resolution instead of its default.
1303
1304 2007-10-28  Adam Roben  <aroben@apple.com>
1305
1306         Fix SunSpider on Windows
1307
1308         Reviewed by Eric.
1309
1310         * sunspider: Don't use colons in filenames.
1311
1312 2007-10-28  Darin Adler  <darin@apple.com>
1313
1314         Reviewed by Adam.
1315
1316         * resources/sunspider-analyze-results.js: Tweak the output format a little.
1317         Change so that when there's only one run we don't write out confidence
1318         intervals at all rather than writing "NaN" over and over again.
1319
1320         * sunspider: Use the time and date as a suffix on the results file, that way
1321         all the results are left behind in the tmp directory. This will make it easy
1322         for us to add features that compare past results in the future. Also rename
1323         the .mshark files using the same scheme.
1324
1325         * sunspider-compare-results: Relax the parsing rules so we can parse the
1326         results file exactly as it's emitted from sunspider.
1327
1328 2007-10-25  Adam Roben  <aroben@apple.com>
1329
1330         Strip carriage returns from results
1331
1332         Carriage returns were screwing up sunspider-compare-results on
1333         Windows.
1334
1335         Rubberstamped by Oliver.
1336
1337         * sunspider:
1338
1339 2007-10-24  Darin Adler  <darin@apple.com>
1340
1341         Reviewed by Eric.
1342
1343         * sunspider: Open the profile in shark after completing a run.
1344         * sunspider-compare-results: Tiny perl tweaks.
1345
1346 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
1347
1348         Reviewed by Darin Adler.
1349
1350         - Don't hardcode my path to testkjs
1351         
1352         * sunspider-compare-results:
1353
1354 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
1355
1356         Reviewed by Darin Adler.
1357         
1358         - Add a compare script to compare results, and improve formatting of standard results a bit.
1359
1360         * resources/sunspider-analyze-results.js:
1361         * resources/sunspider-compare-results.js: Added.
1362         * sunspider-compare-results: Added.
1363
1364 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
1365
1366         Reviewed by Darin Adler.
1367
1368         * resources/sunspider-analyze-results.js: Do the error range
1369         properly, using the t-distribution instead of 1.96 (which was
1370         based on the normal distribution).
1371         
1372         * sunspider: Print results in a way that will be friendlier to the
1373         soon-to-come compare mode.
1374
1375 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
1376
1377         Reviewed by Review.
1378         
1379         - add more tests (probably the final set for now)
1380         
1381         Already balanced. Added date, regexp, control flow, and a few
1382         more string and object/array access tests.
1383         
1384         * tests/LIST:
1385         * tests/access-fannkuch.js: Added.
1386         * tests/access-nbody.js: Added.
1387         * tests/controlflow-recursive.js: Added.
1388         * tests/date-format-tofte.js: Added.
1389         * tests/date-format-xparb.js: Added.
1390         * tests/regexp-dna.js: Added.
1391         * tests/string-validate-input.js: Added.
1392
1393 2007-10-21  Eric Seidel  <eric@webkit.org>
1394
1395         Reviewed by mjs.
1396
1397         * sunspider: make --shark only sample from testkjs
1398
1399 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
1400
1401         Rubber stamped by Eric.
1402         
1403         - added crypto tests (already balanced)
1404
1405         * tests/LIST:
1406         * tests/crypto-aes.js: Added.
1407         * tests/crypto-md5.js: Added.
1408         * tests/crypto-sha1.js: Added.
1409
1410 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
1411
1412         Reviewed by Adam.
1413         
1414         - rebalance new tests
1415         
1416         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
1417         
1418         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1419
1420         This balances them with the existing tests.
1421
1422         * tests/access-binary-trees.js:
1423         * tests/access-nsieve.js:
1424         * tests/bitops-nsieve-bits.js:
1425         * tests/math-partial-sums.js:
1426         * tests/math-spectral-norm.js:
1427         * tests/string-fasta.js:
1428
1429 2007-10-20  Eric Seidel  <eric@webkit.org>
1430
1431         Reviewed by darin.
1432         
1433         Add improved argument handling to sunspider, including
1434         --runs=<number>, --tests=<pattern, --shark, and --help
1435         Also re-factor code into subroutines
1436
1437         * sunspider: improved argument handling
1438
1439 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1440
1441         Reviewed by Mark.
1442
1443         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
1444         
1445         * TODO:
1446         * tests/LIST:
1447         * tests/access-binary-trees.js: Added.
1448         * tests/access-nsieve.js: Added.
1449         * tests/bitops-nsieve-bits.js: Added.
1450         * tests/math-partial-sums.js: Added.
1451         * tests/math-spectral-norm.js: Added.
1452         * tests/string-fasta.js: Added.
1453
1454 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1455
1456         Reviewed by Darin Adler.
1457
1458         * TODO: Updated to note areas that are now well-covered.
1459         
1460         - Change the tests and the driver to avoid leaving large object
1461         graphs hanging around, since that throws off the subsequent tests.
1462         
1463         * resources/sunspider-standalone-driver.js:
1464         * tests/3d-cube.js:
1465         * tests/3d-morph.js:
1466         * tests/3d-raytrace.js:
1467         * tests/bitops-bitwise-and.js:
1468         * tests/string-base64.js:
1469         * tests/string-tagcloud.js:
1470
1471 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1472
1473         Reviewed by Nikolas.
1474         
1475         - Rebalance test complexity.
1476         
1477         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
1478         
1479         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1480
1481         However, for some tests, IE7 was a huge outlier compared to other
1482         major browsers. For those tests, I used Firefox 2.0.0.8 on the
1483         same OS and hardware as a normalization basis.
1484         
1485         * tests/3d-morph.js:
1486         * tests/3d-raytrace.js:
1487         * tests/bitops-3bit-bits-in-byte.js:
1488         * tests/bitops-bits-in-byte.js:
1489         * tests/bitops-bitwise-and.js:
1490         * tests/math-cordic.js:
1491         * tests/string-base64.js:
1492         * tests/string-tagcloud.js:
1493         * tests/string-unpack-code.js:
1494
1495 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1496
1497         Reviewed by Oliver.
1498         
1499         - Fix the browser-hosted driver to be compatible with Safari 2
1500
1501         * hosted/sunspider-driver.html:
1502
1503 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1504
1505         Rubber stamped by Hyatt.
1506         
1507         - Add BSD license notice to cordic test, the original author said to do so.
1508
1509         * tests/math-cordic.js:
1510
1511 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1512
1513         Reviewed by Eric.
1514         
1515         - Support Windows and Windows browsers.
1516
1517         * hosted/sunspider-results.html: Display results correctly in IE.
1518         * sunspider: tweaks to work on Windows, and with spaces in the path.
1519
1520 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1521
1522         Rubber stamped by Oliver.
1523
1524         * tests/string-tagcloud.js: Remove stray print()
1525
1526 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
1527
1528         Reviewed by Oliver.
1529         
1530         Add some string test cases.
1531
1532         * tests/string-base64.js: Added. Does base64 encoding and decoding.
1533         * tests/string-tagcloud.js: Added. Parses a JSON string and
1534         generates tagcloud HTML markup based on the data.        
1535         * tests/string-unpack-code.js: Added. Unpack compressed versions of
1536         four of the most popular JavaScript libraries.
1537         * tests/LIST: Added the new tests.
1538
1539 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
1540
1541         Reviewed by Anders.
1542
1543         * TODO: Updated with info on how to do the stats properly.
1544
1545         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
1546         avoid overly slowing down the benchmark.
1547
1548         Added some 3D-related test cases.
1549         
1550         * tests/3d-cube.js: Added.
1551         * tests/3d-morph.js: Added.
1552         * tests/3d-raytrace.js: Added.
1553         * tests/LIST:
1554
1555 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
1556
1557         Reviewed by Darin Adler.
1558
1559         - New JavaScript benchmark
1560         http://bugs.webkit.org/show_bug.cgi?id=15515
1561         
1562         A start on a new JavaScript benchmark, with standalone and
1563         browser-hosted drivers.
1564         
1565         The test content is still incomplete and the drivers could use
1566         some more features but there is enough here to make a good start.
1567         
1568         The drivers compute a 95% confidence interval on the mean for the
1569         whole test, each category, and each individual test to make it
1570         easier to tell whether differences are statistically
1571         significant. The confidence interval can be narrowed by running
1572         the test more times.
1573         
1574         * TODO: Added. Remaining things that need to be done.
1575         * sunspider: Added. Perl script that acts as the standalone test driver.
1576         * resources/sunspider-standalone-driver.js: Added. JavaScript part
1577         of standalone test driver.
1578         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
1579         analysis code.
1580         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
1581         * make-hosted: Added. Script to generate browser-hosted tests.
1582         * hosted/sunspider.html: Added. Start page for browser-hosted test.
1583         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
1584         * hosted/sunspider-results.html: Added. Results page for browser-hosted
1585         tests.
1586         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
1587         tests.
1588         * tests/LIST: Added. List of tests to use.
1589         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
1590         * tests/bitops-bits-in-byte.js: Added. Ditto.
1591         * tests/bitops-bitwise-and.js: Added. Ditto.
1592         * tests/math-cordic.js: Added. Ditto.
1593