32dd4c99a94c823dbd1017808d02e1d6271b59d1
[WebKit-https.git] / PerformanceTests / SunSpider / ChangeLog
1 2014-06-03  Eva Balazsfalvi  <evab.u-szeged@partner.samsung.com>
2
3         REGRESSION(r169092 and r169102): Skip failing JSC tests poperly on non-x86 Darwin platforms
4         https://bugs.webkit.org/show_bug.cgi?id=133149
5
6         Reviewed by Csaba Osztrogon√°c.
7
8         * profiler-test.yaml: Skip profiler tests only if the architecture isn't x86 and the host is Darwin.
9
10 2014-05-21  Filip Pizlo  <fpizlo@apple.com>
11
12         Unreviewed, roll out http://trac.webkit.org/changeset/169159.
13         
14         This was a unilateral change and wasn't properly reviewed.
15
16         * profiler-test.yaml:
17
18 2014-05-19  Filip Pizlo  <fpizlo@apple.com>
19
20         Take care of some ARM64 test failures
21         https://bugs.webkit.org/show_bug.cgi?id=133090
22
23         Reviewed by Geoffrey Garen.
24
25         * profiler-test.yaml: Skip these on not-x86.
26
27 2014-03-14  Maciej Stachowiak  <mjs@apple.com>
28
29         Replace "Apple Computer, Inc." with "Apple Inc." in copyright headers
30         https://bugs.webkit.org/show_bug.cgi?id=130276
31         <rdar://problem/16266927>
32
33         Reviewed by Simon Fraser.
34
35         * hosted/sunspider.html:
36         * hosted/versions.html:
37         * make-hosted:
38         * resources/TEMPLATE.html:
39         * resources/driver-TEMPLATE.html:
40         * resources/results-TEMPLATE.html:
41         * resources/sunspider-analyze-results.js:
42         * resources/sunspider-compare-results.js:
43         * resources/sunspider-standalone-compare.js:
44         * resources/sunspider-standalone-driver.js:
45         * sunspider:
46         * sunspider-compare-results:
47         * tests/sunspider-0.9.1/3d-morph.js:
48         * tests/sunspider-0.9.1/3d-raytrace.js:
49         * tests/sunspider-0.9.1/bitops-bitwise-and.js:
50         * tests/sunspider-0.9.1/math-cordic.js:
51         * tests/sunspider-0.9.1/string-tagcloud.js:
52         * tests/sunspider-0.9/3d-morph.js:
53         * tests/sunspider-0.9/3d-raytrace.js:
54         * tests/sunspider-0.9/bitops-bitwise-and.js:
55         * tests/sunspider-0.9/math-cordic.js:
56         * tests/sunspider-0.9/string-tagcloud.js:
57         * tests/sunspider-1.0.1/3d-morph.js:
58         * tests/sunspider-1.0.1/3d-raytrace.js:
59         * tests/sunspider-1.0.1/bitops-bitwise-and.js:
60         * tests/sunspider-1.0.1/math-cordic.js:
61         * tests/sunspider-1.0.1/string-tagcloud.js:
62         * tests/sunspider-1.0.2/3d-morph.js:
63         * tests/sunspider-1.0.2/3d-raytrace.js:
64         * tests/sunspider-1.0.2/bitops-bitwise-and.js:
65         * tests/sunspider-1.0.2/math-cordic.js:
66         * tests/sunspider-1.0.2/string-tagcloud.js:
67         * tests/sunspider-1.0/3d-morph.js:
68         * tests/sunspider-1.0/3d-raytrace.js:
69         * tests/sunspider-1.0/bitops-bitwise-and.js:
70         * tests/sunspider-1.0/math-cordic.js:
71         * tests/sunspider-1.0/string-tagcloud.js:
72
73 2014-02-10  Filip Pizlo  <fpizlo@apple.com>
74
75         Unreviewed, revert the accidental change from r163789.
76
77         * tests/v8-v6/v8-deltablue.js:
78
79 2013-12-10  Filip Pizlo  <fpizlo@apple.com>
80
81         Get rid of forward exit on DoubleAsInt32
82         https://bugs.webkit.org/show_bug.cgi?id=125552
83
84         Reviewed by Oliver Hunt.
85         
86         Use SunSpider as a kind of spot-check for the
87         no-architecture-specific-optimization paths in the compiler.
88
89         * no-architecture-specific-optimizations.yaml: Added.
90
91 2013-10-08  Geoffrey Garen  <ggaren@apple.com>
92
93         Refined power management in SunSpider 1.0.2
94         https://bugs.webkit.org/show_bug.cgi?id=122525
95
96         Reviewed by Gavin Barraclough.
97
98         * hosted/sunspider.css:
99         (#frameparent):
100         (#countdown):
101         * resources/TEMPLATE.html: Provide an explicit countdown element in the
102         parent, instead of hacking it into each child frame.
103
104         * resources/driver-TEMPLATE.html: Be more explicit about how we respond
105         to power management. Just moving setTimeout(0) inside the loop is a
106         little too subtle, and may not work across browsers.
107
108         Also, leave a little extra time after changing the countdown graphic,
109         to give the browser some time to paint.
110
111 2013-10-02  Geoffrey Garen  <ggaren@apple.com>
112
113         Improved painting behavior in SunSpider 1.0.2
114         https://bugs.webkit.org/show_bug.cgi?id=122257
115
116         Reviewed by Oliver Hunt.
117
118         Individual test names with times was a good painting strategy years
119         back, when each benchmark took hundreds of milliseconds to run. Now,
120         the whole suite can finish in a a few hundres milliseconds, and things
121         look pretty flashy.
122
123         Instead, display a countdown that indicates which iteration of the
124         test we're on.
125
126         * resources/TEMPLATE.html: Work around a bug in Firefox by specifying
127         our style inside the .html file instead of in an external stylesheet.
128         Under heavy JavaScript load, Firefox neglects to apply external
129         stylesheets -- probably due to a starved timer.
130         
131         * resources/driver-TEMPLATE.html: Delay a little between runs to ensure
132         that our countdown has enough time to paint.
133
134 2013-09-26  Geoffrey Garen  <ggaren@apple.com>
135
136         Fixed a version number string in SunSpider 1.0.2
137         https://bugs.webkit.org/show_bug.cgi?id=122004
138
139         Reviewed by Gavin Barraclough.
140
141         * resources/driver-TEMPLATE.html:
142         * resources/results-TEMPLATE.html: 1.0.2!
143
144 2013-09-26  Geoffrey Garen  <ggaren@apple.com>
145
146         Tentative SunSpider 1.0.2
147         https://bugs.webkit.org/show_bug.cgi?id=122002
148
149         Reviewed by Gavin Barraclough.
150
151         Be even more aggressive about asking for zero delay between tests by
152         setting a timeout at the beginning of a test instead of the end. This is
153         another step toward reducing noise -- particularly noise caused by
154         power management -- just like the step down from 10ms to 0ms.
155
156         Browsers typically put a 4ms floor on timers. The sooner we set
157         our timer, the sooner it will fire.
158
159         * make-hosted:
160         * resources/driver-TEMPLATE.html: Changed the driver to set the timeout
161         earlier.
162         * tests/sunspider-1.0.2: Added. These tests are unchanged from 1.0.1.
163
164 2013-09-09  Filip Pizlo  <fpizlo@apple.com>
165
166         Stress tests should test the jsc profiler (-p)
167         https://bugs.webkit.org/show_bug.cgi?id=121043
168
169         Reviewed by Mark Hahnenberg.
170         
171         Add a jsc-stress-test that tries to profile SunSpider.
172
173         * profiler-test.yaml: Added.
174
175 2013-08-08  Cosmin Truta  <ctruta@blackberry.com>
176
177         SunSpider: Move the 3d-morph fix from version 1.0 to version 1.0.1
178         https://bugs.webkit.org/show_bug.cgi?id=117993
179
180         Reviewed by George Staikos.
181
182         Bump the SunSpider version to 1.0.1.
183
184         Unroll r149548 in order to restore version 1.0 to its original,
185         and add the 3d-morph fix to version 1.0.1 instead.
186
187         * hosted/sunspider.html: Updated to version 1.0.1.
188         * hosted/versions.html: Updated to version 1.0.1.
189         * make-hosted: Added support for version 1.0.1.
190         * resources/driver-TEMPLATE.html: Updated to version 1.0.1.
191         * resources/results-TEMPLATE.html: Updated to version 1.0.1.
192         * sunspider: Updated to version 1.0.1.
193         * sunspider-compare-results: Updated to version 1.0.1.
194         * tests/sunspider-1.0/3d-morph.js: Rolled back to the original version.
195         * tests/sunspider-1.0.1/LIST: Added.
196         * tests/sunspider-1.0.1/3d-cube.js: Added.
197         * tests/sunspider-1.0.1/3d-morph.js: Added.
198         * tests/sunspider-1.0.1/3d-raytrace.js: Added.
199         * tests/sunspider-1.0.1/access-binary-trees.js: Added.
200         * tests/sunspider-1.0.1/access-fannkuch.js: Added.
201         * tests/sunspider-1.0.1/access-nbody.js: Added.
202         * tests/sunspider-1.0.1/access-nsieve.js: Added.
203         * tests/sunspider-1.0.1/bitops-3bit-bits-in-byte.js: Added.
204         * tests/sunspider-1.0.1/bitops-bits-in-byte.js: Added.
205         * tests/sunspider-1.0.1/bitops-bitwise-and.js: Added.
206         * tests/sunspider-1.0.1/bitops-nsieve-bits.js: Added.
207         * tests/sunspider-1.0.1/controlflow-recursive.js: Added.
208         * tests/sunspider-1.0.1/crypto-aes.js: Added.
209         * tests/sunspider-1.0.1/crypto-md5.js: Added.
210         * tests/sunspider-1.0.1/crypto-sha1.js: Added.
211         * tests/sunspider-1.0.1/date-format-tofte.js: Added.
212         * tests/sunspider-1.0.1/date-format-xparb.js: Added.
213         * tests/sunspider-1.0.1/math-cordic.js: Added.
214         * tests/sunspider-1.0.1/math-partial-sums.js: Added.
215         * tests/sunspider-1.0.1/math-spectral-norm.js: Added.
216         * tests/sunspider-1.0.1/regexp-dna.js: Added.
217         * tests/sunspider-1.0.1/string-base64.js: Added.
218         * tests/sunspider-1.0.1/string-fasta.js: Added.
219         * tests/sunspider-1.0.1/string-tagcloud.js: Added.
220         * tests/sunspider-1.0.1/string-unpack-code.js: Added.
221         * tests/sunspider-1.0.1/string-validate-input.js: Added.
222
223 2013-05-03  Cosmin Truta  <ctruta@blackberry.com>
224
225         SunSpider 1.0: 3d-morph: use epsilon to check result
226         https://bugs.webkit.org/show_bug.cgi?id=115553
227
228         Reviewed by George Staikos.
229
230         SunSpider/3d-morph should be more lenient in validating its results, since they depend on Math.sin, which isn't formally specified.
231
232         * tests/sunspider-1.0/3d-morph.js:
233
234 2013-04-19  Filip Pizlo  <fpizlo@apple.com>
235
236         SunSpider/string-tagcloud should be more lenient in validating its results, since they depend on Math.log, which isn't formally specified
237         https://bugs.webkit.org/show_bug.cgi?id=114895
238
239         Reviewed by Michael Saboff.
240
241         * tests/sunspider-1.0/string-tagcloud.js:
242
243 2013-04-18  Filip Pizlo  <fpizlo@apple.com>
244
245         Whenever it is cheap and non-invasive, SunSpider tests should validate their results to ensure that the browser runs them correctly
246         https://bugs.webkit.org/show_bug.cgi?id=114852
247
248         Reviewed by Geoffrey Garen.
249         
250         This adds low-overhead checking of the results of each SunSpider tests. This is not
251         meant to be an exhaustive test that everything was executed correctly, but rather a
252         simple sanity check that will catch glaring mistakes. The philosophy here is that we're
253         not trying to prevent all forms of cheating, or that we're trying to prove the
254         browser's correctness. Moreover, these checks are meant to incur minimal overhead;
255         they currently clock in at <2% slow-down in SunSpider 1.0.
256         
257         The test expectations were generated using the LLInt on command-line, and I've verified
258         that Firefox 20, Chrome 26, IE 10, Safari 6, and ToT with all of the JITs agree. Note
259         that some tests cannot be precisely validated because they use Math functions, which
260         ECMAScript chooses to not formally specify - those functions like sin() and friends are
261         allowed to return implementation-dependent results. Also some tests cannot be validated
262         at all because their behavior is either intentionally random or is timezone-dependent.
263         But 23 out of 26 tests now have some kind of validation.
264         
265         I've updated the harnesses to show erroneous runs when displaying results.
266
267         * resources/TEMPLATE.html:
268         * resources/driver-TEMPLATE.html:
269         * resources/sunspider-analyze-results.js:
270         (formatResult):
271         (resultLine):
272         (printOutput):
273         * resources/sunspider-compare-results.js:
274         (.formatMean):
275         (.resultLine):
276         (.printOutput):
277         (sunspiderCompareResults):
278         * resources/sunspider-standalone-driver.js:
279         * tests/sunspider-1.0/3d-cube.js:
280         (Init):
281         * tests/sunspider-1.0/3d-morph.js:
282         * tests/sunspider-1.0/3d-raytrace.js:
283         * tests/sunspider-1.0/access-binary-trees.js:
284         * tests/sunspider-1.0/access-fannkuch.js:
285         * tests/sunspider-1.0/access-nbody.js:
286         * tests/sunspider-1.0/access-nsieve.js:
287         (sieve):
288         * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
289         * tests/sunspider-1.0/bitops-bits-in-byte.js:
290         * tests/sunspider-1.0/bitops-bitwise-and.js:
291         * tests/sunspider-1.0/bitops-nsieve-bits.js:
292         * tests/sunspider-1.0/controlflow-recursive.js:
293         * tests/sunspider-1.0/crypto-aes.js:
294         * tests/sunspider-1.0/crypto-md5.js:
295         * tests/sunspider-1.0/crypto-sha1.js:
296         * tests/sunspider-1.0/date-format-tofte.js:
297         * tests/sunspider-1.0/date-format-xparb.js:
298         * tests/sunspider-1.0/math-cordic.js:
299         * tests/sunspider-1.0/math-partial-sums.js:
300         (partial):
301         * tests/sunspider-1.0/math-spectral-norm.js:
302         * tests/sunspider-1.0/regexp-dna.js:
303         * tests/sunspider-1.0/string-base64.js:
304         * tests/sunspider-1.0/string-fasta.js:
305         (fastaRepeat):
306         (fastaRandom):
307         * tests/sunspider-1.0/string-tagcloud.js:
308         * tests/sunspider-1.0/string-unpack-code.js:
309         * tests/sunspider-1.0/string-validate-input.js:
310
311 2013-04-11  Filip Pizlo  <fpizlo@apple.com>
312
313         Get rid of the 10ms delay between test executions
314         https://bugs.webkit.org/show_bug.cgi?id=114458
315         <rdar://problem/13631112>
316
317         Reviewed by Darin Adler and Geoffrey Garen.
318         
319         More details on this in the bug. Long story short, I don't know why the 10ms delay is
320         there, and it appears to be the root of some noisyness and unreliability in the results.
321         Removing it makes results more consistent, which is a good thing.
322
323         * resources/driver-TEMPLATE.html:
324
325 2013-01-16  Eric Seidel  <eric@webkit.org>
326
327         Remove --shark* support from sunspider/run-sunspider now that Shark is dead
328         https://bugs.webkit.org/show_bug.cgi?id=99512
329
330         Reviewed by Darin Adler.
331
332         I tried to use --instruments, but that also seems broken (in the same way before/after this patch).
333
334         * sunspider:
335         (runTestsOnce):
336
337 2012-08-02  Filip Pizlo  <fpizlo@apple.com>
338
339         Unreviewed, revert debugging stuff I landed in http://trac.webkit.org/changeset/124557
340         by accident.
341
342         * tests/v8-v6/v8-crypto.js:
343         (am1):
344         (am2):
345         (am3):
346         (am4):
347         (bnpCopyTo):
348         (bnpFromString):
349         (bnpClamp):
350
351 2012-03-11  Maciej Stachowiak  <mjs@apple.com>
352
353         Add --instruments option to sunspider to profile with Instruments
354         https://bugs.webkit.org/show_bug.cgi?id=80783
355
356         Reviewed by Filip Pizlo.
357
358         * resources/TimeProfile20us.tracetemplate: Added.
359         * sunspider:
360         (runTestsOnce):
361
362 2011-11-15  Andy Wingo  <wingo@igalia.com>
363
364         Extend sunspider driver to be able to run kraken
365         https://bugs.webkit.org/show_bug.cgi?id=71799
366
367         Reviewed by Filip Pizlo.
368
369         * resources/sunspider-standalone-driver.js: Try to load a -data
370         file.  If that succeeds, we have a kraken-like test, so we time
371         the test using `load'.  Otherwise fall back to using `run'.
372
373 2011-08-11  Mark Rowe  <mrowe@apple.com>
374
375         Fix sunspider-compare-results to default to the same suite version as the sunspider script.
376
377         Rubber-stamped by Maciej Stachowiak.
378
379         * sunspider-compare-results:
380
381 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
382
383         Update SunSpider versioning to 1.0
384         https://bugs.webkit.org/show_bug.cgi?id=63870
385
386         Reviewed by Dan Bernstein.
387
388         * hosted/sunspider.html:
389         * hosted/versions.html:
390         * resources/driver-TEMPLATE.html:
391         * resources/results-TEMPLATE.html:
392         * sunspider:
393
394 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
395
396         Sunspider 0.9.1 harness doesn't actually close() its test documents
397         https://bugs.webkit.org/show_bug.cgi?id=47045
398
399         Reviewed by Daniel Bates.
400
401         * hosted/versions.html:
402         * resources/driver-TEMPLATE.html:
403
404 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
405
406         Not Reviewed. 
407         
408         Fix obviou typo in previous commit.
409
410         * tests/sunspider-1.0/bitops-bitwise-and.js:
411
412 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
413
414         Reviewed by Anders Carlsson.
415
416         SunSpider: all four bitops benchmarks can be replaced with NOP
417         https://bugs.webkit.org/show_bug.cgi?id=38446
418
419         * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
420         (TimeFunc): Save the result.
421         * tests/sunspider-1.0/bitops-bits-in-byte.js:
422         (TimeFunc): Save the result.
423         * tests/sunspider-1.0/bitops-bitwise-and.js: More explicitly
424         save the results.
425         * tests/sunspider-1.0/bitops-nsieve-bits.js:
426         (sieve): Save the result.
427
428 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
429
430         SunSpider: string-validate-input.js uses an undeclared variable named 'name', which is a DOM API
431         https://bugs.webkit.org/show_bug.cgi?id=60937
432
433         Reviewed by Dan Bernstein.
434
435         * tests/sunspider-1.0/string-validate-input.js:
436         (doTest): Rename the variable to username
437
438 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
439
440         Reviewed by Darin Adler.
441
442         SunSpider: The main function of math-cordic is dead code
443         https://bugs.webkit.org/show_bug.cgi?id=63863
444         
445         Try to prevent math-cordic from being eliminated as dead code or
446         compiled down to a constant.
447
448         * tests/sunspider-1.0/math-cordic.js:
449         (cordicsincos):
450         (cordic):
451
452 2011-06-30  Maciej Stachowiak  <mjs@apple.com>
453
454         Reviewed by Adam Barth.
455
456         Create sunspider-1.0 directory in preparation for fixing a bunch of bugs
457         https://bugs.webkit.org/show_bug.cgi?id=63782
458
459         * make-hosted: Learn about the new directory.
460         * tests/sunspider-1.0: Copied from PerformanceTests/SunSpider/tests/sunspider-0.9.1.
461
462 2010-11-29  Geoffrey Garen  <ggaren@apple.com>
463
464         Reviewed by Gavin Barraclough.
465
466         Improved accuracy of command-line SunSpider.
467
468         * resources/sunspider-standalone-driver.js: Use the returned number of
469         elapsed milliseconds from "run" and "checkSyntax" instead of measuring
470         ourselves, for slightly more accurate numbers.
471
472 2010-09-16  Gavin Barraclough  <barraclough@apple.com>
473
474         Reviewed by Mark Rowe.
475         https://bugs.webkit.org/show_bug.cgi?id=45924
476
477         Modify the SunSpider harness to allow a path to be passed to the --suite
478         argument, supress the normal prepend-"tests/" behaviour for suite names
479         containing a '/'.
480
481         Also wrap the code in an anonymous function to move variables out of global
482         scope (currently the harness cannot handle running tests the overwrite the
483         variable 'j').
484
485         * resources/sunspider-standalone-driver.js:
486         * sunspider:
487
488 2010-08-27  Michael Saboff  <msaboff@apple.com>
489
490         Reviewed by Stephanie Lewis.
491
492         Added version 5 and version 6 of V8 tests with modifications similar
493         to the version 4 tests so they'll work with the SunSpider test harness.
494         Note that these files are external source and therefore don't conform 
495         to WebKit coding standards.
496         https://bugs.webkit.org/show_bug.cgi?id=44638
497
498         * tests/v8-v5: Added.
499         * tests/v8-v5/LIST: Added.
500         * tests/v8-v5/v8-crypto.js: Added.
501         * tests/v8-v5/v8-deltablue.js: Added.
502         * tests/v8-v5/v8-earley-boyer.js: Added.
503         * tests/v8-v5/v8-raytrace.js: Added.
504         * tests/v8-v5/v8-regexp.js: Added.
505         * tests/v8-v5/v8-richards.js: Added.
506         * tests/v8-v5/v8-splay.js: Added.
507         * tests/v8-v6: Added.
508         * tests/v8-v6/LIST: Added.
509         * tests/v8-v6/v8-crypto.js: Added.
510         * tests/v8-v6/v8-deltablue.js: Added.
511         * tests/v8-v6/v8-earley-boyer.js: Added.
512         * tests/v8-v6/v8-raytrace.js: Added.
513         * tests/v8-v6/v8-regexp.js: Added.
514         * tests/v8-v6/v8-richards.js: Added.
515         * tests/v8-v6/v8-splay.js: Added.
516
517 2010-07-15  Stephanie Lewis  <slewis@apple.com>
518
519         Reviewed by Geoff Garen.
520
521         http://bugs.webkit.org/show_bug.cgi?id=42406
522         Output location of results file.
523
524         * sunspider:
525
526 2010-05-17  Maciej Stachowiak  <mjs@apple.com>
527
528         Reviewed by Mark Rowe.
529
530         Browser-hosted version of SunSpider runs tests twice as many times as advertised
531         https://bugs.webkit.org/show_bug.cgi?id=39201
532
533         * make-hosted: Make sure to reset test list after every test suite.
534
535 2010-05-07  Adam Roben  <aroben@apple.com>
536
537         Fix typo in sunspider-compare-results.js that caused incorrect
538         two-sample t statistics to be computed
539
540         Fixes <http://webkit.org/b/38768> SunSpider computes incorrect
541         two-sample t statistics
542
543         Reviewed by Darin Adler.
544
545         * resources/sunspider-compare-results.js:
546         (sunspiderCompareResults.resultLine): Fixed the calculation of the t
547         variable to match the definition of a two-sample t statistic. Looks
548         like this was just a simple typo.
549
550 2010-04-05  Darin Adler  <darin@apple.com>
551
552         Updated the ignore property for changes to the contents here.
553
554         * .: Modified property svn:ignore.
555
556 2010-02-20  Maciej Stachowiak  <mjs@apple.com>
557
558         Reviewed by Adam Barth.
559
560         Add sunspider-0.9.1 to the site in preparation for announcing it.
561         https://bugs.webkit.org/show_bug.cgi?id=35206
562
563         * hosted/sunspider.css: Remove tabs, which I noticed where present while
564         copying to the WebKitSite directory.
565
566 2010-01-11  Oliver Hunt  <oliver@apple.com>
567
568         Reviewed by Darin Adler.
569
570         Sunspider parse-only tests don't run
571         https://bugs.webkit.org/show_bug.cgi?id=33489
572
573         Fix parse-only test list
574
575         * tests/parse-only/LIST:
576
577 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
578
579         Reviewed by Adele Peterson.
580
581         Make SunSpider version more prominent in the title
582         https://bugs.webkit.org/show_bug.cgi?id=32574
583
584         * hosted/sunspider.html:
585         * hosted/versions.html:
586         * resources/driver-TEMPLATE.html:
587         * resources/results-TEMPLATE.html:
588
589 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
590
591         Reviewed by Darin Adler.
592
593         Make SunSpider refuse to compare results across content versions
594         https://bugs.webkit.org/show_bug.cgi?id=32573
595         
596         The results URL now includes a version indicator, and compare mode
597         will refuse to compare between versions.
598
599         * resources/driver-TEMPLATE.html:
600         * resources/results-TEMPLATE.html:
601
602 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
603
604         Reviewed by Darin Adler.
605
606         Some Browser-hosted SunSpider files are not valid HTML5
607         https://bugs.webkit.org/show_bug.cgi?id=32536
608         
609         Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
610         close tags and adding a meta charset declaration).
611
612         * hosted/sunspider.html:
613         * hosted/versions.html:
614         * resources/TEMPLATE.html:
615         * resources/driver-TEMPLATE.html:
616         * resources/results-TEMPLATE.html:
617
618 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
619
620         Reviewed by Eric Seidel.
621
622         Make sunspider-0.9.1 the default content set (both command-line and hosted)
623         https://bugs.webkit.org/show_bug.cgi?id=32537
624
625         * hosted/sunspider.html: Change default suite link.
626         * sunspider: Change default suite.
627         * sunspider-compare-results: Likewise.
628
629 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
630
631         Reviewed by Sam Weinig.
632
633         in-browser SunSpider suffers excessive penalty under power management
634         https://bugs.webkit.org/show_bug.cgi?id=32505
635         
636         I have made a few changes to address this:
637         
638         1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
639         document.write to reduce triggering of progress UI.
640         2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
641         stepped all the way down the whole time.
642         3) Run the test cycle an extra time for warmup before the runs that count.
643
644         * hosted/sunspider-record-result.js: Removed.
645         * make-hosted:
646         * resources/TEMPLATE.html:
647         * resources/driver-TEMPLATE.html:
648
649 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
650
651         Reviewed by Gavin Barraclough.
652
653         SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
654         https://bugs.webkit.org/show_bug.cgi?id=16806
655
656         Based on a patch by Eric Seidel.
657         
658         Fix the base64 computation to actually compute correct results. The impact on runtime of
659         the test is pretty small, but noticeable for some browsers. But at least it's not
660         doing a wrong and meaningless computation any more.
661         
662         * tests/sunspider-0.9.1/string-base64.js:
663         ():
664         (base64ToString):
665
666 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
667
668         Fixing commit error...
669         
670         I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
671
672         * resources/driver-TEMPLATE.html:
673
674 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
675
676         Reviewed by Sam Weinig.
677
678         Use JSON.parse instead of eval for Web-hosted SunSpider results processing
679         https://bugs.webkit.org/show_bug.cgi?id=32490
680
681         * hosted/json2.js: Added. Incorporated from json.org
682         * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
683         * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
684         results.
685
686 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
687
688         Reviewed by Oliver Hunt.
689
690         Enable Web-hosted version of SunSpider to handle multiple versions
691         https://bugs.webkit.org/show_bug.cgi?id=32478
692
693         * make-hosted: Now generate the hosted version to be able to run both the
694         0.9 and the 0.9.1 test suites.
695         * hosted: Modified property svn:ignore.
696         * hosted/sunspider-driver.html: Removed.
697         * hosted/sunspider-results.html: Removed.
698         * hosted/sunspider.html:
699         * hosted/versions.html: Added.
700         * resources/TEMPLATE.html:
701         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
702         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
703
704 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
705
706         Reviewed by Oliver Hunt.
707
708         Give command-line sunspider the ability to handle multiple suites and versions
709         https://bugs.webkit.org/show_bug.cgi?id=32477
710         
711         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
712         enumerating the tests. Current suites are:
713         
714         sunspider-0.9 - riginal SunSpider content 
715         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
716         ubench - "ubench" microbenchmark suite
717         v8-v4 - version 4 of the v8 benchmark
718         parse-only - some parser tests
719         
720         Sunspider can now be run with the --suite= parameter to select a
721         suite. The default is --suite=sunspider-0.9.
722
723         * .: Modified property svn:ignore to ignore new results directories.
724         * make-hosted: Generate based on sunspider-0.9 suite.
725         * resources/TEMPLATE.html: Fix an HTML compliance problem.
726         * resources/sunspider-standalone-driver.js: Become aware of suite names.
727         * sunspider: Modified to handle multiple suites as described above.
728         * sunspider-compare-results: ditto
729         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
730         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
731         * tests/LIST-UBENCH: Removed.
732         * tests/LIST-V8: Removed.
733         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
734         * tests/3d-morph.js: ditto
735         * tests/3d-raytrace.js: ditto
736         * tests/access-binary-trees.js: ditto
737         * tests/access-fannkuch.js: ditto
738         * tests/access-nbody.js: ditto
739         * tests/access-nsieve.js: ditto
740         * tests/bitops-3bit-bits-in-byte.js: ditto
741         * tests/bitops-bits-in-byte.js: ditto
742         * tests/bitops-bitwise-and.js: ditto
743         * tests/bitops-nsieve-bits.js: ditto
744         * tests/controlflow-recursive.js: ditto
745         * tests/crypto-aes.js: ditto
746         * tests/crypto-md5.js: ditto
747         * tests/crypto-sha1.js: ditto
748         * tests/date-format-tofte.js: ditto
749         * tests/date-format-xparb.js: ditto
750         * tests/math-cordic.js: ditto
751         * tests/math-partial-sums.js: ditto
752         * tests/math-spectral-norm.js: ditto
753         * tests/regexp-dna.js: ditto
754         * tests/string-base64.js: ditto
755         * tests/string-fasta.js: ditto
756         * tests/string-tagcloud.js: ditto
757         * tests/string-unpack-code.js: ditto
758         * tests/string-validate-input.js: ditto
759         * tests/sunspider-0.9/: Added
760         * tests/sunspider-0.9/*.js: Moved from one level up
761         * tests/sunspider-0.9.1: Added.
762         * tests/sunspider-0.9.1/*.js: Moved from one level up.
763         * tests/v8-crypto.js: Moved to v8-v4/ directory.
764         * tests/v8-deltablue.js: ditto
765         * tests/v8-earley-boyer.js: ditto
766         * tests/v8-raytrace.js: ditto
767         * tests/v8-regexp.js: ditto
768         * tests/v8-richards.js: ditto
769         * tests/v8-splay.js: ditto
770         * tests/v8-v4: Added.
771
772 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
773
774         Reviewed by Darin Adler.
775
776         Platform specific null device replaced with a platform independent.
777         https://bugs.webkit.org/show_bug.cgi?id=29544
778
779         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
780
781 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
782
783         Reviewed by Maciej Stachowiak.
784
785         [Qt] Make sunspider script work on Windows platform.
786         https://bugs.webkit.org/show_bug.cgi?id=29544
787
788         * sunspider:
789         Using platform specific null device instead of /dev/null.
790
791 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
792
793         Reviewed by Oliver Hunt.
794
795         https://bugs.webkit.org/show_bug.cgi?id=26540
796         Currently the SunSpider test driver lacks an option to run a test suite that
797         will test JavaScriptCore parsing performance only.  This patch adds just such
798         a test suite and option to SunSpider as well as the jsc test shell.  I've included
799         three large javascript source files found in the wild: jquery, mootools and prototype.
800         Combined with the concatenation of all three, these form a new testsuite to measure
801         and test pure JavaScriptCore parsing performance.
802
803         * resources/sunspider-standalone-driver.js:
804           Load the js with parse-only flag if it is part of the parse-only suite.
805         * sunspider:
806           Add the parse-only suite to the list of options.
807         * sunspider-compare-results:
808           Add the parse-only suite to the list of options.
809         * tests/LIST-PARSE-ONLY: Added.
810         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
811           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
812         * tests/parse-only/jquery-1.3.2.js: Added.
813           Latest version from jquery.com.
814         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
815           Latest version from mootools.net.
816         * tests/parse-only/prototype-1.6.0.3.js: Added.
817           Latest version from prototypejs.org.
818
819 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
820
821         Update V8 benchmark to version 4.
822         
823         Reviewed by Cameron Zwarich.
824
825         * tests/LIST-V8:
826         * tests/v8-crypto.js:
827         * tests/v8-deltablue.js:
828         * tests/v8-earley-boyer.js:
829         * tests/v8-raytrace.js:
830         * tests/v8-regexp.js: Added.
831         * tests/v8-richards.js:
832         * tests/v8-splay.js: Added.
833
834 2009-05-26  Olivier DOLE  <odole@pleyo.com>
835
836         Reviewed by Maciej.
837
838         Store sunspider test suite results in a specific directory for each
839         different type of benchmark (sunspider, V8 or ubench).
840
841         * sunspider:
842         * sunspider-compare-results:
843
844 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
845
846         Build fix.
847
848         * SunSpider.make:
849
850 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
851
852         Include SunSpider tests in production builds.
853         
854         Reviewed by Adam Roben.
855
856         * SunSpider.make: Added.
857         * xcopy.excludes: Added.
858
859 2009-03-04  Oliver Hunt  <oliver@apple.com>
860
861         Added a few more tests to ubench
862
863         RS=Gavin Barraclough
864
865         Adding a couple of test to cover too few and too many arguments.  This
866         is needed to prevent regressions in all call paths from my current refactoring.
867
868         * tests/LIST-UBENCH:
869         * tests/ubench/function-correct-args.js: Added.
870         (f):
871         * tests/ubench/function-excess-args.js: Added.
872         (f):
873
874 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
875
876         Fix a bug where when V8's or UBENCH's test suite list file is missing
877         the error message is suggesting that SunSpider's list file is missing.
878
879         Reviewed by Sam Weinig.
880
881         * sunspider:
882         (loadTestsList): Fix the error message to use the right list name.
883
884 2008-10-07  David Hyatt  <hyatt@apple.com>
885
886         https://bugs.webkit.org/show_bug.cgi?id=18989
887
888         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
889         we want all browsers to be doing the same thing on the test.
890
891         Reviewed by Maciej
892
893         * tests/regexp-dna.js:
894
895 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
896
897         Rubber-stamped by Maciej Stachowiak.
898
899         The bleeding edge branch of V8 now supports passing fileanmes after
900         '-f', so the SunSpider script should be changed to do this again.
901         This allows all 3 major JavaScript engines to be used with the
902         unmodified SunSpider script.
903
904         * sunspider:
905
906 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
907
908         Reviewed by Maciej Stachowiak.
909
910         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
911         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
912         correct the punctuation in its documentation.
913
914         * sunspider:
915
916 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
917
918         Reviewed by Oliver Hunt.
919
920         Bug 20616: Incorporate V8 benchmarks in testing
921         <https://bugs.webkit.org/show_bug.cgi?id=20616>
922
923         This is a first attempt at running the V8 benchmarks within SunSpider,
924         with the --v8 option. The number of iterations is fixed for each
925         benchmark, and was chosen to make each of the benchmarks run between
926         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
927
928         * sunspider:
929         * tests/LIST-V8: Added.
930         * tests/v8-crypto.js: Added.
931         * tests/v8-deltablue.js: Added.
932         * tests/v8-earley-boyer.js: Added.
933         * tests/v8-raytrace.js: Added.
934         * tests/v8-richards.js: Added.
935
936 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
937
938         Not reviewed.
939
940         Remove a local change that was accidentally committed in r36034.
941
942         * resources/sunspider-standalone-driver.js:
943
944 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
945
946         Rubber-stamped by Oliver Hunt.
947
948         Make SunSpider work with JavaScript shells that don't take filenames
949         after a '-f' argument.
950
951         * sunspider:
952
953 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
954
955         Reviewed by Timothy.
956
957         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
958
959         * sunspider:
960         * tests/LIST-SQUIRRELFISH: Removed.
961
962 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
963
964         Rubber-stamped by Mark Rowe.
965
966         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
967
968         * sunspider:
969
970 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
971
972         Rubber stamped by Oliver.
973         
974         - add newly working tests to squirrelfish list
975
976         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
977
978 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
979
980         Rubber stamped by Oliver.
981         
982         - move ubench tests properly.
983
984         * tests/ubench: Added.
985         * tests/ubench/function-closure.js: Added.
986         * tests/ubench/function-empty.js: Added.
987         * tests/ubench/function-missing-args.js: Added.
988         * tests/ubench/function-sum.js: Added.
989         * tests/ubench/loop-empty-resolve.js: Added.
990         * tests/ubench/loop-empty.js: Added.
991         * tests/ubench/loop-sum.js: Added.
992
993 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
994
995         Rubber stamped by Oliver.
996
997         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
998
999 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
1000
1001         Reviewed by Oliver.
1002
1003         - make --squirrelfish mode use the real harness, but a reduced set of tests
1004
1005         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
1006
1007         * sunspider:
1008         * tests/LIST-SQUIRRELFISH:
1009         * tests/LIST-UBENCH: Added.
1010         * tests/squirrelfish: Moved to tests/ubench
1011
1012 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
1013
1014         Reviewed by Maciej Stachowiak.
1015         
1016         Removed an unused test.
1017
1018         * tests/squirrelfish/loop-resolve.js: Removed.
1019
1020 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
1021
1022         Reviewed by Oliver.
1023         
1024         Add newly runnable tests to --squirrelfish mode.
1025
1026         * tests/LIST-SQUIRRELFISH:
1027
1028 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
1029
1030         Reviewed by Maciej Stachowiak.
1031         
1032         A little more love for --squirrelfish mode:
1033         
1034         Fixed a misplaced sort. Results properly sort now.
1035         
1036         Added a test list just for squirrelfish, and updated it to include
1037         all currently passing SunSpider tests.
1038         
1039         Fixed the pruning regexp to match 3d-morph.js.
1040
1041         * sunspider:
1042
1043 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
1044
1045         Reviewed by Oliver Hunt.
1046         
1047         --squirrelfish mode: pared down tests for squirrelfish to chew on.
1048
1049         * sunspider:
1050         * tests/LIST:
1051         * tests/squirrelfish: Added.
1052         * tests/squirrelfish/function-closure.js: Added.
1053         * tests/squirrelfish/function-empty.js: Added.
1054         * tests/squirrelfish/function-missing-args.js: Added.
1055         * tests/squirrelfish/function-sum.js: Added.
1056         * tests/squirrelfish/loop-empty-resolve.js: Added.
1057         * tests/squirrelfish/loop-empty.js: Added.
1058         * tests/squirrelfish/loop-resolve.js: Added.
1059         * tests/squirrelfish/loop-sum.js: Added.
1060
1061 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
1062
1063         Reviewed by Geoff.
1064
1065         * sunspider: Pause update daemon on Darwin to make results reliable
1066         again (workaround for <rdar://problem/5811127>).
1067
1068 2008-02-07  Eric Seidel  <eric@webkit.org>
1069
1070         Reviewed by Sam.
1071
1072         * sunspider: add an error message when ./tests/LIST is not found
1073
1074 2007-12-29  Alp Toker  <alp@atoker.com>
1075
1076         Fix typo.
1077
1078         * hosted/sunspider.html:
1079
1080 2007-12-29  Alp Toker  <alp@atoker.com>
1081
1082         Reviewed by Maciej.
1083
1084         Provide the current URL in the SunSpider results page. Useful for
1085         copying benchmark results out of browsers without a location bar or
1086         where the location bar can't handle long URLs.
1087
1088         * hosted/sunspider-results.html:
1089
1090 2007-12-20  Eric Seidel  <eric@webkit.org>
1091
1092         Reviewed by mjs.
1093
1094         * sunspider: increase default number of runs for more accurate numbers
1095
1096 2007-12-18  Mark Rowe  <mrowe@apple.com>
1097
1098         Fix typo.
1099
1100         * hosted/sunspider.html:
1101
1102 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
1103
1104         Reviewed by Eric.
1105         
1106         - give an extra digit of precision for small differences.
1107
1108         * resources/sunspider-compare-results.js:
1109
1110 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
1111
1112         Reviewed by Eric.
1113         
1114         - work around IE layout bugs
1115
1116         * hosted/sunspider.css:
1117
1118 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
1119
1120         Reviewed by Mark.
1121         
1122         - fix compare feature
1123
1124         * hosted/sunspider-results.html:
1125
1126 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
1127
1128         Reviewed by Mark.
1129         
1130         - Change the format of the output URL to be shorter, to meet IE constraints
1131
1132         * hosted/sunspider-driver.html:
1133         * hosted/sunspider-results.html:
1134
1135 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
1136
1137         Reviewed by Oliver.
1138         
1139         - Add some explanatory test and a dumb logo to SunSpider web hosted version
1140
1141         * hosted/sunspider-driver.html:
1142         * hosted/sunspider-results.html:
1143         * hosted/sunspider.css:
1144         * hosted/sunspider.html:
1145
1146 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
1147
1148         Reviewed by Oliver.
1149         
1150         - Make the web-hosted version of SunSpider look a little bit less drab.
1151
1152         * hosted/sunspider-driver.html:
1153         * hosted/sunspider-results.html:
1154         * hosted/sunspider.css: Added.
1155         * hosted/sunspider.html:
1156         * resources/TEMPLATE.html:
1157
1158 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
1159
1160         Reviewed by Sam.
1161         
1162         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
1163         
1164         A sample of what new-style results would look like:
1165         
1166         TEST                   COMPARISON            FROM                 TO             DETAILS
1167         ...
1168         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
1169         ...
1170             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
1171         
1172         * resources/sunspider-compare-results.js:
1173
1174 2007-11-14  Eric Seidel  <eric@webkit.org>
1175
1176         Reviewed by Sam.
1177
1178         * sunspider: add --shark-cache for L2 Cache Miss profiling
1179
1180 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
1181
1182         Reviewed by Darin Adler.
1183
1184         http://bugs.webkit.org/show_bug.cgi?id=15925
1185         SunSpider should check for Shark being installed
1186
1187         * sunspider: Explicitly test that shark command line tool is installed.
1188
1189 2007-11-04  Darin Adler  <darin@apple.com>
1190
1191         * hosted/sunspider.html: Tweaked formatting a bit.
1192
1193 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
1194
1195         Reviewed by Darin Adler.
1196
1197         - Add compare mode and a description of the benchmark to the hosted version.
1198
1199         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
1200         * hosted/sunspider.html: Added intro text.
1201         * make-hosted: Copy new file.
1202         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
1203         a function.
1204         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
1205         expects.
1206         * sunspider-compare-results: Use sunspider-standalone-compare.js.
1207
1208 2007-11-02  Darin Adler  <darin@apple.com>
1209
1210         * sunspider: Changed "--base" to "--set-baseline".
1211
1212 2007-11-02  Darin Adler  <darin@apple.com>
1213
1214         Reviewed by Maciej.
1215
1216         * sunspider: Added a "--base" option that records the current test as a baseline.
1217         Moved the code to find the newest .mshark file into a function (since I needed to use
1218         it in the other script and I prefer to copy/paste a function).
1219
1220         * sunspider-compare-results: Added logic to compare the baseline with the last
1221         run when you don't pass any parameters.
1222
1223 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
1224
1225         Reviewed by Eric.
1226         
1227         - Added list of ECMAScript3 features that SunSpider does not cover currently.
1228
1229         * UNCOVERED: Added.
1230
1231 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
1232
1233         Reviewed by Oliver.
1234         
1235         - Give better error messages when sunspider-compare-results fails
1236
1237         * sunspider-compare-results:
1238
1239 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
1240
1241         Reviewed by Darin Adler.
1242         
1243         - fix Date tests to work in command-line SpiderMonkey
1244
1245         * tests/date-format-tofte.js:
1246         * tests/date-format-xparb.js:
1247
1248 2007-10-29  Darin Adler  <darin@apple.com>
1249
1250         Reviewed by Maciej.
1251
1252         * sunspider: Added --shark20 option, to run Shark at its highest sample
1253         resolution instead of its default.
1254
1255 2007-10-28  Adam Roben  <aroben@apple.com>
1256
1257         Fix SunSpider on Windows
1258
1259         Reviewed by Eric.
1260
1261         * sunspider: Don't use colons in filenames.
1262
1263 2007-10-28  Darin Adler  <darin@apple.com>
1264
1265         Reviewed by Adam.
1266
1267         * resources/sunspider-analyze-results.js: Tweak the output format a little.
1268         Change so that when there's only one run we don't write out confidence
1269         intervals at all rather than writing "NaN" over and over again.
1270
1271         * sunspider: Use the time and date as a suffix on the results file, that way
1272         all the results are left behind in the tmp directory. This will make it easy
1273         for us to add features that compare past results in the future. Also rename
1274         the .mshark files using the same scheme.
1275
1276         * sunspider-compare-results: Relax the parsing rules so we can parse the
1277         results file exactly as it's emitted from sunspider.
1278
1279 2007-10-25  Adam Roben  <aroben@apple.com>
1280
1281         Strip carriage returns from results
1282
1283         Carriage returns were screwing up sunspider-compare-results on
1284         Windows.
1285
1286         Rubberstamped by Oliver.
1287
1288         * sunspider:
1289
1290 2007-10-24  Darin Adler  <darin@apple.com>
1291
1292         Reviewed by Eric.
1293
1294         * sunspider: Open the profile in shark after completing a run.
1295         * sunspider-compare-results: Tiny perl tweaks.
1296
1297 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
1298
1299         Reviewed by Darin Adler.
1300
1301         - Don't hardcode my path to testkjs
1302         
1303         * sunspider-compare-results:
1304
1305 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
1306
1307         Reviewed by Darin Adler.
1308         
1309         - Add a compare script to compare results, and improve formatting of standard results a bit.
1310
1311         * resources/sunspider-analyze-results.js:
1312         * resources/sunspider-compare-results.js: Added.
1313         * sunspider-compare-results: Added.
1314
1315 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
1316
1317         Reviewed by Darin Adler.
1318
1319         * resources/sunspider-analyze-results.js: Do the error range
1320         properly, using the t-distribution instead of 1.96 (which was
1321         based on the normal distribution).
1322         
1323         * sunspider: Print results in a way that will be friendlier to the
1324         soon-to-come compare mode.
1325
1326 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
1327
1328         Reviewed by Review.
1329         
1330         - add more tests (probably the final set for now)
1331         
1332         Already balanced. Added date, regexp, control flow, and a few
1333         more string and object/array access tests.
1334         
1335         * tests/LIST:
1336         * tests/access-fannkuch.js: Added.
1337         * tests/access-nbody.js: Added.
1338         * tests/controlflow-recursive.js: Added.
1339         * tests/date-format-tofte.js: Added.
1340         * tests/date-format-xparb.js: Added.
1341         * tests/regexp-dna.js: Added.
1342         * tests/string-validate-input.js: Added.
1343
1344 2007-10-21  Eric Seidel  <eric@webkit.org>
1345
1346         Reviewed by mjs.
1347
1348         * sunspider: make --shark only sample from testkjs
1349
1350 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
1351
1352         Rubber stamped by Eric.
1353         
1354         - added crypto tests (already balanced)
1355
1356         * tests/LIST:
1357         * tests/crypto-aes.js: Added.
1358         * tests/crypto-md5.js: Added.
1359         * tests/crypto-sha1.js: Added.
1360
1361 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
1362
1363         Reviewed by Adam.
1364         
1365         - rebalance new tests
1366         
1367         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
1368         
1369         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1370
1371         This balances them with the existing tests.
1372
1373         * tests/access-binary-trees.js:
1374         * tests/access-nsieve.js:
1375         * tests/bitops-nsieve-bits.js:
1376         * tests/math-partial-sums.js:
1377         * tests/math-spectral-norm.js:
1378         * tests/string-fasta.js:
1379
1380 2007-10-20  Eric Seidel  <eric@webkit.org>
1381
1382         Reviewed by darin.
1383         
1384         Add improved argument handling to sunspider, including
1385         --runs=<number>, --tests=<pattern, --shark, and --help
1386         Also re-factor code into subroutines
1387
1388         * sunspider: improved argument handling
1389
1390 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1391
1392         Reviewed by Mark.
1393
1394         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
1395         
1396         * TODO:
1397         * tests/LIST:
1398         * tests/access-binary-trees.js: Added.
1399         * tests/access-nsieve.js: Added.
1400         * tests/bitops-nsieve-bits.js: Added.
1401         * tests/math-partial-sums.js: Added.
1402         * tests/math-spectral-norm.js: Added.
1403         * tests/string-fasta.js: Added.
1404
1405 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1406
1407         Reviewed by Darin Adler.
1408
1409         * TODO: Updated to note areas that are now well-covered.
1410         
1411         - Change the tests and the driver to avoid leaving large object
1412         graphs hanging around, since that throws off the subsequent tests.
1413         
1414         * resources/sunspider-standalone-driver.js:
1415         * tests/3d-cube.js:
1416         * tests/3d-morph.js:
1417         * tests/3d-raytrace.js:
1418         * tests/bitops-bitwise-and.js:
1419         * tests/string-base64.js:
1420         * tests/string-tagcloud.js:
1421
1422 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1423
1424         Reviewed by Nikolas.
1425         
1426         - Rebalance test complexity.
1427         
1428         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
1429         
1430         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1431
1432         However, for some tests, IE7 was a huge outlier compared to other
1433         major browsers. For those tests, I used Firefox 2.0.0.8 on the
1434         same OS and hardware as a normalization basis.
1435         
1436         * tests/3d-morph.js:
1437         * tests/3d-raytrace.js:
1438         * tests/bitops-3bit-bits-in-byte.js:
1439         * tests/bitops-bits-in-byte.js:
1440         * tests/bitops-bitwise-and.js:
1441         * tests/math-cordic.js:
1442         * tests/string-base64.js:
1443         * tests/string-tagcloud.js:
1444         * tests/string-unpack-code.js:
1445
1446 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1447
1448         Reviewed by Oliver.
1449         
1450         - Fix the browser-hosted driver to be compatible with Safari 2
1451
1452         * hosted/sunspider-driver.html:
1453
1454 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1455
1456         Rubber stamped by Hyatt.
1457         
1458         - Add BSD license notice to cordic test, the original author said to do so.
1459
1460         * tests/math-cordic.js:
1461
1462 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1463
1464         Reviewed by Eric.
1465         
1466         - Support Windows and Windows browsers.
1467
1468         * hosted/sunspider-results.html: Display results correctly in IE.
1469         * sunspider: tweaks to work on Windows, and with spaces in the path.
1470
1471 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1472
1473         Rubber stamped by Oliver.
1474
1475         * tests/string-tagcloud.js: Remove stray print()
1476
1477 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
1478
1479         Reviewed by Oliver.
1480         
1481         Add some string test cases.
1482
1483         * tests/string-base64.js: Added. Does base64 encoding and decoding.
1484         * tests/string-tagcloud.js: Added. Parses a JSON string and
1485         generates tagcloud HTML markup based on the data.        
1486         * tests/string-unpack-code.js: Added. Unpack compressed versions of
1487         four of the most popular JavaScript libraries.
1488         * tests/LIST: Added the new tests.
1489
1490 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
1491
1492         Reviewed by Anders.
1493
1494         * TODO: Updated with info on how to do the stats properly.
1495
1496         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
1497         avoid overly slowing down the benchmark.
1498
1499         Added some 3D-related test cases.
1500         
1501         * tests/3d-cube.js: Added.
1502         * tests/3d-morph.js: Added.
1503         * tests/3d-raytrace.js: Added.
1504         * tests/LIST:
1505
1506 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
1507
1508         Reviewed by Darin Adler.
1509
1510         - New JavaScript benchmark
1511         http://bugs.webkit.org/show_bug.cgi?id=15515
1512         
1513         A start on a new JavaScript benchmark, with standalone and
1514         browser-hosted drivers.
1515         
1516         The test content is still incomplete and the drivers could use
1517         some more features but there is enough here to make a good start.
1518         
1519         The drivers compute a 95% confidence interval on the mean for the
1520         whole test, each category, and each individual test to make it
1521         easier to tell whether differences are statistically
1522         significant. The confidence interval can be narrowed by running
1523         the test more times.
1524         
1525         * TODO: Added. Remaining things that need to be done.
1526         * sunspider: Added. Perl script that acts as the standalone test driver.
1527         * resources/sunspider-standalone-driver.js: Added. JavaScript part
1528         of standalone test driver.
1529         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
1530         analysis code.
1531         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
1532         * make-hosted: Added. Script to generate browser-hosted tests.
1533         * hosted/sunspider.html: Added. Start page for browser-hosted test.
1534         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
1535         * hosted/sunspider-results.html: Added. Results page for browser-hosted
1536         tests.
1537         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
1538         tests.
1539         * tests/LIST: Added. List of tests to use.
1540         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
1541         * tests/bitops-bits-in-byte.js: Added. Ditto.
1542         * tests/bitops-bitwise-and.js: Added. Ditto.
1543         * tests/math-cordic.js: Added. Ditto.
1544