Unreviewed, roll out http://trac.webkit.org/changeset/169159.
[WebKit-https.git] / PerformanceTests / SunSpider / ChangeLog
1 2014-05-21  Filip Pizlo  <fpizlo@apple.com>
2
3         Unreviewed, roll out http://trac.webkit.org/changeset/169159.
4         
5         This was a unilateral change and wasn't properly reviewed.
6
7         * profiler-test.yaml:
8
9 2014-05-19  Filip Pizlo  <fpizlo@apple.com>
10
11         Take care of some ARM64 test failures
12         https://bugs.webkit.org/show_bug.cgi?id=133090
13
14         Reviewed by Geoffrey Garen.
15
16         * profiler-test.yaml: Skip these on not-x86.
17
18 2014-03-14  Maciej Stachowiak  <mjs@apple.com>
19
20         Replace "Apple Computer, Inc." with "Apple Inc." in copyright headers
21         https://bugs.webkit.org/show_bug.cgi?id=130276
22         <rdar://problem/16266927>
23
24         Reviewed by Simon Fraser.
25
26         * hosted/sunspider.html:
27         * hosted/versions.html:
28         * make-hosted:
29         * resources/TEMPLATE.html:
30         * resources/driver-TEMPLATE.html:
31         * resources/results-TEMPLATE.html:
32         * resources/sunspider-analyze-results.js:
33         * resources/sunspider-compare-results.js:
34         * resources/sunspider-standalone-compare.js:
35         * resources/sunspider-standalone-driver.js:
36         * sunspider:
37         * sunspider-compare-results:
38         * tests/sunspider-0.9.1/3d-morph.js:
39         * tests/sunspider-0.9.1/3d-raytrace.js:
40         * tests/sunspider-0.9.1/bitops-bitwise-and.js:
41         * tests/sunspider-0.9.1/math-cordic.js:
42         * tests/sunspider-0.9.1/string-tagcloud.js:
43         * tests/sunspider-0.9/3d-morph.js:
44         * tests/sunspider-0.9/3d-raytrace.js:
45         * tests/sunspider-0.9/bitops-bitwise-and.js:
46         * tests/sunspider-0.9/math-cordic.js:
47         * tests/sunspider-0.9/string-tagcloud.js:
48         * tests/sunspider-1.0.1/3d-morph.js:
49         * tests/sunspider-1.0.1/3d-raytrace.js:
50         * tests/sunspider-1.0.1/bitops-bitwise-and.js:
51         * tests/sunspider-1.0.1/math-cordic.js:
52         * tests/sunspider-1.0.1/string-tagcloud.js:
53         * tests/sunspider-1.0.2/3d-morph.js:
54         * tests/sunspider-1.0.2/3d-raytrace.js:
55         * tests/sunspider-1.0.2/bitops-bitwise-and.js:
56         * tests/sunspider-1.0.2/math-cordic.js:
57         * tests/sunspider-1.0.2/string-tagcloud.js:
58         * tests/sunspider-1.0/3d-morph.js:
59         * tests/sunspider-1.0/3d-raytrace.js:
60         * tests/sunspider-1.0/bitops-bitwise-and.js:
61         * tests/sunspider-1.0/math-cordic.js:
62         * tests/sunspider-1.0/string-tagcloud.js:
63
64 2014-02-10  Filip Pizlo  <fpizlo@apple.com>
65
66         Unreviewed, revert the accidental change from r163789.
67
68         * tests/v8-v6/v8-deltablue.js:
69
70 2013-12-10  Filip Pizlo  <fpizlo@apple.com>
71
72         Get rid of forward exit on DoubleAsInt32
73         https://bugs.webkit.org/show_bug.cgi?id=125552
74
75         Reviewed by Oliver Hunt.
76         
77         Use SunSpider as a kind of spot-check for the
78         no-architecture-specific-optimization paths in the compiler.
79
80         * no-architecture-specific-optimizations.yaml: Added.
81
82 2013-10-08  Geoffrey Garen  <ggaren@apple.com>
83
84         Refined power management in SunSpider 1.0.2
85         https://bugs.webkit.org/show_bug.cgi?id=122525
86
87         Reviewed by Gavin Barraclough.
88
89         * hosted/sunspider.css:
90         (#frameparent):
91         (#countdown):
92         * resources/TEMPLATE.html: Provide an explicit countdown element in the
93         parent, instead of hacking it into each child frame.
94
95         * resources/driver-TEMPLATE.html: Be more explicit about how we respond
96         to power management. Just moving setTimeout(0) inside the loop is a
97         little too subtle, and may not work across browsers.
98
99         Also, leave a little extra time after changing the countdown graphic,
100         to give the browser some time to paint.
101
102 2013-10-02  Geoffrey Garen  <ggaren@apple.com>
103
104         Improved painting behavior in SunSpider 1.0.2
105         https://bugs.webkit.org/show_bug.cgi?id=122257
106
107         Reviewed by Oliver Hunt.
108
109         Individual test names with times was a good painting strategy years
110         back, when each benchmark took hundreds of milliseconds to run. Now,
111         the whole suite can finish in a a few hundres milliseconds, and things
112         look pretty flashy.
113
114         Instead, display a countdown that indicates which iteration of the
115         test we're on.
116
117         * resources/TEMPLATE.html: Work around a bug in Firefox by specifying
118         our style inside the .html file instead of in an external stylesheet.
119         Under heavy JavaScript load, Firefox neglects to apply external
120         stylesheets -- probably due to a starved timer.
121         
122         * resources/driver-TEMPLATE.html: Delay a little between runs to ensure
123         that our countdown has enough time to paint.
124
125 2013-09-26  Geoffrey Garen  <ggaren@apple.com>
126
127         Fixed a version number string in SunSpider 1.0.2
128         https://bugs.webkit.org/show_bug.cgi?id=122004
129
130         Reviewed by Gavin Barraclough.
131
132         * resources/driver-TEMPLATE.html:
133         * resources/results-TEMPLATE.html: 1.0.2!
134
135 2013-09-26  Geoffrey Garen  <ggaren@apple.com>
136
137         Tentative SunSpider 1.0.2
138         https://bugs.webkit.org/show_bug.cgi?id=122002
139
140         Reviewed by Gavin Barraclough.
141
142         Be even more aggressive about asking for zero delay between tests by
143         setting a timeout at the beginning of a test instead of the end. This is
144         another step toward reducing noise -- particularly noise caused by
145         power management -- just like the step down from 10ms to 0ms.
146
147         Browsers typically put a 4ms floor on timers. The sooner we set
148         our timer, the sooner it will fire.
149
150         * make-hosted:
151         * resources/driver-TEMPLATE.html: Changed the driver to set the timeout
152         earlier.
153         * tests/sunspider-1.0.2: Added. These tests are unchanged from 1.0.1.
154
155 2013-09-09  Filip Pizlo  <fpizlo@apple.com>
156
157         Stress tests should test the jsc profiler (-p)
158         https://bugs.webkit.org/show_bug.cgi?id=121043
159
160         Reviewed by Mark Hahnenberg.
161         
162         Add a jsc-stress-test that tries to profile SunSpider.
163
164         * profiler-test.yaml: Added.
165
166 2013-08-08  Cosmin Truta  <ctruta@blackberry.com>
167
168         SunSpider: Move the 3d-morph fix from version 1.0 to version 1.0.1
169         https://bugs.webkit.org/show_bug.cgi?id=117993
170
171         Reviewed by George Staikos.
172
173         Bump the SunSpider version to 1.0.1.
174
175         Unroll r149548 in order to restore version 1.0 to its original,
176         and add the 3d-morph fix to version 1.0.1 instead.
177
178         * hosted/sunspider.html: Updated to version 1.0.1.
179         * hosted/versions.html: Updated to version 1.0.1.
180         * make-hosted: Added support for version 1.0.1.
181         * resources/driver-TEMPLATE.html: Updated to version 1.0.1.
182         * resources/results-TEMPLATE.html: Updated to version 1.0.1.
183         * sunspider: Updated to version 1.0.1.
184         * sunspider-compare-results: Updated to version 1.0.1.
185         * tests/sunspider-1.0/3d-morph.js: Rolled back to the original version.
186         * tests/sunspider-1.0.1/LIST: Added.
187         * tests/sunspider-1.0.1/3d-cube.js: Added.
188         * tests/sunspider-1.0.1/3d-morph.js: Added.
189         * tests/sunspider-1.0.1/3d-raytrace.js: Added.
190         * tests/sunspider-1.0.1/access-binary-trees.js: Added.
191         * tests/sunspider-1.0.1/access-fannkuch.js: Added.
192         * tests/sunspider-1.0.1/access-nbody.js: Added.
193         * tests/sunspider-1.0.1/access-nsieve.js: Added.
194         * tests/sunspider-1.0.1/bitops-3bit-bits-in-byte.js: Added.
195         * tests/sunspider-1.0.1/bitops-bits-in-byte.js: Added.
196         * tests/sunspider-1.0.1/bitops-bitwise-and.js: Added.
197         * tests/sunspider-1.0.1/bitops-nsieve-bits.js: Added.
198         * tests/sunspider-1.0.1/controlflow-recursive.js: Added.
199         * tests/sunspider-1.0.1/crypto-aes.js: Added.
200         * tests/sunspider-1.0.1/crypto-md5.js: Added.
201         * tests/sunspider-1.0.1/crypto-sha1.js: Added.
202         * tests/sunspider-1.0.1/date-format-tofte.js: Added.
203         * tests/sunspider-1.0.1/date-format-xparb.js: Added.
204         * tests/sunspider-1.0.1/math-cordic.js: Added.
205         * tests/sunspider-1.0.1/math-partial-sums.js: Added.
206         * tests/sunspider-1.0.1/math-spectral-norm.js: Added.
207         * tests/sunspider-1.0.1/regexp-dna.js: Added.
208         * tests/sunspider-1.0.1/string-base64.js: Added.
209         * tests/sunspider-1.0.1/string-fasta.js: Added.
210         * tests/sunspider-1.0.1/string-tagcloud.js: Added.
211         * tests/sunspider-1.0.1/string-unpack-code.js: Added.
212         * tests/sunspider-1.0.1/string-validate-input.js: Added.
213
214 2013-05-03  Cosmin Truta  <ctruta@blackberry.com>
215
216         SunSpider 1.0: 3d-morph: use epsilon to check result
217         https://bugs.webkit.org/show_bug.cgi?id=115553
218
219         Reviewed by George Staikos.
220
221         SunSpider/3d-morph should be more lenient in validating its results, since they depend on Math.sin, which isn't formally specified.
222
223         * tests/sunspider-1.0/3d-morph.js:
224
225 2013-04-19  Filip Pizlo  <fpizlo@apple.com>
226
227         SunSpider/string-tagcloud should be more lenient in validating its results, since they depend on Math.log, which isn't formally specified
228         https://bugs.webkit.org/show_bug.cgi?id=114895
229
230         Reviewed by Michael Saboff.
231
232         * tests/sunspider-1.0/string-tagcloud.js:
233
234 2013-04-18  Filip Pizlo  <fpizlo@apple.com>
235
236         Whenever it is cheap and non-invasive, SunSpider tests should validate their results to ensure that the browser runs them correctly
237         https://bugs.webkit.org/show_bug.cgi?id=114852
238
239         Reviewed by Geoffrey Garen.
240         
241         This adds low-overhead checking of the results of each SunSpider tests. This is not
242         meant to be an exhaustive test that everything was executed correctly, but rather a
243         simple sanity check that will catch glaring mistakes. The philosophy here is that we're
244         not trying to prevent all forms of cheating, or that we're trying to prove the
245         browser's correctness. Moreover, these checks are meant to incur minimal overhead;
246         they currently clock in at <2% slow-down in SunSpider 1.0.
247         
248         The test expectations were generated using the LLInt on command-line, and I've verified
249         that Firefox 20, Chrome 26, IE 10, Safari 6, and ToT with all of the JITs agree. Note
250         that some tests cannot be precisely validated because they use Math functions, which
251         ECMAScript chooses to not formally specify - those functions like sin() and friends are
252         allowed to return implementation-dependent results. Also some tests cannot be validated
253         at all because their behavior is either intentionally random or is timezone-dependent.
254         But 23 out of 26 tests now have some kind of validation.
255         
256         I've updated the harnesses to show erroneous runs when displaying results.
257
258         * resources/TEMPLATE.html:
259         * resources/driver-TEMPLATE.html:
260         * resources/sunspider-analyze-results.js:
261         (formatResult):
262         (resultLine):
263         (printOutput):
264         * resources/sunspider-compare-results.js:
265         (.formatMean):
266         (.resultLine):
267         (.printOutput):
268         (sunspiderCompareResults):
269         * resources/sunspider-standalone-driver.js:
270         * tests/sunspider-1.0/3d-cube.js:
271         (Init):
272         * tests/sunspider-1.0/3d-morph.js:
273         * tests/sunspider-1.0/3d-raytrace.js:
274         * tests/sunspider-1.0/access-binary-trees.js:
275         * tests/sunspider-1.0/access-fannkuch.js:
276         * tests/sunspider-1.0/access-nbody.js:
277         * tests/sunspider-1.0/access-nsieve.js:
278         (sieve):
279         * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
280         * tests/sunspider-1.0/bitops-bits-in-byte.js:
281         * tests/sunspider-1.0/bitops-bitwise-and.js:
282         * tests/sunspider-1.0/bitops-nsieve-bits.js:
283         * tests/sunspider-1.0/controlflow-recursive.js:
284         * tests/sunspider-1.0/crypto-aes.js:
285         * tests/sunspider-1.0/crypto-md5.js:
286         * tests/sunspider-1.0/crypto-sha1.js:
287         * tests/sunspider-1.0/date-format-tofte.js:
288         * tests/sunspider-1.0/date-format-xparb.js:
289         * tests/sunspider-1.0/math-cordic.js:
290         * tests/sunspider-1.0/math-partial-sums.js:
291         (partial):
292         * tests/sunspider-1.0/math-spectral-norm.js:
293         * tests/sunspider-1.0/regexp-dna.js:
294         * tests/sunspider-1.0/string-base64.js:
295         * tests/sunspider-1.0/string-fasta.js:
296         (fastaRepeat):
297         (fastaRandom):
298         * tests/sunspider-1.0/string-tagcloud.js:
299         * tests/sunspider-1.0/string-unpack-code.js:
300         * tests/sunspider-1.0/string-validate-input.js:
301
302 2013-04-11  Filip Pizlo  <fpizlo@apple.com>
303
304         Get rid of the 10ms delay between test executions
305         https://bugs.webkit.org/show_bug.cgi?id=114458
306         <rdar://problem/13631112>
307
308         Reviewed by Darin Adler and Geoffrey Garen.
309         
310         More details on this in the bug. Long story short, I don't know why the 10ms delay is
311         there, and it appears to be the root of some noisyness and unreliability in the results.
312         Removing it makes results more consistent, which is a good thing.
313
314         * resources/driver-TEMPLATE.html:
315
316 2013-01-16  Eric Seidel  <eric@webkit.org>
317
318         Remove --shark* support from sunspider/run-sunspider now that Shark is dead
319         https://bugs.webkit.org/show_bug.cgi?id=99512
320
321         Reviewed by Darin Adler.
322
323         I tried to use --instruments, but that also seems broken (in the same way before/after this patch).
324
325         * sunspider:
326         (runTestsOnce):
327
328 2012-08-02  Filip Pizlo  <fpizlo@apple.com>
329
330         Unreviewed, revert debugging stuff I landed in http://trac.webkit.org/changeset/124557
331         by accident.
332
333         * tests/v8-v6/v8-crypto.js:
334         (am1):
335         (am2):
336         (am3):
337         (am4):
338         (bnpCopyTo):
339         (bnpFromString):
340         (bnpClamp):
341
342 2012-03-11  Maciej Stachowiak  <mjs@apple.com>
343
344         Add --instruments option to sunspider to profile with Instruments
345         https://bugs.webkit.org/show_bug.cgi?id=80783
346
347         Reviewed by Filip Pizlo.
348
349         * resources/TimeProfile20us.tracetemplate: Added.
350         * sunspider:
351         (runTestsOnce):
352
353 2011-11-15  Andy Wingo  <wingo@igalia.com>
354
355         Extend sunspider driver to be able to run kraken
356         https://bugs.webkit.org/show_bug.cgi?id=71799
357
358         Reviewed by Filip Pizlo.
359
360         * resources/sunspider-standalone-driver.js: Try to load a -data
361         file.  If that succeeds, we have a kraken-like test, so we time
362         the test using `load'.  Otherwise fall back to using `run'.
363
364 2011-08-11  Mark Rowe  <mrowe@apple.com>
365
366         Fix sunspider-compare-results to default to the same suite version as the sunspider script.
367
368         Rubber-stamped by Maciej Stachowiak.
369
370         * sunspider-compare-results:
371
372 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
373
374         Update SunSpider versioning to 1.0
375         https://bugs.webkit.org/show_bug.cgi?id=63870
376
377         Reviewed by Dan Bernstein.
378
379         * hosted/sunspider.html:
380         * hosted/versions.html:
381         * resources/driver-TEMPLATE.html:
382         * resources/results-TEMPLATE.html:
383         * sunspider:
384
385 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
386
387         Sunspider 0.9.1 harness doesn't actually close() its test documents
388         https://bugs.webkit.org/show_bug.cgi?id=47045
389
390         Reviewed by Daniel Bates.
391
392         * hosted/versions.html:
393         * resources/driver-TEMPLATE.html:
394
395 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
396
397         Not Reviewed. 
398         
399         Fix obviou typo in previous commit.
400
401         * tests/sunspider-1.0/bitops-bitwise-and.js:
402
403 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
404
405         Reviewed by Anders Carlsson.
406
407         SunSpider: all four bitops benchmarks can be replaced with NOP
408         https://bugs.webkit.org/show_bug.cgi?id=38446
409
410         * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
411         (TimeFunc): Save the result.
412         * tests/sunspider-1.0/bitops-bits-in-byte.js:
413         (TimeFunc): Save the result.
414         * tests/sunspider-1.0/bitops-bitwise-and.js: More explicitly
415         save the results.
416         * tests/sunspider-1.0/bitops-nsieve-bits.js:
417         (sieve): Save the result.
418
419 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
420
421         SunSpider: string-validate-input.js uses an undeclared variable named 'name', which is a DOM API
422         https://bugs.webkit.org/show_bug.cgi?id=60937
423
424         Reviewed by Dan Bernstein.
425
426         * tests/sunspider-1.0/string-validate-input.js:
427         (doTest): Rename the variable to username
428
429 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
430
431         Reviewed by Darin Adler.
432
433         SunSpider: The main function of math-cordic is dead code
434         https://bugs.webkit.org/show_bug.cgi?id=63863
435         
436         Try to prevent math-cordic from being eliminated as dead code or
437         compiled down to a constant.
438
439         * tests/sunspider-1.0/math-cordic.js:
440         (cordicsincos):
441         (cordic):
442
443 2011-06-30  Maciej Stachowiak  <mjs@apple.com>
444
445         Reviewed by Adam Barth.
446
447         Create sunspider-1.0 directory in preparation for fixing a bunch of bugs
448         https://bugs.webkit.org/show_bug.cgi?id=63782
449
450         * make-hosted: Learn about the new directory.
451         * tests/sunspider-1.0: Copied from PerformanceTests/SunSpider/tests/sunspider-0.9.1.
452
453 2010-11-29  Geoffrey Garen  <ggaren@apple.com>
454
455         Reviewed by Gavin Barraclough.
456
457         Improved accuracy of command-line SunSpider.
458
459         * resources/sunspider-standalone-driver.js: Use the returned number of
460         elapsed milliseconds from "run" and "checkSyntax" instead of measuring
461         ourselves, for slightly more accurate numbers.
462
463 2010-09-16  Gavin Barraclough  <barraclough@apple.com>
464
465         Reviewed by Mark Rowe.
466         https://bugs.webkit.org/show_bug.cgi?id=45924
467
468         Modify the SunSpider harness to allow a path to be passed to the --suite
469         argument, supress the normal prepend-"tests/" behaviour for suite names
470         containing a '/'.
471
472         Also wrap the code in an anonymous function to move variables out of global
473         scope (currently the harness cannot handle running tests the overwrite the
474         variable 'j').
475
476         * resources/sunspider-standalone-driver.js:
477         * sunspider:
478
479 2010-08-27  Michael Saboff  <msaboff@apple.com>
480
481         Reviewed by Stephanie Lewis.
482
483         Added version 5 and version 6 of V8 tests with modifications similar
484         to the version 4 tests so they'll work with the SunSpider test harness.
485         Note that these files are external source and therefore don't conform 
486         to WebKit coding standards.
487         https://bugs.webkit.org/show_bug.cgi?id=44638
488
489         * tests/v8-v5: Added.
490         * tests/v8-v5/LIST: Added.
491         * tests/v8-v5/v8-crypto.js: Added.
492         * tests/v8-v5/v8-deltablue.js: Added.
493         * tests/v8-v5/v8-earley-boyer.js: Added.
494         * tests/v8-v5/v8-raytrace.js: Added.
495         * tests/v8-v5/v8-regexp.js: Added.
496         * tests/v8-v5/v8-richards.js: Added.
497         * tests/v8-v5/v8-splay.js: Added.
498         * tests/v8-v6: Added.
499         * tests/v8-v6/LIST: Added.
500         * tests/v8-v6/v8-crypto.js: Added.
501         * tests/v8-v6/v8-deltablue.js: Added.
502         * tests/v8-v6/v8-earley-boyer.js: Added.
503         * tests/v8-v6/v8-raytrace.js: Added.
504         * tests/v8-v6/v8-regexp.js: Added.
505         * tests/v8-v6/v8-richards.js: Added.
506         * tests/v8-v6/v8-splay.js: Added.
507
508 2010-07-15  Stephanie Lewis  <slewis@apple.com>
509
510         Reviewed by Geoff Garen.
511
512         http://bugs.webkit.org/show_bug.cgi?id=42406
513         Output location of results file.
514
515         * sunspider:
516
517 2010-05-17  Maciej Stachowiak  <mjs@apple.com>
518
519         Reviewed by Mark Rowe.
520
521         Browser-hosted version of SunSpider runs tests twice as many times as advertised
522         https://bugs.webkit.org/show_bug.cgi?id=39201
523
524         * make-hosted: Make sure to reset test list after every test suite.
525
526 2010-05-07  Adam Roben  <aroben@apple.com>
527
528         Fix typo in sunspider-compare-results.js that caused incorrect
529         two-sample t statistics to be computed
530
531         Fixes <http://webkit.org/b/38768> SunSpider computes incorrect
532         two-sample t statistics
533
534         Reviewed by Darin Adler.
535
536         * resources/sunspider-compare-results.js:
537         (sunspiderCompareResults.resultLine): Fixed the calculation of the t
538         variable to match the definition of a two-sample t statistic. Looks
539         like this was just a simple typo.
540
541 2010-04-05  Darin Adler  <darin@apple.com>
542
543         Updated the ignore property for changes to the contents here.
544
545         * .: Modified property svn:ignore.
546
547 2010-02-20  Maciej Stachowiak  <mjs@apple.com>
548
549         Reviewed by Adam Barth.
550
551         Add sunspider-0.9.1 to the site in preparation for announcing it.
552         https://bugs.webkit.org/show_bug.cgi?id=35206
553
554         * hosted/sunspider.css: Remove tabs, which I noticed where present while
555         copying to the WebKitSite directory.
556
557 2010-01-11  Oliver Hunt  <oliver@apple.com>
558
559         Reviewed by Darin Adler.
560
561         Sunspider parse-only tests don't run
562         https://bugs.webkit.org/show_bug.cgi?id=33489
563
564         Fix parse-only test list
565
566         * tests/parse-only/LIST:
567
568 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
569
570         Reviewed by Adele Peterson.
571
572         Make SunSpider version more prominent in the title
573         https://bugs.webkit.org/show_bug.cgi?id=32574
574
575         * hosted/sunspider.html:
576         * hosted/versions.html:
577         * resources/driver-TEMPLATE.html:
578         * resources/results-TEMPLATE.html:
579
580 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
581
582         Reviewed by Darin Adler.
583
584         Make SunSpider refuse to compare results across content versions
585         https://bugs.webkit.org/show_bug.cgi?id=32573
586         
587         The results URL now includes a version indicator, and compare mode
588         will refuse to compare between versions.
589
590         * resources/driver-TEMPLATE.html:
591         * resources/results-TEMPLATE.html:
592
593 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
594
595         Reviewed by Darin Adler.
596
597         Some Browser-hosted SunSpider files are not valid HTML5
598         https://bugs.webkit.org/show_bug.cgi?id=32536
599         
600         Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
601         close tags and adding a meta charset declaration).
602
603         * hosted/sunspider.html:
604         * hosted/versions.html:
605         * resources/TEMPLATE.html:
606         * resources/driver-TEMPLATE.html:
607         * resources/results-TEMPLATE.html:
608
609 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
610
611         Reviewed by Eric Seidel.
612
613         Make sunspider-0.9.1 the default content set (both command-line and hosted)
614         https://bugs.webkit.org/show_bug.cgi?id=32537
615
616         * hosted/sunspider.html: Change default suite link.
617         * sunspider: Change default suite.
618         * sunspider-compare-results: Likewise.
619
620 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
621
622         Reviewed by Sam Weinig.
623
624         in-browser SunSpider suffers excessive penalty under power management
625         https://bugs.webkit.org/show_bug.cgi?id=32505
626         
627         I have made a few changes to address this:
628         
629         1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
630         document.write to reduce triggering of progress UI.
631         2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
632         stepped all the way down the whole time.
633         3) Run the test cycle an extra time for warmup before the runs that count.
634
635         * hosted/sunspider-record-result.js: Removed.
636         * make-hosted:
637         * resources/TEMPLATE.html:
638         * resources/driver-TEMPLATE.html:
639
640 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
641
642         Reviewed by Gavin Barraclough.
643
644         SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
645         https://bugs.webkit.org/show_bug.cgi?id=16806
646
647         Based on a patch by Eric Seidel.
648         
649         Fix the base64 computation to actually compute correct results. The impact on runtime of
650         the test is pretty small, but noticeable for some browsers. But at least it's not
651         doing a wrong and meaningless computation any more.
652         
653         * tests/sunspider-0.9.1/string-base64.js:
654         ():
655         (base64ToString):
656
657 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
658
659         Fixing commit error...
660         
661         I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
662
663         * resources/driver-TEMPLATE.html:
664
665 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
666
667         Reviewed by Sam Weinig.
668
669         Use JSON.parse instead of eval for Web-hosted SunSpider results processing
670         https://bugs.webkit.org/show_bug.cgi?id=32490
671
672         * hosted/json2.js: Added. Incorporated from json.org
673         * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
674         * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
675         results.
676
677 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
678
679         Reviewed by Oliver Hunt.
680
681         Enable Web-hosted version of SunSpider to handle multiple versions
682         https://bugs.webkit.org/show_bug.cgi?id=32478
683
684         * make-hosted: Now generate the hosted version to be able to run both the
685         0.9 and the 0.9.1 test suites.
686         * hosted: Modified property svn:ignore.
687         * hosted/sunspider-driver.html: Removed.
688         * hosted/sunspider-results.html: Removed.
689         * hosted/sunspider.html:
690         * hosted/versions.html: Added.
691         * resources/TEMPLATE.html:
692         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
693         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
694
695 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
696
697         Reviewed by Oliver Hunt.
698
699         Give command-line sunspider the ability to handle multiple suites and versions
700         https://bugs.webkit.org/show_bug.cgi?id=32477
701         
702         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
703         enumerating the tests. Current suites are:
704         
705         sunspider-0.9 - riginal SunSpider content 
706         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
707         ubench - "ubench" microbenchmark suite
708         v8-v4 - version 4 of the v8 benchmark
709         parse-only - some parser tests
710         
711         Sunspider can now be run with the --suite= parameter to select a
712         suite. The default is --suite=sunspider-0.9.
713
714         * .: Modified property svn:ignore to ignore new results directories.
715         * make-hosted: Generate based on sunspider-0.9 suite.
716         * resources/TEMPLATE.html: Fix an HTML compliance problem.
717         * resources/sunspider-standalone-driver.js: Become aware of suite names.
718         * sunspider: Modified to handle multiple suites as described above.
719         * sunspider-compare-results: ditto
720         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
721         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
722         * tests/LIST-UBENCH: Removed.
723         * tests/LIST-V8: Removed.
724         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
725         * tests/3d-morph.js: ditto
726         * tests/3d-raytrace.js: ditto
727         * tests/access-binary-trees.js: ditto
728         * tests/access-fannkuch.js: ditto
729         * tests/access-nbody.js: ditto
730         * tests/access-nsieve.js: ditto
731         * tests/bitops-3bit-bits-in-byte.js: ditto
732         * tests/bitops-bits-in-byte.js: ditto
733         * tests/bitops-bitwise-and.js: ditto
734         * tests/bitops-nsieve-bits.js: ditto
735         * tests/controlflow-recursive.js: ditto
736         * tests/crypto-aes.js: ditto
737         * tests/crypto-md5.js: ditto
738         * tests/crypto-sha1.js: ditto
739         * tests/date-format-tofte.js: ditto
740         * tests/date-format-xparb.js: ditto
741         * tests/math-cordic.js: ditto
742         * tests/math-partial-sums.js: ditto
743         * tests/math-spectral-norm.js: ditto
744         * tests/regexp-dna.js: ditto
745         * tests/string-base64.js: ditto
746         * tests/string-fasta.js: ditto
747         * tests/string-tagcloud.js: ditto
748         * tests/string-unpack-code.js: ditto
749         * tests/string-validate-input.js: ditto
750         * tests/sunspider-0.9/: Added
751         * tests/sunspider-0.9/*.js: Moved from one level up
752         * tests/sunspider-0.9.1: Added.
753         * tests/sunspider-0.9.1/*.js: Moved from one level up.
754         * tests/v8-crypto.js: Moved to v8-v4/ directory.
755         * tests/v8-deltablue.js: ditto
756         * tests/v8-earley-boyer.js: ditto
757         * tests/v8-raytrace.js: ditto
758         * tests/v8-regexp.js: ditto
759         * tests/v8-richards.js: ditto
760         * tests/v8-splay.js: ditto
761         * tests/v8-v4: Added.
762
763 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
764
765         Reviewed by Darin Adler.
766
767         Platform specific null device replaced with a platform independent.
768         https://bugs.webkit.org/show_bug.cgi?id=29544
769
770         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
771
772 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
773
774         Reviewed by Maciej Stachowiak.
775
776         [Qt] Make sunspider script work on Windows platform.
777         https://bugs.webkit.org/show_bug.cgi?id=29544
778
779         * sunspider:
780         Using platform specific null device instead of /dev/null.
781
782 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
783
784         Reviewed by Oliver Hunt.
785
786         https://bugs.webkit.org/show_bug.cgi?id=26540
787         Currently the SunSpider test driver lacks an option to run a test suite that
788         will test JavaScriptCore parsing performance only.  This patch adds just such
789         a test suite and option to SunSpider as well as the jsc test shell.  I've included
790         three large javascript source files found in the wild: jquery, mootools and prototype.
791         Combined with the concatenation of all three, these form a new testsuite to measure
792         and test pure JavaScriptCore parsing performance.
793
794         * resources/sunspider-standalone-driver.js:
795           Load the js with parse-only flag if it is part of the parse-only suite.
796         * sunspider:
797           Add the parse-only suite to the list of options.
798         * sunspider-compare-results:
799           Add the parse-only suite to the list of options.
800         * tests/LIST-PARSE-ONLY: Added.
801         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
802           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
803         * tests/parse-only/jquery-1.3.2.js: Added.
804           Latest version from jquery.com.
805         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
806           Latest version from mootools.net.
807         * tests/parse-only/prototype-1.6.0.3.js: Added.
808           Latest version from prototypejs.org.
809
810 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
811
812         Update V8 benchmark to version 4.
813         
814         Reviewed by Cameron Zwarich.
815
816         * tests/LIST-V8:
817         * tests/v8-crypto.js:
818         * tests/v8-deltablue.js:
819         * tests/v8-earley-boyer.js:
820         * tests/v8-raytrace.js:
821         * tests/v8-regexp.js: Added.
822         * tests/v8-richards.js:
823         * tests/v8-splay.js: Added.
824
825 2009-05-26  Olivier DOLE  <odole@pleyo.com>
826
827         Reviewed by Maciej.
828
829         Store sunspider test suite results in a specific directory for each
830         different type of benchmark (sunspider, V8 or ubench).
831
832         * sunspider:
833         * sunspider-compare-results:
834
835 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
836
837         Build fix.
838
839         * SunSpider.make:
840
841 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
842
843         Include SunSpider tests in production builds.
844         
845         Reviewed by Adam Roben.
846
847         * SunSpider.make: Added.
848         * xcopy.excludes: Added.
849
850 2009-03-04  Oliver Hunt  <oliver@apple.com>
851
852         Added a few more tests to ubench
853
854         RS=Gavin Barraclough
855
856         Adding a couple of test to cover too few and too many arguments.  This
857         is needed to prevent regressions in all call paths from my current refactoring.
858
859         * tests/LIST-UBENCH:
860         * tests/ubench/function-correct-args.js: Added.
861         (f):
862         * tests/ubench/function-excess-args.js: Added.
863         (f):
864
865 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
866
867         Fix a bug where when V8's or UBENCH's test suite list file is missing
868         the error message is suggesting that SunSpider's list file is missing.
869
870         Reviewed by Sam Weinig.
871
872         * sunspider:
873         (loadTestsList): Fix the error message to use the right list name.
874
875 2008-10-07  David Hyatt  <hyatt@apple.com>
876
877         https://bugs.webkit.org/show_bug.cgi?id=18989
878
879         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
880         we want all browsers to be doing the same thing on the test.
881
882         Reviewed by Maciej
883
884         * tests/regexp-dna.js:
885
886 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
887
888         Rubber-stamped by Maciej Stachowiak.
889
890         The bleeding edge branch of V8 now supports passing fileanmes after
891         '-f', so the SunSpider script should be changed to do this again.
892         This allows all 3 major JavaScript engines to be used with the
893         unmodified SunSpider script.
894
895         * sunspider:
896
897 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
898
899         Reviewed by Maciej Stachowiak.
900
901         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
902         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
903         correct the punctuation in its documentation.
904
905         * sunspider:
906
907 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
908
909         Reviewed by Oliver Hunt.
910
911         Bug 20616: Incorporate V8 benchmarks in testing
912         <https://bugs.webkit.org/show_bug.cgi?id=20616>
913
914         This is a first attempt at running the V8 benchmarks within SunSpider,
915         with the --v8 option. The number of iterations is fixed for each
916         benchmark, and was chosen to make each of the benchmarks run between
917         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
918
919         * sunspider:
920         * tests/LIST-V8: Added.
921         * tests/v8-crypto.js: Added.
922         * tests/v8-deltablue.js: Added.
923         * tests/v8-earley-boyer.js: Added.
924         * tests/v8-raytrace.js: Added.
925         * tests/v8-richards.js: Added.
926
927 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
928
929         Not reviewed.
930
931         Remove a local change that was accidentally committed in r36034.
932
933         * resources/sunspider-standalone-driver.js:
934
935 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
936
937         Rubber-stamped by Oliver Hunt.
938
939         Make SunSpider work with JavaScript shells that don't take filenames
940         after a '-f' argument.
941
942         * sunspider:
943
944 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
945
946         Reviewed by Timothy.
947
948         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
949
950         * sunspider:
951         * tests/LIST-SQUIRRELFISH: Removed.
952
953 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
954
955         Rubber-stamped by Mark Rowe.
956
957         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
958
959         * sunspider:
960
961 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
962
963         Rubber stamped by Oliver.
964         
965         - add newly working tests to squirrelfish list
966
967         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
968
969 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
970
971         Rubber stamped by Oliver.
972         
973         - move ubench tests properly.
974
975         * tests/ubench: Added.
976         * tests/ubench/function-closure.js: Added.
977         * tests/ubench/function-empty.js: Added.
978         * tests/ubench/function-missing-args.js: Added.
979         * tests/ubench/function-sum.js: Added.
980         * tests/ubench/loop-empty-resolve.js: Added.
981         * tests/ubench/loop-empty.js: Added.
982         * tests/ubench/loop-sum.js: Added.
983
984 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
985
986         Rubber stamped by Oliver.
987
988         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
989
990 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
991
992         Reviewed by Oliver.
993
994         - make --squirrelfish mode use the real harness, but a reduced set of tests
995
996         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
997
998         * sunspider:
999         * tests/LIST-SQUIRRELFISH:
1000         * tests/LIST-UBENCH: Added.
1001         * tests/squirrelfish: Moved to tests/ubench
1002
1003 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
1004
1005         Reviewed by Maciej Stachowiak.
1006         
1007         Removed an unused test.
1008
1009         * tests/squirrelfish/loop-resolve.js: Removed.
1010
1011 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
1012
1013         Reviewed by Oliver.
1014         
1015         Add newly runnable tests to --squirrelfish mode.
1016
1017         * tests/LIST-SQUIRRELFISH:
1018
1019 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
1020
1021         Reviewed by Maciej Stachowiak.
1022         
1023         A little more love for --squirrelfish mode:
1024         
1025         Fixed a misplaced sort. Results properly sort now.
1026         
1027         Added a test list just for squirrelfish, and updated it to include
1028         all currently passing SunSpider tests.
1029         
1030         Fixed the pruning regexp to match 3d-morph.js.
1031
1032         * sunspider:
1033
1034 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
1035
1036         Reviewed by Oliver Hunt.
1037         
1038         --squirrelfish mode: pared down tests for squirrelfish to chew on.
1039
1040         * sunspider:
1041         * tests/LIST:
1042         * tests/squirrelfish: Added.
1043         * tests/squirrelfish/function-closure.js: Added.
1044         * tests/squirrelfish/function-empty.js: Added.
1045         * tests/squirrelfish/function-missing-args.js: Added.
1046         * tests/squirrelfish/function-sum.js: Added.
1047         * tests/squirrelfish/loop-empty-resolve.js: Added.
1048         * tests/squirrelfish/loop-empty.js: Added.
1049         * tests/squirrelfish/loop-resolve.js: Added.
1050         * tests/squirrelfish/loop-sum.js: Added.
1051
1052 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
1053
1054         Reviewed by Geoff.
1055
1056         * sunspider: Pause update daemon on Darwin to make results reliable
1057         again (workaround for <rdar://problem/5811127>).
1058
1059 2008-02-07  Eric Seidel  <eric@webkit.org>
1060
1061         Reviewed by Sam.
1062
1063         * sunspider: add an error message when ./tests/LIST is not found
1064
1065 2007-12-29  Alp Toker  <alp@atoker.com>
1066
1067         Fix typo.
1068
1069         * hosted/sunspider.html:
1070
1071 2007-12-29  Alp Toker  <alp@atoker.com>
1072
1073         Reviewed by Maciej.
1074
1075         Provide the current URL in the SunSpider results page. Useful for
1076         copying benchmark results out of browsers without a location bar or
1077         where the location bar can't handle long URLs.
1078
1079         * hosted/sunspider-results.html:
1080
1081 2007-12-20  Eric Seidel  <eric@webkit.org>
1082
1083         Reviewed by mjs.
1084
1085         * sunspider: increase default number of runs for more accurate numbers
1086
1087 2007-12-18  Mark Rowe  <mrowe@apple.com>
1088
1089         Fix typo.
1090
1091         * hosted/sunspider.html:
1092
1093 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
1094
1095         Reviewed by Eric.
1096         
1097         - give an extra digit of precision for small differences.
1098
1099         * resources/sunspider-compare-results.js:
1100
1101 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
1102
1103         Reviewed by Eric.
1104         
1105         - work around IE layout bugs
1106
1107         * hosted/sunspider.css:
1108
1109 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
1110
1111         Reviewed by Mark.
1112         
1113         - fix compare feature
1114
1115         * hosted/sunspider-results.html:
1116
1117 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
1118
1119         Reviewed by Mark.
1120         
1121         - Change the format of the output URL to be shorter, to meet IE constraints
1122
1123         * hosted/sunspider-driver.html:
1124         * hosted/sunspider-results.html:
1125
1126 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
1127
1128         Reviewed by Oliver.
1129         
1130         - Add some explanatory test and a dumb logo to SunSpider web hosted version
1131
1132         * hosted/sunspider-driver.html:
1133         * hosted/sunspider-results.html:
1134         * hosted/sunspider.css:
1135         * hosted/sunspider.html:
1136
1137 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
1138
1139         Reviewed by Oliver.
1140         
1141         - Make the web-hosted version of SunSpider look a little bit less drab.
1142
1143         * hosted/sunspider-driver.html:
1144         * hosted/sunspider-results.html:
1145         * hosted/sunspider.css: Added.
1146         * hosted/sunspider.html:
1147         * resources/TEMPLATE.html:
1148
1149 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
1150
1151         Reviewed by Sam.
1152         
1153         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
1154         
1155         A sample of what new-style results would look like:
1156         
1157         TEST                   COMPARISON            FROM                 TO             DETAILS
1158         ...
1159         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
1160         ...
1161             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
1162         
1163         * resources/sunspider-compare-results.js:
1164
1165 2007-11-14  Eric Seidel  <eric@webkit.org>
1166
1167         Reviewed by Sam.
1168
1169         * sunspider: add --shark-cache for L2 Cache Miss profiling
1170
1171 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
1172
1173         Reviewed by Darin Adler.
1174
1175         http://bugs.webkit.org/show_bug.cgi?id=15925
1176         SunSpider should check for Shark being installed
1177
1178         * sunspider: Explicitly test that shark command line tool is installed.
1179
1180 2007-11-04  Darin Adler  <darin@apple.com>
1181
1182         * hosted/sunspider.html: Tweaked formatting a bit.
1183
1184 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
1185
1186         Reviewed by Darin Adler.
1187
1188         - Add compare mode and a description of the benchmark to the hosted version.
1189
1190         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
1191         * hosted/sunspider.html: Added intro text.
1192         * make-hosted: Copy new file.
1193         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
1194         a function.
1195         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
1196         expects.
1197         * sunspider-compare-results: Use sunspider-standalone-compare.js.
1198
1199 2007-11-02  Darin Adler  <darin@apple.com>
1200
1201         * sunspider: Changed "--base" to "--set-baseline".
1202
1203 2007-11-02  Darin Adler  <darin@apple.com>
1204
1205         Reviewed by Maciej.
1206
1207         * sunspider: Added a "--base" option that records the current test as a baseline.
1208         Moved the code to find the newest .mshark file into a function (since I needed to use
1209         it in the other script and I prefer to copy/paste a function).
1210
1211         * sunspider-compare-results: Added logic to compare the baseline with the last
1212         run when you don't pass any parameters.
1213
1214 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
1215
1216         Reviewed by Eric.
1217         
1218         - Added list of ECMAScript3 features that SunSpider does not cover currently.
1219
1220         * UNCOVERED: Added.
1221
1222 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
1223
1224         Reviewed by Oliver.
1225         
1226         - Give better error messages when sunspider-compare-results fails
1227
1228         * sunspider-compare-results:
1229
1230 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
1231
1232         Reviewed by Darin Adler.
1233         
1234         - fix Date tests to work in command-line SpiderMonkey
1235
1236         * tests/date-format-tofte.js:
1237         * tests/date-format-xparb.js:
1238
1239 2007-10-29  Darin Adler  <darin@apple.com>
1240
1241         Reviewed by Maciej.
1242
1243         * sunspider: Added --shark20 option, to run Shark at its highest sample
1244         resolution instead of its default.
1245
1246 2007-10-28  Adam Roben  <aroben@apple.com>
1247
1248         Fix SunSpider on Windows
1249
1250         Reviewed by Eric.
1251
1252         * sunspider: Don't use colons in filenames.
1253
1254 2007-10-28  Darin Adler  <darin@apple.com>
1255
1256         Reviewed by Adam.
1257
1258         * resources/sunspider-analyze-results.js: Tweak the output format a little.
1259         Change so that when there's only one run we don't write out confidence
1260         intervals at all rather than writing "NaN" over and over again.
1261
1262         * sunspider: Use the time and date as a suffix on the results file, that way
1263         all the results are left behind in the tmp directory. This will make it easy
1264         for us to add features that compare past results in the future. Also rename
1265         the .mshark files using the same scheme.
1266
1267         * sunspider-compare-results: Relax the parsing rules so we can parse the
1268         results file exactly as it's emitted from sunspider.
1269
1270 2007-10-25  Adam Roben  <aroben@apple.com>
1271
1272         Strip carriage returns from results
1273
1274         Carriage returns were screwing up sunspider-compare-results on
1275         Windows.
1276
1277         Rubberstamped by Oliver.
1278
1279         * sunspider:
1280
1281 2007-10-24  Darin Adler  <darin@apple.com>
1282
1283         Reviewed by Eric.
1284
1285         * sunspider: Open the profile in shark after completing a run.
1286         * sunspider-compare-results: Tiny perl tweaks.
1287
1288 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
1289
1290         Reviewed by Darin Adler.
1291
1292         - Don't hardcode my path to testkjs
1293         
1294         * sunspider-compare-results:
1295
1296 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
1297
1298         Reviewed by Darin Adler.
1299         
1300         - Add a compare script to compare results, and improve formatting of standard results a bit.
1301
1302         * resources/sunspider-analyze-results.js:
1303         * resources/sunspider-compare-results.js: Added.
1304         * sunspider-compare-results: Added.
1305
1306 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
1307
1308         Reviewed by Darin Adler.
1309
1310         * resources/sunspider-analyze-results.js: Do the error range
1311         properly, using the t-distribution instead of 1.96 (which was
1312         based on the normal distribution).
1313         
1314         * sunspider: Print results in a way that will be friendlier to the
1315         soon-to-come compare mode.
1316
1317 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
1318
1319         Reviewed by Review.
1320         
1321         - add more tests (probably the final set for now)
1322         
1323         Already balanced. Added date, regexp, control flow, and a few
1324         more string and object/array access tests.
1325         
1326         * tests/LIST:
1327         * tests/access-fannkuch.js: Added.
1328         * tests/access-nbody.js: Added.
1329         * tests/controlflow-recursive.js: Added.
1330         * tests/date-format-tofte.js: Added.
1331         * tests/date-format-xparb.js: Added.
1332         * tests/regexp-dna.js: Added.
1333         * tests/string-validate-input.js: Added.
1334
1335 2007-10-21  Eric Seidel  <eric@webkit.org>
1336
1337         Reviewed by mjs.
1338
1339         * sunspider: make --shark only sample from testkjs
1340
1341 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
1342
1343         Rubber stamped by Eric.
1344         
1345         - added crypto tests (already balanced)
1346
1347         * tests/LIST:
1348         * tests/crypto-aes.js: Added.
1349         * tests/crypto-md5.js: Added.
1350         * tests/crypto-sha1.js: Added.
1351
1352 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
1353
1354         Reviewed by Adam.
1355         
1356         - rebalance new tests
1357         
1358         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
1359         
1360         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1361
1362         This balances them with the existing tests.
1363
1364         * tests/access-binary-trees.js:
1365         * tests/access-nsieve.js:
1366         * tests/bitops-nsieve-bits.js:
1367         * tests/math-partial-sums.js:
1368         * tests/math-spectral-norm.js:
1369         * tests/string-fasta.js:
1370
1371 2007-10-20  Eric Seidel  <eric@webkit.org>
1372
1373         Reviewed by darin.
1374         
1375         Add improved argument handling to sunspider, including
1376         --runs=<number>, --tests=<pattern, --shark, and --help
1377         Also re-factor code into subroutines
1378
1379         * sunspider: improved argument handling
1380
1381 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1382
1383         Reviewed by Mark.
1384
1385         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
1386         
1387         * TODO:
1388         * tests/LIST:
1389         * tests/access-binary-trees.js: Added.
1390         * tests/access-nsieve.js: Added.
1391         * tests/bitops-nsieve-bits.js: Added.
1392         * tests/math-partial-sums.js: Added.
1393         * tests/math-spectral-norm.js: Added.
1394         * tests/string-fasta.js: Added.
1395
1396 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1397
1398         Reviewed by Darin Adler.
1399
1400         * TODO: Updated to note areas that are now well-covered.
1401         
1402         - Change the tests and the driver to avoid leaving large object
1403         graphs hanging around, since that throws off the subsequent tests.
1404         
1405         * resources/sunspider-standalone-driver.js:
1406         * tests/3d-cube.js:
1407         * tests/3d-morph.js:
1408         * tests/3d-raytrace.js:
1409         * tests/bitops-bitwise-and.js:
1410         * tests/string-base64.js:
1411         * tests/string-tagcloud.js:
1412
1413 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1414
1415         Reviewed by Nikolas.
1416         
1417         - Rebalance test complexity.
1418         
1419         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
1420         
1421         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1422
1423         However, for some tests, IE7 was a huge outlier compared to other
1424         major browsers. For those tests, I used Firefox 2.0.0.8 on the
1425         same OS and hardware as a normalization basis.
1426         
1427         * tests/3d-morph.js:
1428         * tests/3d-raytrace.js:
1429         * tests/bitops-3bit-bits-in-byte.js:
1430         * tests/bitops-bits-in-byte.js:
1431         * tests/bitops-bitwise-and.js:
1432         * tests/math-cordic.js:
1433         * tests/string-base64.js:
1434         * tests/string-tagcloud.js:
1435         * tests/string-unpack-code.js:
1436
1437 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1438
1439         Reviewed by Oliver.
1440         
1441         - Fix the browser-hosted driver to be compatible with Safari 2
1442
1443         * hosted/sunspider-driver.html:
1444
1445 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1446
1447         Rubber stamped by Hyatt.
1448         
1449         - Add BSD license notice to cordic test, the original author said to do so.
1450
1451         * tests/math-cordic.js:
1452
1453 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1454
1455         Reviewed by Eric.
1456         
1457         - Support Windows and Windows browsers.
1458
1459         * hosted/sunspider-results.html: Display results correctly in IE.
1460         * sunspider: tweaks to work on Windows, and with spaces in the path.
1461
1462 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1463
1464         Rubber stamped by Oliver.
1465
1466         * tests/string-tagcloud.js: Remove stray print()
1467
1468 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
1469
1470         Reviewed by Oliver.
1471         
1472         Add some string test cases.
1473
1474         * tests/string-base64.js: Added. Does base64 encoding and decoding.
1475         * tests/string-tagcloud.js: Added. Parses a JSON string and
1476         generates tagcloud HTML markup based on the data.        
1477         * tests/string-unpack-code.js: Added. Unpack compressed versions of
1478         four of the most popular JavaScript libraries.
1479         * tests/LIST: Added the new tests.
1480
1481 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
1482
1483         Reviewed by Anders.
1484
1485         * TODO: Updated with info on how to do the stats properly.
1486
1487         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
1488         avoid overly slowing down the benchmark.
1489
1490         Added some 3D-related test cases.
1491         
1492         * tests/3d-cube.js: Added.
1493         * tests/3d-morph.js: Added.
1494         * tests/3d-raytrace.js: Added.
1495         * tests/LIST:
1496
1497 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
1498
1499         Reviewed by Darin Adler.
1500
1501         - New JavaScript benchmark
1502         http://bugs.webkit.org/show_bug.cgi?id=15515
1503         
1504         A start on a new JavaScript benchmark, with standalone and
1505         browser-hosted drivers.
1506         
1507         The test content is still incomplete and the drivers could use
1508         some more features but there is enough here to make a good start.
1509         
1510         The drivers compute a 95% confidence interval on the mean for the
1511         whole test, each category, and each individual test to make it
1512         easier to tell whether differences are statistically
1513         significant. The confidence interval can be narrowed by running
1514         the test more times.
1515         
1516         * TODO: Added. Remaining things that need to be done.
1517         * sunspider: Added. Perl script that acts as the standalone test driver.
1518         * resources/sunspider-standalone-driver.js: Added. JavaScript part
1519         of standalone test driver.
1520         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
1521         analysis code.
1522         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
1523         * make-hosted: Added. Script to generate browser-hosted tests.
1524         * hosted/sunspider.html: Added. Start page for browser-hosted test.
1525         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
1526         * hosted/sunspider-results.html: Added. Results page for browser-hosted
1527         tests.
1528         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
1529         tests.
1530         * tests/LIST: Added. List of tests to use.
1531         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
1532         * tests/bitops-bits-in-byte.js: Added. Ditto.
1533         * tests/bitops-bitwise-and.js: Added. Ditto.
1534         * tests/math-cordic.js: Added. Ditto.
1535