ba79e11453830b27645dbc4e94bea222b62c89f7
[WebKit-https.git] / PerformanceTests / SunSpider / ChangeLog
1 2015-01-27  Csaba Osztrogon√°c  <ossy@webkit.org>
2
3         [Win] JSC profiler tests asserts in debug mode
4         https://bugs.webkit.org/show_bug.cgi?id=140931
5
6         Unreviewed gardening, skip failing tests.
7
8         * profiler-test.yaml:
9
10 2014-11-04  Michael Saboff  <msaboff@apple.com>
11
12         Disable failing v8-deltablue.js for ARM
13         https://bugs.webkit.org/show_bug.cgi?id=138382
14
15         Reviewed by Geoffrey Garen.
16
17         This test is failing on ARM 32 bit.
18
19         * tests/v8-v6/v8-deltablue.js:
20
21 2014-07-08  Gergo Balogh  <gbalogh.u-szeged@partner.samsung.com>
22
23         Add option to specify output file location in sunspider.
24         https://bugs.webkit.org/show_bug.cgi?id=134339
25
26         Reviewed by Darin Adler.
27
28         * sunspider:
29
30 2014-06-03  Eva Balazsfalvi  <evab.u-szeged@partner.samsung.com>
31
32         REGRESSION(r169092 and r169102): Skip failing JSC tests poperly on non-x86 Darwin platforms
33         https://bugs.webkit.org/show_bug.cgi?id=133149
34
35         Reviewed by Csaba Osztrogon√°c.
36
37         * profiler-test.yaml: Skip profiler tests only if the architecture isn't x86 and the host is Darwin.
38
39 2014-05-21  Filip Pizlo  <fpizlo@apple.com>
40
41         Unreviewed, roll out http://trac.webkit.org/changeset/169159.
42         
43         This was a unilateral change and wasn't properly reviewed.
44
45         * profiler-test.yaml:
46
47 2014-05-19  Filip Pizlo  <fpizlo@apple.com>
48
49         Take care of some ARM64 test failures
50         https://bugs.webkit.org/show_bug.cgi?id=133090
51
52         Reviewed by Geoffrey Garen.
53
54         * profiler-test.yaml: Skip these on not-x86.
55
56 2014-03-14  Maciej Stachowiak  <mjs@apple.com>
57
58         Replace "Apple Computer, Inc." with "Apple Inc." in copyright headers
59         https://bugs.webkit.org/show_bug.cgi?id=130276
60         <rdar://problem/16266927>
61
62         Reviewed by Simon Fraser.
63
64         * hosted/sunspider.html:
65         * hosted/versions.html:
66         * make-hosted:
67         * resources/TEMPLATE.html:
68         * resources/driver-TEMPLATE.html:
69         * resources/results-TEMPLATE.html:
70         * resources/sunspider-analyze-results.js:
71         * resources/sunspider-compare-results.js:
72         * resources/sunspider-standalone-compare.js:
73         * resources/sunspider-standalone-driver.js:
74         * sunspider:
75         * sunspider-compare-results:
76         * tests/sunspider-0.9.1/3d-morph.js:
77         * tests/sunspider-0.9.1/3d-raytrace.js:
78         * tests/sunspider-0.9.1/bitops-bitwise-and.js:
79         * tests/sunspider-0.9.1/math-cordic.js:
80         * tests/sunspider-0.9.1/string-tagcloud.js:
81         * tests/sunspider-0.9/3d-morph.js:
82         * tests/sunspider-0.9/3d-raytrace.js:
83         * tests/sunspider-0.9/bitops-bitwise-and.js:
84         * tests/sunspider-0.9/math-cordic.js:
85         * tests/sunspider-0.9/string-tagcloud.js:
86         * tests/sunspider-1.0.1/3d-morph.js:
87         * tests/sunspider-1.0.1/3d-raytrace.js:
88         * tests/sunspider-1.0.1/bitops-bitwise-and.js:
89         * tests/sunspider-1.0.1/math-cordic.js:
90         * tests/sunspider-1.0.1/string-tagcloud.js:
91         * tests/sunspider-1.0.2/3d-morph.js:
92         * tests/sunspider-1.0.2/3d-raytrace.js:
93         * tests/sunspider-1.0.2/bitops-bitwise-and.js:
94         * tests/sunspider-1.0.2/math-cordic.js:
95         * tests/sunspider-1.0.2/string-tagcloud.js:
96         * tests/sunspider-1.0/3d-morph.js:
97         * tests/sunspider-1.0/3d-raytrace.js:
98         * tests/sunspider-1.0/bitops-bitwise-and.js:
99         * tests/sunspider-1.0/math-cordic.js:
100         * tests/sunspider-1.0/string-tagcloud.js:
101
102 2014-02-10  Filip Pizlo  <fpizlo@apple.com>
103
104         Unreviewed, revert the accidental change from r163789.
105
106         * tests/v8-v6/v8-deltablue.js:
107
108 2013-12-10  Filip Pizlo  <fpizlo@apple.com>
109
110         Get rid of forward exit on DoubleAsInt32
111         https://bugs.webkit.org/show_bug.cgi?id=125552
112
113         Reviewed by Oliver Hunt.
114         
115         Use SunSpider as a kind of spot-check for the
116         no-architecture-specific-optimization paths in the compiler.
117
118         * no-architecture-specific-optimizations.yaml: Added.
119
120 2013-10-08  Geoffrey Garen  <ggaren@apple.com>
121
122         Refined power management in SunSpider 1.0.2
123         https://bugs.webkit.org/show_bug.cgi?id=122525
124
125         Reviewed by Gavin Barraclough.
126
127         * hosted/sunspider.css:
128         (#frameparent):
129         (#countdown):
130         * resources/TEMPLATE.html: Provide an explicit countdown element in the
131         parent, instead of hacking it into each child frame.
132
133         * resources/driver-TEMPLATE.html: Be more explicit about how we respond
134         to power management. Just moving setTimeout(0) inside the loop is a
135         little too subtle, and may not work across browsers.
136
137         Also, leave a little extra time after changing the countdown graphic,
138         to give the browser some time to paint.
139
140 2013-10-02  Geoffrey Garen  <ggaren@apple.com>
141
142         Improved painting behavior in SunSpider 1.0.2
143         https://bugs.webkit.org/show_bug.cgi?id=122257
144
145         Reviewed by Oliver Hunt.
146
147         Individual test names with times was a good painting strategy years
148         back, when each benchmark took hundreds of milliseconds to run. Now,
149         the whole suite can finish in a a few hundres milliseconds, and things
150         look pretty flashy.
151
152         Instead, display a countdown that indicates which iteration of the
153         test we're on.
154
155         * resources/TEMPLATE.html: Work around a bug in Firefox by specifying
156         our style inside the .html file instead of in an external stylesheet.
157         Under heavy JavaScript load, Firefox neglects to apply external
158         stylesheets -- probably due to a starved timer.
159         
160         * resources/driver-TEMPLATE.html: Delay a little between runs to ensure
161         that our countdown has enough time to paint.
162
163 2013-09-26  Geoffrey Garen  <ggaren@apple.com>
164
165         Fixed a version number string in SunSpider 1.0.2
166         https://bugs.webkit.org/show_bug.cgi?id=122004
167
168         Reviewed by Gavin Barraclough.
169
170         * resources/driver-TEMPLATE.html:
171         * resources/results-TEMPLATE.html: 1.0.2!
172
173 2013-09-26  Geoffrey Garen  <ggaren@apple.com>
174
175         Tentative SunSpider 1.0.2
176         https://bugs.webkit.org/show_bug.cgi?id=122002
177
178         Reviewed by Gavin Barraclough.
179
180         Be even more aggressive about asking for zero delay between tests by
181         setting a timeout at the beginning of a test instead of the end. This is
182         another step toward reducing noise -- particularly noise caused by
183         power management -- just like the step down from 10ms to 0ms.
184
185         Browsers typically put a 4ms floor on timers. The sooner we set
186         our timer, the sooner it will fire.
187
188         * make-hosted:
189         * resources/driver-TEMPLATE.html: Changed the driver to set the timeout
190         earlier.
191         * tests/sunspider-1.0.2: Added. These tests are unchanged from 1.0.1.
192
193 2013-09-09  Filip Pizlo  <fpizlo@apple.com>
194
195         Stress tests should test the jsc profiler (-p)
196         https://bugs.webkit.org/show_bug.cgi?id=121043
197
198         Reviewed by Mark Hahnenberg.
199         
200         Add a jsc-stress-test that tries to profile SunSpider.
201
202         * profiler-test.yaml: Added.
203
204 2013-08-08  Cosmin Truta  <ctruta@blackberry.com>
205
206         SunSpider: Move the 3d-morph fix from version 1.0 to version 1.0.1
207         https://bugs.webkit.org/show_bug.cgi?id=117993
208
209         Reviewed by George Staikos.
210
211         Bump the SunSpider version to 1.0.1.
212
213         Unroll r149548 in order to restore version 1.0 to its original,
214         and add the 3d-morph fix to version 1.0.1 instead.
215
216         * hosted/sunspider.html: Updated to version 1.0.1.
217         * hosted/versions.html: Updated to version 1.0.1.
218         * make-hosted: Added support for version 1.0.1.
219         * resources/driver-TEMPLATE.html: Updated to version 1.0.1.
220         * resources/results-TEMPLATE.html: Updated to version 1.0.1.
221         * sunspider: Updated to version 1.0.1.
222         * sunspider-compare-results: Updated to version 1.0.1.
223         * tests/sunspider-1.0/3d-morph.js: Rolled back to the original version.
224         * tests/sunspider-1.0.1/LIST: Added.
225         * tests/sunspider-1.0.1/3d-cube.js: Added.
226         * tests/sunspider-1.0.1/3d-morph.js: Added.
227         * tests/sunspider-1.0.1/3d-raytrace.js: Added.
228         * tests/sunspider-1.0.1/access-binary-trees.js: Added.
229         * tests/sunspider-1.0.1/access-fannkuch.js: Added.
230         * tests/sunspider-1.0.1/access-nbody.js: Added.
231         * tests/sunspider-1.0.1/access-nsieve.js: Added.
232         * tests/sunspider-1.0.1/bitops-3bit-bits-in-byte.js: Added.
233         * tests/sunspider-1.0.1/bitops-bits-in-byte.js: Added.
234         * tests/sunspider-1.0.1/bitops-bitwise-and.js: Added.
235         * tests/sunspider-1.0.1/bitops-nsieve-bits.js: Added.
236         * tests/sunspider-1.0.1/controlflow-recursive.js: Added.
237         * tests/sunspider-1.0.1/crypto-aes.js: Added.
238         * tests/sunspider-1.0.1/crypto-md5.js: Added.
239         * tests/sunspider-1.0.1/crypto-sha1.js: Added.
240         * tests/sunspider-1.0.1/date-format-tofte.js: Added.
241         * tests/sunspider-1.0.1/date-format-xparb.js: Added.
242         * tests/sunspider-1.0.1/math-cordic.js: Added.
243         * tests/sunspider-1.0.1/math-partial-sums.js: Added.
244         * tests/sunspider-1.0.1/math-spectral-norm.js: Added.
245         * tests/sunspider-1.0.1/regexp-dna.js: Added.
246         * tests/sunspider-1.0.1/string-base64.js: Added.
247         * tests/sunspider-1.0.1/string-fasta.js: Added.
248         * tests/sunspider-1.0.1/string-tagcloud.js: Added.
249         * tests/sunspider-1.0.1/string-unpack-code.js: Added.
250         * tests/sunspider-1.0.1/string-validate-input.js: Added.
251
252 2013-05-03  Cosmin Truta  <ctruta@blackberry.com>
253
254         SunSpider 1.0: 3d-morph: use epsilon to check result
255         https://bugs.webkit.org/show_bug.cgi?id=115553
256
257         Reviewed by George Staikos.
258
259         SunSpider/3d-morph should be more lenient in validating its results, since they depend on Math.sin, which isn't formally specified.
260
261         * tests/sunspider-1.0/3d-morph.js:
262
263 2013-04-19  Filip Pizlo  <fpizlo@apple.com>
264
265         SunSpider/string-tagcloud should be more lenient in validating its results, since they depend on Math.log, which isn't formally specified
266         https://bugs.webkit.org/show_bug.cgi?id=114895
267
268         Reviewed by Michael Saboff.
269
270         * tests/sunspider-1.0/string-tagcloud.js:
271
272 2013-04-18  Filip Pizlo  <fpizlo@apple.com>
273
274         Whenever it is cheap and non-invasive, SunSpider tests should validate their results to ensure that the browser runs them correctly
275         https://bugs.webkit.org/show_bug.cgi?id=114852
276
277         Reviewed by Geoffrey Garen.
278         
279         This adds low-overhead checking of the results of each SunSpider tests. This is not
280         meant to be an exhaustive test that everything was executed correctly, but rather a
281         simple sanity check that will catch glaring mistakes. The philosophy here is that we're
282         not trying to prevent all forms of cheating, or that we're trying to prove the
283         browser's correctness. Moreover, these checks are meant to incur minimal overhead;
284         they currently clock in at <2% slow-down in SunSpider 1.0.
285         
286         The test expectations were generated using the LLInt on command-line, and I've verified
287         that Firefox 20, Chrome 26, IE 10, Safari 6, and ToT with all of the JITs agree. Note
288         that some tests cannot be precisely validated because they use Math functions, which
289         ECMAScript chooses to not formally specify - those functions like sin() and friends are
290         allowed to return implementation-dependent results. Also some tests cannot be validated
291         at all because their behavior is either intentionally random or is timezone-dependent.
292         But 23 out of 26 tests now have some kind of validation.
293         
294         I've updated the harnesses to show erroneous runs when displaying results.
295
296         * resources/TEMPLATE.html:
297         * resources/driver-TEMPLATE.html:
298         * resources/sunspider-analyze-results.js:
299         (formatResult):
300         (resultLine):
301         (printOutput):
302         * resources/sunspider-compare-results.js:
303         (.formatMean):
304         (.resultLine):
305         (.printOutput):
306         (sunspiderCompareResults):
307         * resources/sunspider-standalone-driver.js:
308         * tests/sunspider-1.0/3d-cube.js:
309         (Init):
310         * tests/sunspider-1.0/3d-morph.js:
311         * tests/sunspider-1.0/3d-raytrace.js:
312         * tests/sunspider-1.0/access-binary-trees.js:
313         * tests/sunspider-1.0/access-fannkuch.js:
314         * tests/sunspider-1.0/access-nbody.js:
315         * tests/sunspider-1.0/access-nsieve.js:
316         (sieve):
317         * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
318         * tests/sunspider-1.0/bitops-bits-in-byte.js:
319         * tests/sunspider-1.0/bitops-bitwise-and.js:
320         * tests/sunspider-1.0/bitops-nsieve-bits.js:
321         * tests/sunspider-1.0/controlflow-recursive.js:
322         * tests/sunspider-1.0/crypto-aes.js:
323         * tests/sunspider-1.0/crypto-md5.js:
324         * tests/sunspider-1.0/crypto-sha1.js:
325         * tests/sunspider-1.0/date-format-tofte.js:
326         * tests/sunspider-1.0/date-format-xparb.js:
327         * tests/sunspider-1.0/math-cordic.js:
328         * tests/sunspider-1.0/math-partial-sums.js:
329         (partial):
330         * tests/sunspider-1.0/math-spectral-norm.js:
331         * tests/sunspider-1.0/regexp-dna.js:
332         * tests/sunspider-1.0/string-base64.js:
333         * tests/sunspider-1.0/string-fasta.js:
334         (fastaRepeat):
335         (fastaRandom):
336         * tests/sunspider-1.0/string-tagcloud.js:
337         * tests/sunspider-1.0/string-unpack-code.js:
338         * tests/sunspider-1.0/string-validate-input.js:
339
340 2013-04-11  Filip Pizlo  <fpizlo@apple.com>
341
342         Get rid of the 10ms delay between test executions
343         https://bugs.webkit.org/show_bug.cgi?id=114458
344         <rdar://problem/13631112>
345
346         Reviewed by Darin Adler and Geoffrey Garen.
347         
348         More details on this in the bug. Long story short, I don't know why the 10ms delay is
349         there, and it appears to be the root of some noisyness and unreliability in the results.
350         Removing it makes results more consistent, which is a good thing.
351
352         * resources/driver-TEMPLATE.html:
353
354 2013-01-16  Eric Seidel  <eric@webkit.org>
355
356         Remove --shark* support from sunspider/run-sunspider now that Shark is dead
357         https://bugs.webkit.org/show_bug.cgi?id=99512
358
359         Reviewed by Darin Adler.
360
361         I tried to use --instruments, but that also seems broken (in the same way before/after this patch).
362
363         * sunspider:
364         (runTestsOnce):
365
366 2012-08-02  Filip Pizlo  <fpizlo@apple.com>
367
368         Unreviewed, revert debugging stuff I landed in http://trac.webkit.org/changeset/124557
369         by accident.
370
371         * tests/v8-v6/v8-crypto.js:
372         (am1):
373         (am2):
374         (am3):
375         (am4):
376         (bnpCopyTo):
377         (bnpFromString):
378         (bnpClamp):
379
380 2012-03-11  Maciej Stachowiak  <mjs@apple.com>
381
382         Add --instruments option to sunspider to profile with Instruments
383         https://bugs.webkit.org/show_bug.cgi?id=80783
384
385         Reviewed by Filip Pizlo.
386
387         * resources/TimeProfile20us.tracetemplate: Added.
388         * sunspider:
389         (runTestsOnce):
390
391 2011-11-15  Andy Wingo  <wingo@igalia.com>
392
393         Extend sunspider driver to be able to run kraken
394         https://bugs.webkit.org/show_bug.cgi?id=71799
395
396         Reviewed by Filip Pizlo.
397
398         * resources/sunspider-standalone-driver.js: Try to load a -data
399         file.  If that succeeds, we have a kraken-like test, so we time
400         the test using `load'.  Otherwise fall back to using `run'.
401
402 2011-08-11  Mark Rowe  <mrowe@apple.com>
403
404         Fix sunspider-compare-results to default to the same suite version as the sunspider script.
405
406         Rubber-stamped by Maciej Stachowiak.
407
408         * sunspider-compare-results:
409
410 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
411
412         Update SunSpider versioning to 1.0
413         https://bugs.webkit.org/show_bug.cgi?id=63870
414
415         Reviewed by Dan Bernstein.
416
417         * hosted/sunspider.html:
418         * hosted/versions.html:
419         * resources/driver-TEMPLATE.html:
420         * resources/results-TEMPLATE.html:
421         * sunspider:
422
423 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
424
425         Sunspider 0.9.1 harness doesn't actually close() its test documents
426         https://bugs.webkit.org/show_bug.cgi?id=47045
427
428         Reviewed by Daniel Bates.
429
430         * hosted/versions.html:
431         * resources/driver-TEMPLATE.html:
432
433 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
434
435         Not Reviewed. 
436         
437         Fix obviou typo in previous commit.
438
439         * tests/sunspider-1.0/bitops-bitwise-and.js:
440
441 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
442
443         Reviewed by Anders Carlsson.
444
445         SunSpider: all four bitops benchmarks can be replaced with NOP
446         https://bugs.webkit.org/show_bug.cgi?id=38446
447
448         * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
449         (TimeFunc): Save the result.
450         * tests/sunspider-1.0/bitops-bits-in-byte.js:
451         (TimeFunc): Save the result.
452         * tests/sunspider-1.0/bitops-bitwise-and.js: More explicitly
453         save the results.
454         * tests/sunspider-1.0/bitops-nsieve-bits.js:
455         (sieve): Save the result.
456
457 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
458
459         SunSpider: string-validate-input.js uses an undeclared variable named 'name', which is a DOM API
460         https://bugs.webkit.org/show_bug.cgi?id=60937
461
462         Reviewed by Dan Bernstein.
463
464         * tests/sunspider-1.0/string-validate-input.js:
465         (doTest): Rename the variable to username
466
467 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
468
469         Reviewed by Darin Adler.
470
471         SunSpider: The main function of math-cordic is dead code
472         https://bugs.webkit.org/show_bug.cgi?id=63863
473         
474         Try to prevent math-cordic from being eliminated as dead code or
475         compiled down to a constant.
476
477         * tests/sunspider-1.0/math-cordic.js:
478         (cordicsincos):
479         (cordic):
480
481 2011-06-30  Maciej Stachowiak  <mjs@apple.com>
482
483         Reviewed by Adam Barth.
484
485         Create sunspider-1.0 directory in preparation for fixing a bunch of bugs
486         https://bugs.webkit.org/show_bug.cgi?id=63782
487
488         * make-hosted: Learn about the new directory.
489         * tests/sunspider-1.0: Copied from PerformanceTests/SunSpider/tests/sunspider-0.9.1.
490
491 2010-11-29  Geoffrey Garen  <ggaren@apple.com>
492
493         Reviewed by Gavin Barraclough.
494
495         Improved accuracy of command-line SunSpider.
496
497         * resources/sunspider-standalone-driver.js: Use the returned number of
498         elapsed milliseconds from "run" and "checkSyntax" instead of measuring
499         ourselves, for slightly more accurate numbers.
500
501 2010-09-16  Gavin Barraclough  <barraclough@apple.com>
502
503         Reviewed by Mark Rowe.
504         https://bugs.webkit.org/show_bug.cgi?id=45924
505
506         Modify the SunSpider harness to allow a path to be passed to the --suite
507         argument, supress the normal prepend-"tests/" behaviour for suite names
508         containing a '/'.
509
510         Also wrap the code in an anonymous function to move variables out of global
511         scope (currently the harness cannot handle running tests the overwrite the
512         variable 'j').
513
514         * resources/sunspider-standalone-driver.js:
515         * sunspider:
516
517 2010-08-27  Michael Saboff  <msaboff@apple.com>
518
519         Reviewed by Stephanie Lewis.
520
521         Added version 5 and version 6 of V8 tests with modifications similar
522         to the version 4 tests so they'll work with the SunSpider test harness.
523         Note that these files are external source and therefore don't conform 
524         to WebKit coding standards.
525         https://bugs.webkit.org/show_bug.cgi?id=44638
526
527         * tests/v8-v5: Added.
528         * tests/v8-v5/LIST: Added.
529         * tests/v8-v5/v8-crypto.js: Added.
530         * tests/v8-v5/v8-deltablue.js: Added.
531         * tests/v8-v5/v8-earley-boyer.js: Added.
532         * tests/v8-v5/v8-raytrace.js: Added.
533         * tests/v8-v5/v8-regexp.js: Added.
534         * tests/v8-v5/v8-richards.js: Added.
535         * tests/v8-v5/v8-splay.js: Added.
536         * tests/v8-v6: Added.
537         * tests/v8-v6/LIST: Added.
538         * tests/v8-v6/v8-crypto.js: Added.
539         * tests/v8-v6/v8-deltablue.js: Added.
540         * tests/v8-v6/v8-earley-boyer.js: Added.
541         * tests/v8-v6/v8-raytrace.js: Added.
542         * tests/v8-v6/v8-regexp.js: Added.
543         * tests/v8-v6/v8-richards.js: Added.
544         * tests/v8-v6/v8-splay.js: Added.
545
546 2010-07-15  Stephanie Lewis  <slewis@apple.com>
547
548         Reviewed by Geoff Garen.
549
550         http://bugs.webkit.org/show_bug.cgi?id=42406
551         Output location of results file.
552
553         * sunspider:
554
555 2010-05-17  Maciej Stachowiak  <mjs@apple.com>
556
557         Reviewed by Mark Rowe.
558
559         Browser-hosted version of SunSpider runs tests twice as many times as advertised
560         https://bugs.webkit.org/show_bug.cgi?id=39201
561
562         * make-hosted: Make sure to reset test list after every test suite.
563
564 2010-05-07  Adam Roben  <aroben@apple.com>
565
566         Fix typo in sunspider-compare-results.js that caused incorrect
567         two-sample t statistics to be computed
568
569         Fixes <http://webkit.org/b/38768> SunSpider computes incorrect
570         two-sample t statistics
571
572         Reviewed by Darin Adler.
573
574         * resources/sunspider-compare-results.js:
575         (sunspiderCompareResults.resultLine): Fixed the calculation of the t
576         variable to match the definition of a two-sample t statistic. Looks
577         like this was just a simple typo.
578
579 2010-04-05  Darin Adler  <darin@apple.com>
580
581         Updated the ignore property for changes to the contents here.
582
583         * .: Modified property svn:ignore.
584
585 2010-02-20  Maciej Stachowiak  <mjs@apple.com>
586
587         Reviewed by Adam Barth.
588
589         Add sunspider-0.9.1 to the site in preparation for announcing it.
590         https://bugs.webkit.org/show_bug.cgi?id=35206
591
592         * hosted/sunspider.css: Remove tabs, which I noticed where present while
593         copying to the WebKitSite directory.
594
595 2010-01-11  Oliver Hunt  <oliver@apple.com>
596
597         Reviewed by Darin Adler.
598
599         Sunspider parse-only tests don't run
600         https://bugs.webkit.org/show_bug.cgi?id=33489
601
602         Fix parse-only test list
603
604         * tests/parse-only/LIST:
605
606 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
607
608         Reviewed by Adele Peterson.
609
610         Make SunSpider version more prominent in the title
611         https://bugs.webkit.org/show_bug.cgi?id=32574
612
613         * hosted/sunspider.html:
614         * hosted/versions.html:
615         * resources/driver-TEMPLATE.html:
616         * resources/results-TEMPLATE.html:
617
618 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
619
620         Reviewed by Darin Adler.
621
622         Make SunSpider refuse to compare results across content versions
623         https://bugs.webkit.org/show_bug.cgi?id=32573
624         
625         The results URL now includes a version indicator, and compare mode
626         will refuse to compare between versions.
627
628         * resources/driver-TEMPLATE.html:
629         * resources/results-TEMPLATE.html:
630
631 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
632
633         Reviewed by Darin Adler.
634
635         Some Browser-hosted SunSpider files are not valid HTML5
636         https://bugs.webkit.org/show_bug.cgi?id=32536
637         
638         Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
639         close tags and adding a meta charset declaration).
640
641         * hosted/sunspider.html:
642         * hosted/versions.html:
643         * resources/TEMPLATE.html:
644         * resources/driver-TEMPLATE.html:
645         * resources/results-TEMPLATE.html:
646
647 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
648
649         Reviewed by Eric Seidel.
650
651         Make sunspider-0.9.1 the default content set (both command-line and hosted)
652         https://bugs.webkit.org/show_bug.cgi?id=32537
653
654         * hosted/sunspider.html: Change default suite link.
655         * sunspider: Change default suite.
656         * sunspider-compare-results: Likewise.
657
658 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
659
660         Reviewed by Sam Weinig.
661
662         in-browser SunSpider suffers excessive penalty under power management
663         https://bugs.webkit.org/show_bug.cgi?id=32505
664         
665         I have made a few changes to address this:
666         
667         1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
668         document.write to reduce triggering of progress UI.
669         2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
670         stepped all the way down the whole time.
671         3) Run the test cycle an extra time for warmup before the runs that count.
672
673         * hosted/sunspider-record-result.js: Removed.
674         * make-hosted:
675         * resources/TEMPLATE.html:
676         * resources/driver-TEMPLATE.html:
677
678 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
679
680         Reviewed by Gavin Barraclough.
681
682         SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
683         https://bugs.webkit.org/show_bug.cgi?id=16806
684
685         Based on a patch by Eric Seidel.
686         
687         Fix the base64 computation to actually compute correct results. The impact on runtime of
688         the test is pretty small, but noticeable for some browsers. But at least it's not
689         doing a wrong and meaningless computation any more.
690         
691         * tests/sunspider-0.9.1/string-base64.js:
692         ():
693         (base64ToString):
694
695 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
696
697         Fixing commit error...
698         
699         I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
700
701         * resources/driver-TEMPLATE.html:
702
703 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
704
705         Reviewed by Sam Weinig.
706
707         Use JSON.parse instead of eval for Web-hosted SunSpider results processing
708         https://bugs.webkit.org/show_bug.cgi?id=32490
709
710         * hosted/json2.js: Added. Incorporated from json.org
711         * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
712         * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
713         results.
714
715 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
716
717         Reviewed by Oliver Hunt.
718
719         Enable Web-hosted version of SunSpider to handle multiple versions
720         https://bugs.webkit.org/show_bug.cgi?id=32478
721
722         * make-hosted: Now generate the hosted version to be able to run both the
723         0.9 and the 0.9.1 test suites.
724         * hosted: Modified property svn:ignore.
725         * hosted/sunspider-driver.html: Removed.
726         * hosted/sunspider-results.html: Removed.
727         * hosted/sunspider.html:
728         * hosted/versions.html: Added.
729         * resources/TEMPLATE.html:
730         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
731         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
732
733 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
734
735         Reviewed by Oliver Hunt.
736
737         Give command-line sunspider the ability to handle multiple suites and versions
738         https://bugs.webkit.org/show_bug.cgi?id=32477
739         
740         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
741         enumerating the tests. Current suites are:
742         
743         sunspider-0.9 - riginal SunSpider content 
744         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
745         ubench - "ubench" microbenchmark suite
746         v8-v4 - version 4 of the v8 benchmark
747         parse-only - some parser tests
748         
749         Sunspider can now be run with the --suite= parameter to select a
750         suite. The default is --suite=sunspider-0.9.
751
752         * .: Modified property svn:ignore to ignore new results directories.
753         * make-hosted: Generate based on sunspider-0.9 suite.
754         * resources/TEMPLATE.html: Fix an HTML compliance problem.
755         * resources/sunspider-standalone-driver.js: Become aware of suite names.
756         * sunspider: Modified to handle multiple suites as described above.
757         * sunspider-compare-results: ditto
758         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
759         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
760         * tests/LIST-UBENCH: Removed.
761         * tests/LIST-V8: Removed.
762         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
763         * tests/3d-morph.js: ditto
764         * tests/3d-raytrace.js: ditto
765         * tests/access-binary-trees.js: ditto
766         * tests/access-fannkuch.js: ditto
767         * tests/access-nbody.js: ditto
768         * tests/access-nsieve.js: ditto
769         * tests/bitops-3bit-bits-in-byte.js: ditto
770         * tests/bitops-bits-in-byte.js: ditto
771         * tests/bitops-bitwise-and.js: ditto
772         * tests/bitops-nsieve-bits.js: ditto
773         * tests/controlflow-recursive.js: ditto
774         * tests/crypto-aes.js: ditto
775         * tests/crypto-md5.js: ditto
776         * tests/crypto-sha1.js: ditto
777         * tests/date-format-tofte.js: ditto
778         * tests/date-format-xparb.js: ditto
779         * tests/math-cordic.js: ditto
780         * tests/math-partial-sums.js: ditto
781         * tests/math-spectral-norm.js: ditto
782         * tests/regexp-dna.js: ditto
783         * tests/string-base64.js: ditto
784         * tests/string-fasta.js: ditto
785         * tests/string-tagcloud.js: ditto
786         * tests/string-unpack-code.js: ditto
787         * tests/string-validate-input.js: ditto
788         * tests/sunspider-0.9/: Added
789         * tests/sunspider-0.9/*.js: Moved from one level up
790         * tests/sunspider-0.9.1: Added.
791         * tests/sunspider-0.9.1/*.js: Moved from one level up.
792         * tests/v8-crypto.js: Moved to v8-v4/ directory.
793         * tests/v8-deltablue.js: ditto
794         * tests/v8-earley-boyer.js: ditto
795         * tests/v8-raytrace.js: ditto
796         * tests/v8-regexp.js: ditto
797         * tests/v8-richards.js: ditto
798         * tests/v8-splay.js: ditto
799         * tests/v8-v4: Added.
800
801 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
802
803         Reviewed by Darin Adler.
804
805         Platform specific null device replaced with a platform independent.
806         https://bugs.webkit.org/show_bug.cgi?id=29544
807
808         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
809
810 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
811
812         Reviewed by Maciej Stachowiak.
813
814         [Qt] Make sunspider script work on Windows platform.
815         https://bugs.webkit.org/show_bug.cgi?id=29544
816
817         * sunspider:
818         Using platform specific null device instead of /dev/null.
819
820 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
821
822         Reviewed by Oliver Hunt.
823
824         https://bugs.webkit.org/show_bug.cgi?id=26540
825         Currently the SunSpider test driver lacks an option to run a test suite that
826         will test JavaScriptCore parsing performance only.  This patch adds just such
827         a test suite and option to SunSpider as well as the jsc test shell.  I've included
828         three large javascript source files found in the wild: jquery, mootools and prototype.
829         Combined with the concatenation of all three, these form a new testsuite to measure
830         and test pure JavaScriptCore parsing performance.
831
832         * resources/sunspider-standalone-driver.js:
833           Load the js with parse-only flag if it is part of the parse-only suite.
834         * sunspider:
835           Add the parse-only suite to the list of options.
836         * sunspider-compare-results:
837           Add the parse-only suite to the list of options.
838         * tests/LIST-PARSE-ONLY: Added.
839         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
840           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
841         * tests/parse-only/jquery-1.3.2.js: Added.
842           Latest version from jquery.com.
843         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
844           Latest version from mootools.net.
845         * tests/parse-only/prototype-1.6.0.3.js: Added.
846           Latest version from prototypejs.org.
847
848 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
849
850         Update V8 benchmark to version 4.
851         
852         Reviewed by Cameron Zwarich.
853
854         * tests/LIST-V8:
855         * tests/v8-crypto.js:
856         * tests/v8-deltablue.js:
857         * tests/v8-earley-boyer.js:
858         * tests/v8-raytrace.js:
859         * tests/v8-regexp.js: Added.
860         * tests/v8-richards.js:
861         * tests/v8-splay.js: Added.
862
863 2009-05-26  Olivier DOLE  <odole@pleyo.com>
864
865         Reviewed by Maciej.
866
867         Store sunspider test suite results in a specific directory for each
868         different type of benchmark (sunspider, V8 or ubench).
869
870         * sunspider:
871         * sunspider-compare-results:
872
873 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
874
875         Build fix.
876
877         * SunSpider.make:
878
879 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
880
881         Include SunSpider tests in production builds.
882         
883         Reviewed by Adam Roben.
884
885         * SunSpider.make: Added.
886         * xcopy.excludes: Added.
887
888 2009-03-04  Oliver Hunt  <oliver@apple.com>
889
890         Added a few more tests to ubench
891
892         RS=Gavin Barraclough
893
894         Adding a couple of test to cover too few and too many arguments.  This
895         is needed to prevent regressions in all call paths from my current refactoring.
896
897         * tests/LIST-UBENCH:
898         * tests/ubench/function-correct-args.js: Added.
899         (f):
900         * tests/ubench/function-excess-args.js: Added.
901         (f):
902
903 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
904
905         Fix a bug where when V8's or UBENCH's test suite list file is missing
906         the error message is suggesting that SunSpider's list file is missing.
907
908         Reviewed by Sam Weinig.
909
910         * sunspider:
911         (loadTestsList): Fix the error message to use the right list name.
912
913 2008-10-07  David Hyatt  <hyatt@apple.com>
914
915         https://bugs.webkit.org/show_bug.cgi?id=18989
916
917         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
918         we want all browsers to be doing the same thing on the test.
919
920         Reviewed by Maciej
921
922         * tests/regexp-dna.js:
923
924 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
925
926         Rubber-stamped by Maciej Stachowiak.
927
928         The bleeding edge branch of V8 now supports passing fileanmes after
929         '-f', so the SunSpider script should be changed to do this again.
930         This allows all 3 major JavaScript engines to be used with the
931         unmodified SunSpider script.
932
933         * sunspider:
934
935 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
936
937         Reviewed by Maciej Stachowiak.
938
939         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
940         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
941         correct the punctuation in its documentation.
942
943         * sunspider:
944
945 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
946
947         Reviewed by Oliver Hunt.
948
949         Bug 20616: Incorporate V8 benchmarks in testing
950         <https://bugs.webkit.org/show_bug.cgi?id=20616>
951
952         This is a first attempt at running the V8 benchmarks within SunSpider,
953         with the --v8 option. The number of iterations is fixed for each
954         benchmark, and was chosen to make each of the benchmarks run between
955         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
956
957         * sunspider:
958         * tests/LIST-V8: Added.
959         * tests/v8-crypto.js: Added.
960         * tests/v8-deltablue.js: Added.
961         * tests/v8-earley-boyer.js: Added.
962         * tests/v8-raytrace.js: Added.
963         * tests/v8-richards.js: Added.
964
965 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
966
967         Not reviewed.
968
969         Remove a local change that was accidentally committed in r36034.
970
971         * resources/sunspider-standalone-driver.js:
972
973 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
974
975         Rubber-stamped by Oliver Hunt.
976
977         Make SunSpider work with JavaScript shells that don't take filenames
978         after a '-f' argument.
979
980         * sunspider:
981
982 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
983
984         Reviewed by Timothy.
985
986         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
987
988         * sunspider:
989         * tests/LIST-SQUIRRELFISH: Removed.
990
991 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
992
993         Rubber-stamped by Mark Rowe.
994
995         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
996
997         * sunspider:
998
999 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
1000
1001         Rubber stamped by Oliver.
1002         
1003         - add newly working tests to squirrelfish list
1004
1005         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
1006
1007 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
1008
1009         Rubber stamped by Oliver.
1010         
1011         - move ubench tests properly.
1012
1013         * tests/ubench: Added.
1014         * tests/ubench/function-closure.js: Added.
1015         * tests/ubench/function-empty.js: Added.
1016         * tests/ubench/function-missing-args.js: Added.
1017         * tests/ubench/function-sum.js: Added.
1018         * tests/ubench/loop-empty-resolve.js: Added.
1019         * tests/ubench/loop-empty.js: Added.
1020         * tests/ubench/loop-sum.js: Added.
1021
1022 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
1023
1024         Rubber stamped by Oliver.
1025
1026         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
1027
1028 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
1029
1030         Reviewed by Oliver.
1031
1032         - make --squirrelfish mode use the real harness, but a reduced set of tests
1033
1034         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
1035
1036         * sunspider:
1037         * tests/LIST-SQUIRRELFISH:
1038         * tests/LIST-UBENCH: Added.
1039         * tests/squirrelfish: Moved to tests/ubench
1040
1041 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
1042
1043         Reviewed by Maciej Stachowiak.
1044         
1045         Removed an unused test.
1046
1047         * tests/squirrelfish/loop-resolve.js: Removed.
1048
1049 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
1050
1051         Reviewed by Oliver.
1052         
1053         Add newly runnable tests to --squirrelfish mode.
1054
1055         * tests/LIST-SQUIRRELFISH:
1056
1057 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
1058
1059         Reviewed by Maciej Stachowiak.
1060         
1061         A little more love for --squirrelfish mode:
1062         
1063         Fixed a misplaced sort. Results properly sort now.
1064         
1065         Added a test list just for squirrelfish, and updated it to include
1066         all currently passing SunSpider tests.
1067         
1068         Fixed the pruning regexp to match 3d-morph.js.
1069
1070         * sunspider:
1071
1072 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
1073
1074         Reviewed by Oliver Hunt.
1075         
1076         --squirrelfish mode: pared down tests for squirrelfish to chew on.
1077
1078         * sunspider:
1079         * tests/LIST:
1080         * tests/squirrelfish: Added.
1081         * tests/squirrelfish/function-closure.js: Added.
1082         * tests/squirrelfish/function-empty.js: Added.
1083         * tests/squirrelfish/function-missing-args.js: Added.
1084         * tests/squirrelfish/function-sum.js: Added.
1085         * tests/squirrelfish/loop-empty-resolve.js: Added.
1086         * tests/squirrelfish/loop-empty.js: Added.
1087         * tests/squirrelfish/loop-resolve.js: Added.
1088         * tests/squirrelfish/loop-sum.js: Added.
1089
1090 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
1091
1092         Reviewed by Geoff.
1093
1094         * sunspider: Pause update daemon on Darwin to make results reliable
1095         again (workaround for <rdar://problem/5811127>).
1096
1097 2008-02-07  Eric Seidel  <eric@webkit.org>
1098
1099         Reviewed by Sam.
1100
1101         * sunspider: add an error message when ./tests/LIST is not found
1102
1103 2007-12-29  Alp Toker  <alp@atoker.com>
1104
1105         Fix typo.
1106
1107         * hosted/sunspider.html:
1108
1109 2007-12-29  Alp Toker  <alp@atoker.com>
1110
1111         Reviewed by Maciej.
1112
1113         Provide the current URL in the SunSpider results page. Useful for
1114         copying benchmark results out of browsers without a location bar or
1115         where the location bar can't handle long URLs.
1116
1117         * hosted/sunspider-results.html:
1118
1119 2007-12-20  Eric Seidel  <eric@webkit.org>
1120
1121         Reviewed by mjs.
1122
1123         * sunspider: increase default number of runs for more accurate numbers
1124
1125 2007-12-18  Mark Rowe  <mrowe@apple.com>
1126
1127         Fix typo.
1128
1129         * hosted/sunspider.html:
1130
1131 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
1132
1133         Reviewed by Eric.
1134         
1135         - give an extra digit of precision for small differences.
1136
1137         * resources/sunspider-compare-results.js:
1138
1139 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
1140
1141         Reviewed by Eric.
1142         
1143         - work around IE layout bugs
1144
1145         * hosted/sunspider.css:
1146
1147 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
1148
1149         Reviewed by Mark.
1150         
1151         - fix compare feature
1152
1153         * hosted/sunspider-results.html:
1154
1155 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
1156
1157         Reviewed by Mark.
1158         
1159         - Change the format of the output URL to be shorter, to meet IE constraints
1160
1161         * hosted/sunspider-driver.html:
1162         * hosted/sunspider-results.html:
1163
1164 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
1165
1166         Reviewed by Oliver.
1167         
1168         - Add some explanatory test and a dumb logo to SunSpider web hosted version
1169
1170         * hosted/sunspider-driver.html:
1171         * hosted/sunspider-results.html:
1172         * hosted/sunspider.css:
1173         * hosted/sunspider.html:
1174
1175 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
1176
1177         Reviewed by Oliver.
1178         
1179         - Make the web-hosted version of SunSpider look a little bit less drab.
1180
1181         * hosted/sunspider-driver.html:
1182         * hosted/sunspider-results.html:
1183         * hosted/sunspider.css: Added.
1184         * hosted/sunspider.html:
1185         * resources/TEMPLATE.html:
1186
1187 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
1188
1189         Reviewed by Sam.
1190         
1191         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
1192         
1193         A sample of what new-style results would look like:
1194         
1195         TEST                   COMPARISON            FROM                 TO             DETAILS
1196         ...
1197         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
1198         ...
1199             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
1200         
1201         * resources/sunspider-compare-results.js:
1202
1203 2007-11-14  Eric Seidel  <eric@webkit.org>
1204
1205         Reviewed by Sam.
1206
1207         * sunspider: add --shark-cache for L2 Cache Miss profiling
1208
1209 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
1210
1211         Reviewed by Darin Adler.
1212
1213         http://bugs.webkit.org/show_bug.cgi?id=15925
1214         SunSpider should check for Shark being installed
1215
1216         * sunspider: Explicitly test that shark command line tool is installed.
1217
1218 2007-11-04  Darin Adler  <darin@apple.com>
1219
1220         * hosted/sunspider.html: Tweaked formatting a bit.
1221
1222 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
1223
1224         Reviewed by Darin Adler.
1225
1226         - Add compare mode and a description of the benchmark to the hosted version.
1227
1228         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
1229         * hosted/sunspider.html: Added intro text.
1230         * make-hosted: Copy new file.
1231         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
1232         a function.
1233         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
1234         expects.
1235         * sunspider-compare-results: Use sunspider-standalone-compare.js.
1236
1237 2007-11-02  Darin Adler  <darin@apple.com>
1238
1239         * sunspider: Changed "--base" to "--set-baseline".
1240
1241 2007-11-02  Darin Adler  <darin@apple.com>
1242
1243         Reviewed by Maciej.
1244
1245         * sunspider: Added a "--base" option that records the current test as a baseline.
1246         Moved the code to find the newest .mshark file into a function (since I needed to use
1247         it in the other script and I prefer to copy/paste a function).
1248
1249         * sunspider-compare-results: Added logic to compare the baseline with the last
1250         run when you don't pass any parameters.
1251
1252 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
1253
1254         Reviewed by Eric.
1255         
1256         - Added list of ECMAScript3 features that SunSpider does not cover currently.
1257
1258         * UNCOVERED: Added.
1259
1260 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
1261
1262         Reviewed by Oliver.
1263         
1264         - Give better error messages when sunspider-compare-results fails
1265
1266         * sunspider-compare-results:
1267
1268 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
1269
1270         Reviewed by Darin Adler.
1271         
1272         - fix Date tests to work in command-line SpiderMonkey
1273
1274         * tests/date-format-tofte.js:
1275         * tests/date-format-xparb.js:
1276
1277 2007-10-29  Darin Adler  <darin@apple.com>
1278
1279         Reviewed by Maciej.
1280
1281         * sunspider: Added --shark20 option, to run Shark at its highest sample
1282         resolution instead of its default.
1283
1284 2007-10-28  Adam Roben  <aroben@apple.com>
1285
1286         Fix SunSpider on Windows
1287
1288         Reviewed by Eric.
1289
1290         * sunspider: Don't use colons in filenames.
1291
1292 2007-10-28  Darin Adler  <darin@apple.com>
1293
1294         Reviewed by Adam.
1295
1296         * resources/sunspider-analyze-results.js: Tweak the output format a little.
1297         Change so that when there's only one run we don't write out confidence
1298         intervals at all rather than writing "NaN" over and over again.
1299
1300         * sunspider: Use the time and date as a suffix on the results file, that way
1301         all the results are left behind in the tmp directory. This will make it easy
1302         for us to add features that compare past results in the future. Also rename
1303         the .mshark files using the same scheme.
1304
1305         * sunspider-compare-results: Relax the parsing rules so we can parse the
1306         results file exactly as it's emitted from sunspider.
1307
1308 2007-10-25  Adam Roben  <aroben@apple.com>
1309
1310         Strip carriage returns from results
1311
1312         Carriage returns were screwing up sunspider-compare-results on
1313         Windows.
1314
1315         Rubberstamped by Oliver.
1316
1317         * sunspider:
1318
1319 2007-10-24  Darin Adler  <darin@apple.com>
1320
1321         Reviewed by Eric.
1322
1323         * sunspider: Open the profile in shark after completing a run.
1324         * sunspider-compare-results: Tiny perl tweaks.
1325
1326 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
1327
1328         Reviewed by Darin Adler.
1329
1330         - Don't hardcode my path to testkjs
1331         
1332         * sunspider-compare-results:
1333
1334 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
1335
1336         Reviewed by Darin Adler.
1337         
1338         - Add a compare script to compare results, and improve formatting of standard results a bit.
1339
1340         * resources/sunspider-analyze-results.js:
1341         * resources/sunspider-compare-results.js: Added.
1342         * sunspider-compare-results: Added.
1343
1344 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
1345
1346         Reviewed by Darin Adler.
1347
1348         * resources/sunspider-analyze-results.js: Do the error range
1349         properly, using the t-distribution instead of 1.96 (which was
1350         based on the normal distribution).
1351         
1352         * sunspider: Print results in a way that will be friendlier to the
1353         soon-to-come compare mode.
1354
1355 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
1356
1357         Reviewed by Review.
1358         
1359         - add more tests (probably the final set for now)
1360         
1361         Already balanced. Added date, regexp, control flow, and a few
1362         more string and object/array access tests.
1363         
1364         * tests/LIST:
1365         * tests/access-fannkuch.js: Added.
1366         * tests/access-nbody.js: Added.
1367         * tests/controlflow-recursive.js: Added.
1368         * tests/date-format-tofte.js: Added.
1369         * tests/date-format-xparb.js: Added.
1370         * tests/regexp-dna.js: Added.
1371         * tests/string-validate-input.js: Added.
1372
1373 2007-10-21  Eric Seidel  <eric@webkit.org>
1374
1375         Reviewed by mjs.
1376
1377         * sunspider: make --shark only sample from testkjs
1378
1379 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
1380
1381         Rubber stamped by Eric.
1382         
1383         - added crypto tests (already balanced)
1384
1385         * tests/LIST:
1386         * tests/crypto-aes.js: Added.
1387         * tests/crypto-md5.js: Added.
1388         * tests/crypto-sha1.js: Added.
1389
1390 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
1391
1392         Reviewed by Adam.
1393         
1394         - rebalance new tests
1395         
1396         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
1397         
1398         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1399
1400         This balances them with the existing tests.
1401
1402         * tests/access-binary-trees.js:
1403         * tests/access-nsieve.js:
1404         * tests/bitops-nsieve-bits.js:
1405         * tests/math-partial-sums.js:
1406         * tests/math-spectral-norm.js:
1407         * tests/string-fasta.js:
1408
1409 2007-10-20  Eric Seidel  <eric@webkit.org>
1410
1411         Reviewed by darin.
1412         
1413         Add improved argument handling to sunspider, including
1414         --runs=<number>, --tests=<pattern, --shark, and --help
1415         Also re-factor code into subroutines
1416
1417         * sunspider: improved argument handling
1418
1419 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1420
1421         Reviewed by Mark.
1422
1423         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
1424         
1425         * TODO:
1426         * tests/LIST:
1427         * tests/access-binary-trees.js: Added.
1428         * tests/access-nsieve.js: Added.
1429         * tests/bitops-nsieve-bits.js: Added.
1430         * tests/math-partial-sums.js: Added.
1431         * tests/math-spectral-norm.js: Added.
1432         * tests/string-fasta.js: Added.
1433
1434 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1435
1436         Reviewed by Darin Adler.
1437
1438         * TODO: Updated to note areas that are now well-covered.
1439         
1440         - Change the tests and the driver to avoid leaving large object
1441         graphs hanging around, since that throws off the subsequent tests.
1442         
1443         * resources/sunspider-standalone-driver.js:
1444         * tests/3d-cube.js:
1445         * tests/3d-morph.js:
1446         * tests/3d-raytrace.js:
1447         * tests/bitops-bitwise-and.js:
1448         * tests/string-base64.js:
1449         * tests/string-tagcloud.js:
1450
1451 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1452
1453         Reviewed by Nikolas.
1454         
1455         - Rebalance test complexity.
1456         
1457         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
1458         
1459         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1460
1461         However, for some tests, IE7 was a huge outlier compared to other
1462         major browsers. For those tests, I used Firefox 2.0.0.8 on the
1463         same OS and hardware as a normalization basis.
1464         
1465         * tests/3d-morph.js:
1466         * tests/3d-raytrace.js:
1467         * tests/bitops-3bit-bits-in-byte.js:
1468         * tests/bitops-bits-in-byte.js:
1469         * tests/bitops-bitwise-and.js:
1470         * tests/math-cordic.js:
1471         * tests/string-base64.js:
1472         * tests/string-tagcloud.js:
1473         * tests/string-unpack-code.js:
1474
1475 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1476
1477         Reviewed by Oliver.
1478         
1479         - Fix the browser-hosted driver to be compatible with Safari 2
1480
1481         * hosted/sunspider-driver.html:
1482
1483 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1484
1485         Rubber stamped by Hyatt.
1486         
1487         - Add BSD license notice to cordic test, the original author said to do so.
1488
1489         * tests/math-cordic.js:
1490
1491 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1492
1493         Reviewed by Eric.
1494         
1495         - Support Windows and Windows browsers.
1496
1497         * hosted/sunspider-results.html: Display results correctly in IE.
1498         * sunspider: tweaks to work on Windows, and with spaces in the path.
1499
1500 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1501
1502         Rubber stamped by Oliver.
1503
1504         * tests/string-tagcloud.js: Remove stray print()
1505
1506 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
1507
1508         Reviewed by Oliver.
1509         
1510         Add some string test cases.
1511
1512         * tests/string-base64.js: Added. Does base64 encoding and decoding.
1513         * tests/string-tagcloud.js: Added. Parses a JSON string and
1514         generates tagcloud HTML markup based on the data.        
1515         * tests/string-unpack-code.js: Added. Unpack compressed versions of
1516         four of the most popular JavaScript libraries.
1517         * tests/LIST: Added the new tests.
1518
1519 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
1520
1521         Reviewed by Anders.
1522
1523         * TODO: Updated with info on how to do the stats properly.
1524
1525         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
1526         avoid overly slowing down the benchmark.
1527
1528         Added some 3D-related test cases.
1529         
1530         * tests/3d-cube.js: Added.
1531         * tests/3d-morph.js: Added.
1532         * tests/3d-raytrace.js: Added.
1533         * tests/LIST:
1534
1535 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
1536
1537         Reviewed by Darin Adler.
1538
1539         - New JavaScript benchmark
1540         http://bugs.webkit.org/show_bug.cgi?id=15515
1541         
1542         A start on a new JavaScript benchmark, with standalone and
1543         browser-hosted drivers.
1544         
1545         The test content is still incomplete and the drivers could use
1546         some more features but there is enough here to make a good start.
1547         
1548         The drivers compute a 95% confidence interval on the mean for the
1549         whole test, each category, and each individual test to make it
1550         easier to tell whether differences are statistically
1551         significant. The confidence interval can be narrowed by running
1552         the test more times.
1553         
1554         * TODO: Added. Remaining things that need to be done.
1555         * sunspider: Added. Perl script that acts as the standalone test driver.
1556         * resources/sunspider-standalone-driver.js: Added. JavaScript part
1557         of standalone test driver.
1558         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
1559         analysis code.
1560         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
1561         * make-hosted: Added. Script to generate browser-hosted tests.
1562         * hosted/sunspider.html: Added. Start page for browser-hosted test.
1563         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
1564         * hosted/sunspider-results.html: Added. Results page for browser-hosted
1565         tests.
1566         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
1567         tests.
1568         * tests/LIST: Added. List of tests to use.
1569         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
1570         * tests/bitops-bits-in-byte.js: Added. Ditto.
1571         * tests/bitops-bitwise-and.js: Added. Ditto.
1572         * tests/math-cordic.js: Added. Ditto.
1573