4f61b11026cc1e3f47d1c1e75bd44ee8ce2a60d6
[WebKit-https.git] / PerformanceTests / SunSpider / ChangeLog
1 2013-10-08  Geoffrey Garen  <ggaren@apple.com>
2
3         Refined power management in SunSpider 1.0.2
4         https://bugs.webkit.org/show_bug.cgi?id=122525
5
6         Reviewed by Gavin Barraclough.
7
8         * hosted/sunspider.css:
9         (#frameparent):
10         (#countdown):
11         * resources/TEMPLATE.html: Provide an explicit countdown element in the
12         parent, instead of hacking it into each child frame.
13
14         * resources/driver-TEMPLATE.html: Be more explicit about how we respond
15         to power management. Just moving setTimeout(0) inside the loop is a
16         little too subtle, and may not work across browsers.
17
18         Also, leave a little extra time after changing the countdown graphic,
19         to give the browser some time to paint.
20
21 2013-10-02  Geoffrey Garen  <ggaren@apple.com>
22
23         Improved painting behavior in SunSpider 1.0.2
24         https://bugs.webkit.org/show_bug.cgi?id=122257
25
26         Reviewed by Oliver Hunt.
27
28         Individual test names with times was a good painting strategy years
29         back, when each benchmark took hundreds of milliseconds to run. Now,
30         the whole suite can finish in a a few hundres milliseconds, and things
31         look pretty flashy.
32
33         Instead, display a countdown that indicates which iteration of the
34         test we're on.
35
36         * resources/TEMPLATE.html: Work around a bug in Firefox by specifying
37         our style inside the .html file instead of in an external stylesheet.
38         Under heavy JavaScript load, Firefox neglects to apply external
39         stylesheets -- probably due to a starved timer.
40         
41         * resources/driver-TEMPLATE.html: Delay a little between runs to ensure
42         that our countdown has enough time to paint.
43
44 2013-09-26  Geoffrey Garen  <ggaren@apple.com>
45
46         Fixed a version number string in SunSpider 1.0.2
47         https://bugs.webkit.org/show_bug.cgi?id=122004
48
49         Reviewed by Gavin Barraclough.
50
51         * resources/driver-TEMPLATE.html:
52         * resources/results-TEMPLATE.html: 1.0.2!
53
54 2013-09-26  Geoffrey Garen  <ggaren@apple.com>
55
56         Tentative SunSpider 1.0.2
57         https://bugs.webkit.org/show_bug.cgi?id=122002
58
59         Reviewed by Gavin Barraclough.
60
61         Be even more aggressive about asking for zero delay between tests by
62         setting a timeout at the beginning of a test instead of the end. This is
63         another step toward reducing noise -- particularly noise caused by
64         power management -- just like the step down from 10ms to 0ms.
65
66         Browsers typically put a 4ms floor on timers. The sooner we set
67         our timer, the sooner it will fire.
68
69         * make-hosted:
70         * resources/driver-TEMPLATE.html: Changed the driver to set the timeout
71         earlier.
72         * tests/sunspider-1.0.2: Added. These tests are unchanged from 1.0.1.
73
74 2013-09-09  Filip Pizlo  <fpizlo@apple.com>
75
76         Stress tests should test the jsc profiler (-p)
77         https://bugs.webkit.org/show_bug.cgi?id=121043
78
79         Reviewed by Mark Hahnenberg.
80         
81         Add a jsc-stress-test that tries to profile SunSpider.
82
83         * profiler-test.yaml: Added.
84
85 2013-08-08  Cosmin Truta  <ctruta@blackberry.com>
86
87         SunSpider: Move the 3d-morph fix from version 1.0 to version 1.0.1
88         https://bugs.webkit.org/show_bug.cgi?id=117993
89
90         Reviewed by George Staikos.
91
92         Bump the SunSpider version to 1.0.1.
93
94         Unroll r149548 in order to restore version 1.0 to its original,
95         and add the 3d-morph fix to version 1.0.1 instead.
96
97         * hosted/sunspider.html: Updated to version 1.0.1.
98         * hosted/versions.html: Updated to version 1.0.1.
99         * make-hosted: Added support for version 1.0.1.
100         * resources/driver-TEMPLATE.html: Updated to version 1.0.1.
101         * resources/results-TEMPLATE.html: Updated to version 1.0.1.
102         * sunspider: Updated to version 1.0.1.
103         * sunspider-compare-results: Updated to version 1.0.1.
104         * tests/sunspider-1.0/3d-morph.js: Rolled back to the original version.
105         * tests/sunspider-1.0.1/LIST: Added.
106         * tests/sunspider-1.0.1/3d-cube.js: Added.
107         * tests/sunspider-1.0.1/3d-morph.js: Added.
108         * tests/sunspider-1.0.1/3d-raytrace.js: Added.
109         * tests/sunspider-1.0.1/access-binary-trees.js: Added.
110         * tests/sunspider-1.0.1/access-fannkuch.js: Added.
111         * tests/sunspider-1.0.1/access-nbody.js: Added.
112         * tests/sunspider-1.0.1/access-nsieve.js: Added.
113         * tests/sunspider-1.0.1/bitops-3bit-bits-in-byte.js: Added.
114         * tests/sunspider-1.0.1/bitops-bits-in-byte.js: Added.
115         * tests/sunspider-1.0.1/bitops-bitwise-and.js: Added.
116         * tests/sunspider-1.0.1/bitops-nsieve-bits.js: Added.
117         * tests/sunspider-1.0.1/controlflow-recursive.js: Added.
118         * tests/sunspider-1.0.1/crypto-aes.js: Added.
119         * tests/sunspider-1.0.1/crypto-md5.js: Added.
120         * tests/sunspider-1.0.1/crypto-sha1.js: Added.
121         * tests/sunspider-1.0.1/date-format-tofte.js: Added.
122         * tests/sunspider-1.0.1/date-format-xparb.js: Added.
123         * tests/sunspider-1.0.1/math-cordic.js: Added.
124         * tests/sunspider-1.0.1/math-partial-sums.js: Added.
125         * tests/sunspider-1.0.1/math-spectral-norm.js: Added.
126         * tests/sunspider-1.0.1/regexp-dna.js: Added.
127         * tests/sunspider-1.0.1/string-base64.js: Added.
128         * tests/sunspider-1.0.1/string-fasta.js: Added.
129         * tests/sunspider-1.0.1/string-tagcloud.js: Added.
130         * tests/sunspider-1.0.1/string-unpack-code.js: Added.
131         * tests/sunspider-1.0.1/string-validate-input.js: Added.
132
133 2013-05-03  Cosmin Truta  <ctruta@blackberry.com>
134
135         SunSpider 1.0: 3d-morph: use epsilon to check result
136         https://bugs.webkit.org/show_bug.cgi?id=115553
137
138         Reviewed by George Staikos.
139
140         SunSpider/3d-morph should be more lenient in validating its results, since they depend on Math.sin, which isn't formally specified.
141
142         * tests/sunspider-1.0/3d-morph.js:
143
144 2013-04-19  Filip Pizlo  <fpizlo@apple.com>
145
146         SunSpider/string-tagcloud should be more lenient in validating its results, since they depend on Math.log, which isn't formally specified
147         https://bugs.webkit.org/show_bug.cgi?id=114895
148
149         Reviewed by Michael Saboff.
150
151         * tests/sunspider-1.0/string-tagcloud.js:
152
153 2013-04-18  Filip Pizlo  <fpizlo@apple.com>
154
155         Whenever it is cheap and non-invasive, SunSpider tests should validate their results to ensure that the browser runs them correctly
156         https://bugs.webkit.org/show_bug.cgi?id=114852
157
158         Reviewed by Geoffrey Garen.
159         
160         This adds low-overhead checking of the results of each SunSpider tests. This is not
161         meant to be an exhaustive test that everything was executed correctly, but rather a
162         simple sanity check that will catch glaring mistakes. The philosophy here is that we're
163         not trying to prevent all forms of cheating, or that we're trying to prove the
164         browser's correctness. Moreover, these checks are meant to incur minimal overhead;
165         they currently clock in at <2% slow-down in SunSpider 1.0.
166         
167         The test expectations were generated using the LLInt on command-line, and I've verified
168         that Firefox 20, Chrome 26, IE 10, Safari 6, and ToT with all of the JITs agree. Note
169         that some tests cannot be precisely validated because they use Math functions, which
170         ECMAScript chooses to not formally specify - those functions like sin() and friends are
171         allowed to return implementation-dependent results. Also some tests cannot be validated
172         at all because their behavior is either intentionally random or is timezone-dependent.
173         But 23 out of 26 tests now have some kind of validation.
174         
175         I've updated the harnesses to show erroneous runs when displaying results.
176
177         * resources/TEMPLATE.html:
178         * resources/driver-TEMPLATE.html:
179         * resources/sunspider-analyze-results.js:
180         (formatResult):
181         (resultLine):
182         (printOutput):
183         * resources/sunspider-compare-results.js:
184         (.formatMean):
185         (.resultLine):
186         (.printOutput):
187         (sunspiderCompareResults):
188         * resources/sunspider-standalone-driver.js:
189         * tests/sunspider-1.0/3d-cube.js:
190         (Init):
191         * tests/sunspider-1.0/3d-morph.js:
192         * tests/sunspider-1.0/3d-raytrace.js:
193         * tests/sunspider-1.0/access-binary-trees.js:
194         * tests/sunspider-1.0/access-fannkuch.js:
195         * tests/sunspider-1.0/access-nbody.js:
196         * tests/sunspider-1.0/access-nsieve.js:
197         (sieve):
198         * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
199         * tests/sunspider-1.0/bitops-bits-in-byte.js:
200         * tests/sunspider-1.0/bitops-bitwise-and.js:
201         * tests/sunspider-1.0/bitops-nsieve-bits.js:
202         * tests/sunspider-1.0/controlflow-recursive.js:
203         * tests/sunspider-1.0/crypto-aes.js:
204         * tests/sunspider-1.0/crypto-md5.js:
205         * tests/sunspider-1.0/crypto-sha1.js:
206         * tests/sunspider-1.0/date-format-tofte.js:
207         * tests/sunspider-1.0/date-format-xparb.js:
208         * tests/sunspider-1.0/math-cordic.js:
209         * tests/sunspider-1.0/math-partial-sums.js:
210         (partial):
211         * tests/sunspider-1.0/math-spectral-norm.js:
212         * tests/sunspider-1.0/regexp-dna.js:
213         * tests/sunspider-1.0/string-base64.js:
214         * tests/sunspider-1.0/string-fasta.js:
215         (fastaRepeat):
216         (fastaRandom):
217         * tests/sunspider-1.0/string-tagcloud.js:
218         * tests/sunspider-1.0/string-unpack-code.js:
219         * tests/sunspider-1.0/string-validate-input.js:
220
221 2013-04-11  Filip Pizlo  <fpizlo@apple.com>
222
223         Get rid of the 10ms delay between test executions
224         https://bugs.webkit.org/show_bug.cgi?id=114458
225         <rdar://problem/13631112>
226
227         Reviewed by Darin Adler and Geoffrey Garen.
228         
229         More details on this in the bug. Long story short, I don't know why the 10ms delay is
230         there, and it appears to be the root of some noisyness and unreliability in the results.
231         Removing it makes results more consistent, which is a good thing.
232
233         * resources/driver-TEMPLATE.html:
234
235 2013-01-16  Eric Seidel  <eric@webkit.org>
236
237         Remove --shark* support from sunspider/run-sunspider now that Shark is dead
238         https://bugs.webkit.org/show_bug.cgi?id=99512
239
240         Reviewed by Darin Adler.
241
242         I tried to use --instruments, but that also seems broken (in the same way before/after this patch).
243
244         * sunspider:
245         (runTestsOnce):
246
247 2012-08-02  Filip Pizlo  <fpizlo@apple.com>
248
249         Unreviewed, revert debugging stuff I landed in http://trac.webkit.org/changeset/124557
250         by accident.
251
252         * tests/v8-v6/v8-crypto.js:
253         (am1):
254         (am2):
255         (am3):
256         (am4):
257         (bnpCopyTo):
258         (bnpFromString):
259         (bnpClamp):
260
261 2012-03-11  Maciej Stachowiak  <mjs@apple.com>
262
263         Add --instruments option to sunspider to profile with Instruments
264         https://bugs.webkit.org/show_bug.cgi?id=80783
265
266         Reviewed by Filip Pizlo.
267
268         * resources/TimeProfile20us.tracetemplate: Added.
269         * sunspider:
270         (runTestsOnce):
271
272 2011-11-15  Andy Wingo  <wingo@igalia.com>
273
274         Extend sunspider driver to be able to run kraken
275         https://bugs.webkit.org/show_bug.cgi?id=71799
276
277         Reviewed by Filip Pizlo.
278
279         * resources/sunspider-standalone-driver.js: Try to load a -data
280         file.  If that succeeds, we have a kraken-like test, so we time
281         the test using `load'.  Otherwise fall back to using `run'.
282
283 2011-08-11  Mark Rowe  <mrowe@apple.com>
284
285         Fix sunspider-compare-results to default to the same suite version as the sunspider script.
286
287         Rubber-stamped by Maciej Stachowiak.
288
289         * sunspider-compare-results:
290
291 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
292
293         Update SunSpider versioning to 1.0
294         https://bugs.webkit.org/show_bug.cgi?id=63870
295
296         Reviewed by Dan Bernstein.
297
298         * hosted/sunspider.html:
299         * hosted/versions.html:
300         * resources/driver-TEMPLATE.html:
301         * resources/results-TEMPLATE.html:
302         * sunspider:
303
304 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
305
306         Sunspider 0.9.1 harness doesn't actually close() its test documents
307         https://bugs.webkit.org/show_bug.cgi?id=47045
308
309         Reviewed by Daniel Bates.
310
311         * hosted/versions.html:
312         * resources/driver-TEMPLATE.html:
313
314 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
315
316         Not Reviewed. 
317         
318         Fix obviou typo in previous commit.
319
320         * tests/sunspider-1.0/bitops-bitwise-and.js:
321
322 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
323
324         Reviewed by Anders Carlsson.
325
326         SunSpider: all four bitops benchmarks can be replaced with NOP
327         https://bugs.webkit.org/show_bug.cgi?id=38446
328
329         * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
330         (TimeFunc): Save the result.
331         * tests/sunspider-1.0/bitops-bits-in-byte.js:
332         (TimeFunc): Save the result.
333         * tests/sunspider-1.0/bitops-bitwise-and.js: More explicitly
334         save the results.
335         * tests/sunspider-1.0/bitops-nsieve-bits.js:
336         (sieve): Save the result.
337
338 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
339
340         SunSpider: string-validate-input.js uses an undeclared variable named 'name', which is a DOM API
341         https://bugs.webkit.org/show_bug.cgi?id=60937
342
343         Reviewed by Dan Bernstein.
344
345         * tests/sunspider-1.0/string-validate-input.js:
346         (doTest): Rename the variable to username
347
348 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
349
350         Reviewed by Darin Adler.
351
352         SunSpider: The main function of math-cordic is dead code
353         https://bugs.webkit.org/show_bug.cgi?id=63863
354         
355         Try to prevent math-cordic from being eliminated as dead code or
356         compiled down to a constant.
357
358         * tests/sunspider-1.0/math-cordic.js:
359         (cordicsincos):
360         (cordic):
361
362 2011-06-30  Maciej Stachowiak  <mjs@apple.com>
363
364         Reviewed by Adam Barth.
365
366         Create sunspider-1.0 directory in preparation for fixing a bunch of bugs
367         https://bugs.webkit.org/show_bug.cgi?id=63782
368
369         * make-hosted: Learn about the new directory.
370         * tests/sunspider-1.0: Copied from PerformanceTests/SunSpider/tests/sunspider-0.9.1.
371
372 2010-11-29  Geoffrey Garen  <ggaren@apple.com>
373
374         Reviewed by Gavin Barraclough.
375
376         Improved accuracy of command-line SunSpider.
377
378         * resources/sunspider-standalone-driver.js: Use the returned number of
379         elapsed milliseconds from "run" and "checkSyntax" instead of measuring
380         ourselves, for slightly more accurate numbers.
381
382 2010-09-16  Gavin Barraclough  <barraclough@apple.com>
383
384         Reviewed by Mark Rowe.
385         https://bugs.webkit.org/show_bug.cgi?id=45924
386
387         Modify the SunSpider harness to allow a path to be passed to the --suite
388         argument, supress the normal prepend-"tests/" behaviour for suite names
389         containing a '/'.
390
391         Also wrap the code in an anonymous function to move variables out of global
392         scope (currently the harness cannot handle running tests the overwrite the
393         variable 'j').
394
395         * resources/sunspider-standalone-driver.js:
396         * sunspider:
397
398 2010-08-27  Michael Saboff  <msaboff@apple.com>
399
400         Reviewed by Stephanie Lewis.
401
402         Added version 5 and version 6 of V8 tests with modifications similar
403         to the version 4 tests so they'll work with the SunSpider test harness.
404         Note that these files are external source and therefore don't conform 
405         to WebKit coding standards.
406         https://bugs.webkit.org/show_bug.cgi?id=44638
407
408         * tests/v8-v5: Added.
409         * tests/v8-v5/LIST: Added.
410         * tests/v8-v5/v8-crypto.js: Added.
411         * tests/v8-v5/v8-deltablue.js: Added.
412         * tests/v8-v5/v8-earley-boyer.js: Added.
413         * tests/v8-v5/v8-raytrace.js: Added.
414         * tests/v8-v5/v8-regexp.js: Added.
415         * tests/v8-v5/v8-richards.js: Added.
416         * tests/v8-v5/v8-splay.js: Added.
417         * tests/v8-v6: Added.
418         * tests/v8-v6/LIST: Added.
419         * tests/v8-v6/v8-crypto.js: Added.
420         * tests/v8-v6/v8-deltablue.js: Added.
421         * tests/v8-v6/v8-earley-boyer.js: Added.
422         * tests/v8-v6/v8-raytrace.js: Added.
423         * tests/v8-v6/v8-regexp.js: Added.
424         * tests/v8-v6/v8-richards.js: Added.
425         * tests/v8-v6/v8-splay.js: Added.
426
427 2010-07-15  Stephanie Lewis  <slewis@apple.com>
428
429         Reviewed by Geoff Garen.
430
431         http://bugs.webkit.org/show_bug.cgi?id=42406
432         Output location of results file.
433
434         * sunspider:
435
436 2010-05-17  Maciej Stachowiak  <mjs@apple.com>
437
438         Reviewed by Mark Rowe.
439
440         Browser-hosted version of SunSpider runs tests twice as many times as advertised
441         https://bugs.webkit.org/show_bug.cgi?id=39201
442
443         * make-hosted: Make sure to reset test list after every test suite.
444
445 2010-05-07  Adam Roben  <aroben@apple.com>
446
447         Fix typo in sunspider-compare-results.js that caused incorrect
448         two-sample t statistics to be computed
449
450         Fixes <http://webkit.org/b/38768> SunSpider computes incorrect
451         two-sample t statistics
452
453         Reviewed by Darin Adler.
454
455         * resources/sunspider-compare-results.js:
456         (sunspiderCompareResults.resultLine): Fixed the calculation of the t
457         variable to match the definition of a two-sample t statistic. Looks
458         like this was just a simple typo.
459
460 2010-04-05  Darin Adler  <darin@apple.com>
461
462         Updated the ignore property for changes to the contents here.
463
464         * .: Modified property svn:ignore.
465
466 2010-02-20  Maciej Stachowiak  <mjs@apple.com>
467
468         Reviewed by Adam Barth.
469
470         Add sunspider-0.9.1 to the site in preparation for announcing it.
471         https://bugs.webkit.org/show_bug.cgi?id=35206
472
473         * hosted/sunspider.css: Remove tabs, which I noticed where present while
474         copying to the WebKitSite directory.
475
476 2010-01-11  Oliver Hunt  <oliver@apple.com>
477
478         Reviewed by Darin Adler.
479
480         Sunspider parse-only tests don't run
481         https://bugs.webkit.org/show_bug.cgi?id=33489
482
483         Fix parse-only test list
484
485         * tests/parse-only/LIST:
486
487 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
488
489         Reviewed by Adele Peterson.
490
491         Make SunSpider version more prominent in the title
492         https://bugs.webkit.org/show_bug.cgi?id=32574
493
494         * hosted/sunspider.html:
495         * hosted/versions.html:
496         * resources/driver-TEMPLATE.html:
497         * resources/results-TEMPLATE.html:
498
499 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
500
501         Reviewed by Darin Adler.
502
503         Make SunSpider refuse to compare results across content versions
504         https://bugs.webkit.org/show_bug.cgi?id=32573
505         
506         The results URL now includes a version indicator, and compare mode
507         will refuse to compare between versions.
508
509         * resources/driver-TEMPLATE.html:
510         * resources/results-TEMPLATE.html:
511
512 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
513
514         Reviewed by Darin Adler.
515
516         Some Browser-hosted SunSpider files are not valid HTML5
517         https://bugs.webkit.org/show_bug.cgi?id=32536
518         
519         Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
520         close tags and adding a meta charset declaration).
521
522         * hosted/sunspider.html:
523         * hosted/versions.html:
524         * resources/TEMPLATE.html:
525         * resources/driver-TEMPLATE.html:
526         * resources/results-TEMPLATE.html:
527
528 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
529
530         Reviewed by Eric Seidel.
531
532         Make sunspider-0.9.1 the default content set (both command-line and hosted)
533         https://bugs.webkit.org/show_bug.cgi?id=32537
534
535         * hosted/sunspider.html: Change default suite link.
536         * sunspider: Change default suite.
537         * sunspider-compare-results: Likewise.
538
539 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
540
541         Reviewed by Sam Weinig.
542
543         in-browser SunSpider suffers excessive penalty under power management
544         https://bugs.webkit.org/show_bug.cgi?id=32505
545         
546         I have made a few changes to address this:
547         
548         1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
549         document.write to reduce triggering of progress UI.
550         2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
551         stepped all the way down the whole time.
552         3) Run the test cycle an extra time for warmup before the runs that count.
553
554         * hosted/sunspider-record-result.js: Removed.
555         * make-hosted:
556         * resources/TEMPLATE.html:
557         * resources/driver-TEMPLATE.html:
558
559 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
560
561         Reviewed by Gavin Barraclough.
562
563         SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
564         https://bugs.webkit.org/show_bug.cgi?id=16806
565
566         Based on a patch by Eric Seidel.
567         
568         Fix the base64 computation to actually compute correct results. The impact on runtime of
569         the test is pretty small, but noticeable for some browsers. But at least it's not
570         doing a wrong and meaningless computation any more.
571         
572         * tests/sunspider-0.9.1/string-base64.js:
573         ():
574         (base64ToString):
575
576 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
577
578         Fixing commit error...
579         
580         I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
581
582         * resources/driver-TEMPLATE.html:
583
584 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
585
586         Reviewed by Sam Weinig.
587
588         Use JSON.parse instead of eval for Web-hosted SunSpider results processing
589         https://bugs.webkit.org/show_bug.cgi?id=32490
590
591         * hosted/json2.js: Added. Incorporated from json.org
592         * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
593         * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
594         results.
595
596 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
597
598         Reviewed by Oliver Hunt.
599
600         Enable Web-hosted version of SunSpider to handle multiple versions
601         https://bugs.webkit.org/show_bug.cgi?id=32478
602
603         * make-hosted: Now generate the hosted version to be able to run both the
604         0.9 and the 0.9.1 test suites.
605         * hosted: Modified property svn:ignore.
606         * hosted/sunspider-driver.html: Removed.
607         * hosted/sunspider-results.html: Removed.
608         * hosted/sunspider.html:
609         * hosted/versions.html: Added.
610         * resources/TEMPLATE.html:
611         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
612         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
613
614 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
615
616         Reviewed by Oliver Hunt.
617
618         Give command-line sunspider the ability to handle multiple suites and versions
619         https://bugs.webkit.org/show_bug.cgi?id=32477
620         
621         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
622         enumerating the tests. Current suites are:
623         
624         sunspider-0.9 - riginal SunSpider content 
625         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
626         ubench - "ubench" microbenchmark suite
627         v8-v4 - version 4 of the v8 benchmark
628         parse-only - some parser tests
629         
630         Sunspider can now be run with the --suite= parameter to select a
631         suite. The default is --suite=sunspider-0.9.
632
633         * .: Modified property svn:ignore to ignore new results directories.
634         * make-hosted: Generate based on sunspider-0.9 suite.
635         * resources/TEMPLATE.html: Fix an HTML compliance problem.
636         * resources/sunspider-standalone-driver.js: Become aware of suite names.
637         * sunspider: Modified to handle multiple suites as described above.
638         * sunspider-compare-results: ditto
639         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
640         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
641         * tests/LIST-UBENCH: Removed.
642         * tests/LIST-V8: Removed.
643         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
644         * tests/3d-morph.js: ditto
645         * tests/3d-raytrace.js: ditto
646         * tests/access-binary-trees.js: ditto
647         * tests/access-fannkuch.js: ditto
648         * tests/access-nbody.js: ditto
649         * tests/access-nsieve.js: ditto
650         * tests/bitops-3bit-bits-in-byte.js: ditto
651         * tests/bitops-bits-in-byte.js: ditto
652         * tests/bitops-bitwise-and.js: ditto
653         * tests/bitops-nsieve-bits.js: ditto
654         * tests/controlflow-recursive.js: ditto
655         * tests/crypto-aes.js: ditto
656         * tests/crypto-md5.js: ditto
657         * tests/crypto-sha1.js: ditto
658         * tests/date-format-tofte.js: ditto
659         * tests/date-format-xparb.js: ditto
660         * tests/math-cordic.js: ditto
661         * tests/math-partial-sums.js: ditto
662         * tests/math-spectral-norm.js: ditto
663         * tests/regexp-dna.js: ditto
664         * tests/string-base64.js: ditto
665         * tests/string-fasta.js: ditto
666         * tests/string-tagcloud.js: ditto
667         * tests/string-unpack-code.js: ditto
668         * tests/string-validate-input.js: ditto
669         * tests/sunspider-0.9/: Added
670         * tests/sunspider-0.9/*.js: Moved from one level up
671         * tests/sunspider-0.9.1: Added.
672         * tests/sunspider-0.9.1/*.js: Moved from one level up.
673         * tests/v8-crypto.js: Moved to v8-v4/ directory.
674         * tests/v8-deltablue.js: ditto
675         * tests/v8-earley-boyer.js: ditto
676         * tests/v8-raytrace.js: ditto
677         * tests/v8-regexp.js: ditto
678         * tests/v8-richards.js: ditto
679         * tests/v8-splay.js: ditto
680         * tests/v8-v4: Added.
681
682 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
683
684         Reviewed by Darin Adler.
685
686         Platform specific null device replaced with a platform independent.
687         https://bugs.webkit.org/show_bug.cgi?id=29544
688
689         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
690
691 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
692
693         Reviewed by Maciej Stachowiak.
694
695         [Qt] Make sunspider script work on Windows platform.
696         https://bugs.webkit.org/show_bug.cgi?id=29544
697
698         * sunspider:
699         Using platform specific null device instead of /dev/null.
700
701 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
702
703         Reviewed by Oliver Hunt.
704
705         https://bugs.webkit.org/show_bug.cgi?id=26540
706         Currently the SunSpider test driver lacks an option to run a test suite that
707         will test JavaScriptCore parsing performance only.  This patch adds just such
708         a test suite and option to SunSpider as well as the jsc test shell.  I've included
709         three large javascript source files found in the wild: jquery, mootools and prototype.
710         Combined with the concatenation of all three, these form a new testsuite to measure
711         and test pure JavaScriptCore parsing performance.
712
713         * resources/sunspider-standalone-driver.js:
714           Load the js with parse-only flag if it is part of the parse-only suite.
715         * sunspider:
716           Add the parse-only suite to the list of options.
717         * sunspider-compare-results:
718           Add the parse-only suite to the list of options.
719         * tests/LIST-PARSE-ONLY: Added.
720         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
721           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
722         * tests/parse-only/jquery-1.3.2.js: Added.
723           Latest version from jquery.com.
724         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
725           Latest version from mootools.net.
726         * tests/parse-only/prototype-1.6.0.3.js: Added.
727           Latest version from prototypejs.org.
728
729 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
730
731         Update V8 benchmark to version 4.
732         
733         Reviewed by Cameron Zwarich.
734
735         * tests/LIST-V8:
736         * tests/v8-crypto.js:
737         * tests/v8-deltablue.js:
738         * tests/v8-earley-boyer.js:
739         * tests/v8-raytrace.js:
740         * tests/v8-regexp.js: Added.
741         * tests/v8-richards.js:
742         * tests/v8-splay.js: Added.
743
744 2009-05-26  Olivier DOLE  <odole@pleyo.com>
745
746         Reviewed by Maciej.
747
748         Store sunspider test suite results in a specific directory for each
749         different type of benchmark (sunspider, V8 or ubench).
750
751         * sunspider:
752         * sunspider-compare-results:
753
754 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
755
756         Build fix.
757
758         * SunSpider.make:
759
760 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
761
762         Include SunSpider tests in production builds.
763         
764         Reviewed by Adam Roben.
765
766         * SunSpider.make: Added.
767         * xcopy.excludes: Added.
768
769 2009-03-04  Oliver Hunt  <oliver@apple.com>
770
771         Added a few more tests to ubench
772
773         RS=Gavin Barraclough
774
775         Adding a couple of test to cover too few and too many arguments.  This
776         is needed to prevent regressions in all call paths from my current refactoring.
777
778         * tests/LIST-UBENCH:
779         * tests/ubench/function-correct-args.js: Added.
780         (f):
781         * tests/ubench/function-excess-args.js: Added.
782         (f):
783
784 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
785
786         Fix a bug where when V8's or UBENCH's test suite list file is missing
787         the error message is suggesting that SunSpider's list file is missing.
788
789         Reviewed by Sam Weinig.
790
791         * sunspider:
792         (loadTestsList): Fix the error message to use the right list name.
793
794 2008-10-07  David Hyatt  <hyatt@apple.com>
795
796         https://bugs.webkit.org/show_bug.cgi?id=18989
797
798         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
799         we want all browsers to be doing the same thing on the test.
800
801         Reviewed by Maciej
802
803         * tests/regexp-dna.js:
804
805 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
806
807         Rubber-stamped by Maciej Stachowiak.
808
809         The bleeding edge branch of V8 now supports passing fileanmes after
810         '-f', so the SunSpider script should be changed to do this again.
811         This allows all 3 major JavaScript engines to be used with the
812         unmodified SunSpider script.
813
814         * sunspider:
815
816 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
817
818         Reviewed by Maciej Stachowiak.
819
820         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
821         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
822         correct the punctuation in its documentation.
823
824         * sunspider:
825
826 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
827
828         Reviewed by Oliver Hunt.
829
830         Bug 20616: Incorporate V8 benchmarks in testing
831         <https://bugs.webkit.org/show_bug.cgi?id=20616>
832
833         This is a first attempt at running the V8 benchmarks within SunSpider,
834         with the --v8 option. The number of iterations is fixed for each
835         benchmark, and was chosen to make each of the benchmarks run between
836         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
837
838         * sunspider:
839         * tests/LIST-V8: Added.
840         * tests/v8-crypto.js: Added.
841         * tests/v8-deltablue.js: Added.
842         * tests/v8-earley-boyer.js: Added.
843         * tests/v8-raytrace.js: Added.
844         * tests/v8-richards.js: Added.
845
846 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
847
848         Not reviewed.
849
850         Remove a local change that was accidentally committed in r36034.
851
852         * resources/sunspider-standalone-driver.js:
853
854 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
855
856         Rubber-stamped by Oliver Hunt.
857
858         Make SunSpider work with JavaScript shells that don't take filenames
859         after a '-f' argument.
860
861         * sunspider:
862
863 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
864
865         Reviewed by Timothy.
866
867         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
868
869         * sunspider:
870         * tests/LIST-SQUIRRELFISH: Removed.
871
872 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
873
874         Rubber-stamped by Mark Rowe.
875
876         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
877
878         * sunspider:
879
880 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
881
882         Rubber stamped by Oliver.
883         
884         - add newly working tests to squirrelfish list
885
886         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
887
888 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
889
890         Rubber stamped by Oliver.
891         
892         - move ubench tests properly.
893
894         * tests/ubench: Added.
895         * tests/ubench/function-closure.js: Added.
896         * tests/ubench/function-empty.js: Added.
897         * tests/ubench/function-missing-args.js: Added.
898         * tests/ubench/function-sum.js: Added.
899         * tests/ubench/loop-empty-resolve.js: Added.
900         * tests/ubench/loop-empty.js: Added.
901         * tests/ubench/loop-sum.js: Added.
902
903 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
904
905         Rubber stamped by Oliver.
906
907         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
908
909 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
910
911         Reviewed by Oliver.
912
913         - make --squirrelfish mode use the real harness, but a reduced set of tests
914
915         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
916
917         * sunspider:
918         * tests/LIST-SQUIRRELFISH:
919         * tests/LIST-UBENCH: Added.
920         * tests/squirrelfish: Moved to tests/ubench
921
922 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
923
924         Reviewed by Maciej Stachowiak.
925         
926         Removed an unused test.
927
928         * tests/squirrelfish/loop-resolve.js: Removed.
929
930 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
931
932         Reviewed by Oliver.
933         
934         Add newly runnable tests to --squirrelfish mode.
935
936         * tests/LIST-SQUIRRELFISH:
937
938 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
939
940         Reviewed by Maciej Stachowiak.
941         
942         A little more love for --squirrelfish mode:
943         
944         Fixed a misplaced sort. Results properly sort now.
945         
946         Added a test list just for squirrelfish, and updated it to include
947         all currently passing SunSpider tests.
948         
949         Fixed the pruning regexp to match 3d-morph.js.
950
951         * sunspider:
952
953 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
954
955         Reviewed by Oliver Hunt.
956         
957         --squirrelfish mode: pared down tests for squirrelfish to chew on.
958
959         * sunspider:
960         * tests/LIST:
961         * tests/squirrelfish: Added.
962         * tests/squirrelfish/function-closure.js: Added.
963         * tests/squirrelfish/function-empty.js: Added.
964         * tests/squirrelfish/function-missing-args.js: Added.
965         * tests/squirrelfish/function-sum.js: Added.
966         * tests/squirrelfish/loop-empty-resolve.js: Added.
967         * tests/squirrelfish/loop-empty.js: Added.
968         * tests/squirrelfish/loop-resolve.js: Added.
969         * tests/squirrelfish/loop-sum.js: Added.
970
971 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
972
973         Reviewed by Geoff.
974
975         * sunspider: Pause update daemon on Darwin to make results reliable
976         again (workaround for <rdar://problem/5811127>).
977
978 2008-02-07  Eric Seidel  <eric@webkit.org>
979
980         Reviewed by Sam.
981
982         * sunspider: add an error message when ./tests/LIST is not found
983
984 2007-12-29  Alp Toker  <alp@atoker.com>
985
986         Fix typo.
987
988         * hosted/sunspider.html:
989
990 2007-12-29  Alp Toker  <alp@atoker.com>
991
992         Reviewed by Maciej.
993
994         Provide the current URL in the SunSpider results page. Useful for
995         copying benchmark results out of browsers without a location bar or
996         where the location bar can't handle long URLs.
997
998         * hosted/sunspider-results.html:
999
1000 2007-12-20  Eric Seidel  <eric@webkit.org>
1001
1002         Reviewed by mjs.
1003
1004         * sunspider: increase default number of runs for more accurate numbers
1005
1006 2007-12-18  Mark Rowe  <mrowe@apple.com>
1007
1008         Fix typo.
1009
1010         * hosted/sunspider.html:
1011
1012 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
1013
1014         Reviewed by Eric.
1015         
1016         - give an extra digit of precision for small differences.
1017
1018         * resources/sunspider-compare-results.js:
1019
1020 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
1021
1022         Reviewed by Eric.
1023         
1024         - work around IE layout bugs
1025
1026         * hosted/sunspider.css:
1027
1028 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
1029
1030         Reviewed by Mark.
1031         
1032         - fix compare feature
1033
1034         * hosted/sunspider-results.html:
1035
1036 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
1037
1038         Reviewed by Mark.
1039         
1040         - Change the format of the output URL to be shorter, to meet IE constraints
1041
1042         * hosted/sunspider-driver.html:
1043         * hosted/sunspider-results.html:
1044
1045 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
1046
1047         Reviewed by Oliver.
1048         
1049         - Add some explanatory test and a dumb logo to SunSpider web hosted version
1050
1051         * hosted/sunspider-driver.html:
1052         * hosted/sunspider-results.html:
1053         * hosted/sunspider.css:
1054         * hosted/sunspider.html:
1055
1056 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
1057
1058         Reviewed by Oliver.
1059         
1060         - Make the web-hosted version of SunSpider look a little bit less drab.
1061
1062         * hosted/sunspider-driver.html:
1063         * hosted/sunspider-results.html:
1064         * hosted/sunspider.css: Added.
1065         * hosted/sunspider.html:
1066         * resources/TEMPLATE.html:
1067
1068 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
1069
1070         Reviewed by Sam.
1071         
1072         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
1073         
1074         A sample of what new-style results would look like:
1075         
1076         TEST                   COMPARISON            FROM                 TO             DETAILS
1077         ...
1078         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
1079         ...
1080             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
1081         
1082         * resources/sunspider-compare-results.js:
1083
1084 2007-11-14  Eric Seidel  <eric@webkit.org>
1085
1086         Reviewed by Sam.
1087
1088         * sunspider: add --shark-cache for L2 Cache Miss profiling
1089
1090 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
1091
1092         Reviewed by Darin Adler.
1093
1094         http://bugs.webkit.org/show_bug.cgi?id=15925
1095         SunSpider should check for Shark being installed
1096
1097         * sunspider: Explicitly test that shark command line tool is installed.
1098
1099 2007-11-04  Darin Adler  <darin@apple.com>
1100
1101         * hosted/sunspider.html: Tweaked formatting a bit.
1102
1103 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
1104
1105         Reviewed by Darin Adler.
1106
1107         - Add compare mode and a description of the benchmark to the hosted version.
1108
1109         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
1110         * hosted/sunspider.html: Added intro text.
1111         * make-hosted: Copy new file.
1112         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
1113         a function.
1114         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
1115         expects.
1116         * sunspider-compare-results: Use sunspider-standalone-compare.js.
1117
1118 2007-11-02  Darin Adler  <darin@apple.com>
1119
1120         * sunspider: Changed "--base" to "--set-baseline".
1121
1122 2007-11-02  Darin Adler  <darin@apple.com>
1123
1124         Reviewed by Maciej.
1125
1126         * sunspider: Added a "--base" option that records the current test as a baseline.
1127         Moved the code to find the newest .mshark file into a function (since I needed to use
1128         it in the other script and I prefer to copy/paste a function).
1129
1130         * sunspider-compare-results: Added logic to compare the baseline with the last
1131         run when you don't pass any parameters.
1132
1133 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
1134
1135         Reviewed by Eric.
1136         
1137         - Added list of ECMAScript3 features that SunSpider does not cover currently.
1138
1139         * UNCOVERED: Added.
1140
1141 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
1142
1143         Reviewed by Oliver.
1144         
1145         - Give better error messages when sunspider-compare-results fails
1146
1147         * sunspider-compare-results:
1148
1149 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
1150
1151         Reviewed by Darin Adler.
1152         
1153         - fix Date tests to work in command-line SpiderMonkey
1154
1155         * tests/date-format-tofte.js:
1156         * tests/date-format-xparb.js:
1157
1158 2007-10-29  Darin Adler  <darin@apple.com>
1159
1160         Reviewed by Maciej.
1161
1162         * sunspider: Added --shark20 option, to run Shark at its highest sample
1163         resolution instead of its default.
1164
1165 2007-10-28  Adam Roben  <aroben@apple.com>
1166
1167         Fix SunSpider on Windows
1168
1169         Reviewed by Eric.
1170
1171         * sunspider: Don't use colons in filenames.
1172
1173 2007-10-28  Darin Adler  <darin@apple.com>
1174
1175         Reviewed by Adam.
1176
1177         * resources/sunspider-analyze-results.js: Tweak the output format a little.
1178         Change so that when there's only one run we don't write out confidence
1179         intervals at all rather than writing "NaN" over and over again.
1180
1181         * sunspider: Use the time and date as a suffix on the results file, that way
1182         all the results are left behind in the tmp directory. This will make it easy
1183         for us to add features that compare past results in the future. Also rename
1184         the .mshark files using the same scheme.
1185
1186         * sunspider-compare-results: Relax the parsing rules so we can parse the
1187         results file exactly as it's emitted from sunspider.
1188
1189 2007-10-25  Adam Roben  <aroben@apple.com>
1190
1191         Strip carriage returns from results
1192
1193         Carriage returns were screwing up sunspider-compare-results on
1194         Windows.
1195
1196         Rubberstamped by Oliver.
1197
1198         * sunspider:
1199
1200 2007-10-24  Darin Adler  <darin@apple.com>
1201
1202         Reviewed by Eric.
1203
1204         * sunspider: Open the profile in shark after completing a run.
1205         * sunspider-compare-results: Tiny perl tweaks.
1206
1207 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
1208
1209         Reviewed by Darin Adler.
1210
1211         - Don't hardcode my path to testkjs
1212         
1213         * sunspider-compare-results:
1214
1215 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
1216
1217         Reviewed by Darin Adler.
1218         
1219         - Add a compare script to compare results, and improve formatting of standard results a bit.
1220
1221         * resources/sunspider-analyze-results.js:
1222         * resources/sunspider-compare-results.js: Added.
1223         * sunspider-compare-results: Added.
1224
1225 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
1226
1227         Reviewed by Darin Adler.
1228
1229         * resources/sunspider-analyze-results.js: Do the error range
1230         properly, using the t-distribution instead of 1.96 (which was
1231         based on the normal distribution).
1232         
1233         * sunspider: Print results in a way that will be friendlier to the
1234         soon-to-come compare mode.
1235
1236 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
1237
1238         Reviewed by Review.
1239         
1240         - add more tests (probably the final set for now)
1241         
1242         Already balanced. Added date, regexp, control flow, and a few
1243         more string and object/array access tests.
1244         
1245         * tests/LIST:
1246         * tests/access-fannkuch.js: Added.
1247         * tests/access-nbody.js: Added.
1248         * tests/controlflow-recursive.js: Added.
1249         * tests/date-format-tofte.js: Added.
1250         * tests/date-format-xparb.js: Added.
1251         * tests/regexp-dna.js: Added.
1252         * tests/string-validate-input.js: Added.
1253
1254 2007-10-21  Eric Seidel  <eric@webkit.org>
1255
1256         Reviewed by mjs.
1257
1258         * sunspider: make --shark only sample from testkjs
1259
1260 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
1261
1262         Rubber stamped by Eric.
1263         
1264         - added crypto tests (already balanced)
1265
1266         * tests/LIST:
1267         * tests/crypto-aes.js: Added.
1268         * tests/crypto-md5.js: Added.
1269         * tests/crypto-sha1.js: Added.
1270
1271 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
1272
1273         Reviewed by Adam.
1274         
1275         - rebalance new tests
1276         
1277         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
1278         
1279         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1280
1281         This balances them with the existing tests.
1282
1283         * tests/access-binary-trees.js:
1284         * tests/access-nsieve.js:
1285         * tests/bitops-nsieve-bits.js:
1286         * tests/math-partial-sums.js:
1287         * tests/math-spectral-norm.js:
1288         * tests/string-fasta.js:
1289
1290 2007-10-20  Eric Seidel  <eric@webkit.org>
1291
1292         Reviewed by darin.
1293         
1294         Add improved argument handling to sunspider, including
1295         --runs=<number>, --tests=<pattern, --shark, and --help
1296         Also re-factor code into subroutines
1297
1298         * sunspider: improved argument handling
1299
1300 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1301
1302         Reviewed by Mark.
1303
1304         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
1305         
1306         * TODO:
1307         * tests/LIST:
1308         * tests/access-binary-trees.js: Added.
1309         * tests/access-nsieve.js: Added.
1310         * tests/bitops-nsieve-bits.js: Added.
1311         * tests/math-partial-sums.js: Added.
1312         * tests/math-spectral-norm.js: Added.
1313         * tests/string-fasta.js: Added.
1314
1315 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1316
1317         Reviewed by Darin Adler.
1318
1319         * TODO: Updated to note areas that are now well-covered.
1320         
1321         - Change the tests and the driver to avoid leaving large object
1322         graphs hanging around, since that throws off the subsequent tests.
1323         
1324         * resources/sunspider-standalone-driver.js:
1325         * tests/3d-cube.js:
1326         * tests/3d-morph.js:
1327         * tests/3d-raytrace.js:
1328         * tests/bitops-bitwise-and.js:
1329         * tests/string-base64.js:
1330         * tests/string-tagcloud.js:
1331
1332 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1333
1334         Reviewed by Nikolas.
1335         
1336         - Rebalance test complexity.
1337         
1338         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
1339         
1340         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1341
1342         However, for some tests, IE7 was a huge outlier compared to other
1343         major browsers. For those tests, I used Firefox 2.0.0.8 on the
1344         same OS and hardware as a normalization basis.
1345         
1346         * tests/3d-morph.js:
1347         * tests/3d-raytrace.js:
1348         * tests/bitops-3bit-bits-in-byte.js:
1349         * tests/bitops-bits-in-byte.js:
1350         * tests/bitops-bitwise-and.js:
1351         * tests/math-cordic.js:
1352         * tests/string-base64.js:
1353         * tests/string-tagcloud.js:
1354         * tests/string-unpack-code.js:
1355
1356 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1357
1358         Reviewed by Oliver.
1359         
1360         - Fix the browser-hosted driver to be compatible with Safari 2
1361
1362         * hosted/sunspider-driver.html:
1363
1364 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1365
1366         Rubber stamped by Hyatt.
1367         
1368         - Add BSD license notice to cordic test, the original author said to do so.
1369
1370         * tests/math-cordic.js:
1371
1372 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1373
1374         Reviewed by Eric.
1375         
1376         - Support Windows and Windows browsers.
1377
1378         * hosted/sunspider-results.html: Display results correctly in IE.
1379         * sunspider: tweaks to work on Windows, and with spaces in the path.
1380
1381 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1382
1383         Rubber stamped by Oliver.
1384
1385         * tests/string-tagcloud.js: Remove stray print()
1386
1387 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
1388
1389         Reviewed by Oliver.
1390         
1391         Add some string test cases.
1392
1393         * tests/string-base64.js: Added. Does base64 encoding and decoding.
1394         * tests/string-tagcloud.js: Added. Parses a JSON string and
1395         generates tagcloud HTML markup based on the data.        
1396         * tests/string-unpack-code.js: Added. Unpack compressed versions of
1397         four of the most popular JavaScript libraries.
1398         * tests/LIST: Added the new tests.
1399
1400 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
1401
1402         Reviewed by Anders.
1403
1404         * TODO: Updated with info on how to do the stats properly.
1405
1406         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
1407         avoid overly slowing down the benchmark.
1408
1409         Added some 3D-related test cases.
1410         
1411         * tests/3d-cube.js: Added.
1412         * tests/3d-morph.js: Added.
1413         * tests/3d-raytrace.js: Added.
1414         * tests/LIST:
1415
1416 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
1417
1418         Reviewed by Darin Adler.
1419
1420         - New JavaScript benchmark
1421         http://bugs.webkit.org/show_bug.cgi?id=15515
1422         
1423         A start on a new JavaScript benchmark, with standalone and
1424         browser-hosted drivers.
1425         
1426         The test content is still incomplete and the drivers could use
1427         some more features but there is enough here to make a good start.
1428         
1429         The drivers compute a 95% confidence interval on the mean for the
1430         whole test, each category, and each individual test to make it
1431         easier to tell whether differences are statistically
1432         significant. The confidence interval can be narrowed by running
1433         the test more times.
1434         
1435         * TODO: Added. Remaining things that need to be done.
1436         * sunspider: Added. Perl script that acts as the standalone test driver.
1437         * resources/sunspider-standalone-driver.js: Added. JavaScript part
1438         of standalone test driver.
1439         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
1440         analysis code.
1441         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
1442         * make-hosted: Added. Script to generate browser-hosted tests.
1443         * hosted/sunspider.html: Added. Start page for browser-hosted test.
1444         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
1445         * hosted/sunspider-results.html: Added. Results page for browser-hosted
1446         tests.
1447         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
1448         tests.
1449         * tests/LIST: Added. List of tests to use.
1450         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
1451         * tests/bitops-bits-in-byte.js: Added. Ditto.
1452         * tests/bitops-bitwise-and.js: Added. Ditto.
1453         * tests/math-cordic.js: Added. Ditto.
1454