PerformanceTests/SunSpider/ChangeLog

   1 2014-05-21  Eva Balazsfalvi  <evab.u-szeged@partner.samsung.com>
   2
   3         REGRESSION(r169092 and r169102): Skip failing JSC tests on ARM64 properly
   4         https://bugs.webkit.org/show_bug.cgi?id=133149
   5
   6         Reviewed by Csaba Osztrogonác.
   7
   8         * profiler-test.yaml:
   9
  10 2014-05-19  Filip Pizlo  <fpizlo@apple.com>
  11
  12         Take care of some ARM64 test failures
  13         https://bugs.webkit.org/show_bug.cgi?id=133090
  14
  15         Reviewed by Geoffrey Garen.
  16
  17         * profiler-test.yaml: Skip these on not-x86.
  18
  19 2014-03-14  Maciej Stachowiak  <mjs@apple.com>
  20
  21         Replace "Apple Computer, Inc." with "Apple Inc." in copyright headers
  22         https://bugs.webkit.org/show_bug.cgi?id=130276
  23         <rdar://problem/16266927>
  24
  25         Reviewed by Simon Fraser.
  26
  27         * hosted/sunspider.html:
  28         * hosted/versions.html:
  29         * make-hosted:
  30         * resources/TEMPLATE.html:
  31         * resources/driver-TEMPLATE.html:
  32         * resources/results-TEMPLATE.html:
  33         * resources/sunspider-analyze-results.js:
  34         * resources/sunspider-compare-results.js:
  35         * resources/sunspider-standalone-compare.js:
  36         * resources/sunspider-standalone-driver.js:
  37         * sunspider:
  38         * sunspider-compare-results:
  39         * tests/sunspider-0.9.1/3d-morph.js:
  40         * tests/sunspider-0.9.1/3d-raytrace.js:
  41         * tests/sunspider-0.9.1/bitops-bitwise-and.js:
  42         * tests/sunspider-0.9.1/math-cordic.js:
  43         * tests/sunspider-0.9.1/string-tagcloud.js:
  44         * tests/sunspider-0.9/3d-morph.js:
  45         * tests/sunspider-0.9/3d-raytrace.js:
  46         * tests/sunspider-0.9/bitops-bitwise-and.js:
  47         * tests/sunspider-0.9/math-cordic.js:
  48         * tests/sunspider-0.9/string-tagcloud.js:
  49         * tests/sunspider-1.0.1/3d-morph.js:
  50         * tests/sunspider-1.0.1/3d-raytrace.js:
  51         * tests/sunspider-1.0.1/bitops-bitwise-and.js:
  52         * tests/sunspider-1.0.1/math-cordic.js:
  53         * tests/sunspider-1.0.1/string-tagcloud.js:
  54         * tests/sunspider-1.0.2/3d-morph.js:
  55         * tests/sunspider-1.0.2/3d-raytrace.js:
  56         * tests/sunspider-1.0.2/bitops-bitwise-and.js:
  57         * tests/sunspider-1.0.2/math-cordic.js:
  58         * tests/sunspider-1.0.2/string-tagcloud.js:
  59         * tests/sunspider-1.0/3d-morph.js:
  60         * tests/sunspider-1.0/3d-raytrace.js:
  61         * tests/sunspider-1.0/bitops-bitwise-and.js:
  62         * tests/sunspider-1.0/math-cordic.js:
  63         * tests/sunspider-1.0/string-tagcloud.js:
  64
  65 2014-02-10  Filip Pizlo  <fpizlo@apple.com>
  66
  67         Unreviewed, revert the accidental change from r163789.
  68
  69         * tests/v8-v6/v8-deltablue.js:
  70
  71 2013-12-10  Filip Pizlo  <fpizlo@apple.com>
  72
  73         Get rid of forward exit on DoubleAsInt32
  74         https://bugs.webkit.org/show_bug.cgi?id=125552
  75
  76         Reviewed by Oliver Hunt.
  77
  78         Use SunSpider as a kind of spot-check for the
  79         no-architecture-specific-optimization paths in the compiler.
  80
  81         * no-architecture-specific-optimizations.yaml: Added.
  82
  83 2013-10-08  Geoffrey Garen  <ggaren@apple.com>
  84
  85         Refined power management in SunSpider 1.0.2
  86         https://bugs.webkit.org/show_bug.cgi?id=122525
  87
  88         Reviewed by Gavin Barraclough.
  89
  90         * hosted/sunspider.css:
  91         (#frameparent):
  92         (#countdown):
  93         * resources/TEMPLATE.html: Provide an explicit countdown element in the
  94         parent, instead of hacking it into each child frame.
  95
  96         * resources/driver-TEMPLATE.html: Be more explicit about how we respond
  97         to power management. Just moving setTimeout(0) inside the loop is a
  98         little too subtle, and may not work across browsers.
  99
 100         Also, leave a little extra time after changing the countdown graphic,
 101         to give the browser some time to paint.
 102
 103 2013-10-02  Geoffrey Garen  <ggaren@apple.com>
 104
 105         Improved painting behavior in SunSpider 1.0.2
 106         https://bugs.webkit.org/show_bug.cgi?id=122257
 107
 108         Reviewed by Oliver Hunt.
 109
 110         Individual test names with times was a good painting strategy years
 111         back, when each benchmark took hundreds of milliseconds to run. Now,
 112         the whole suite can finish in a a few hundres milliseconds, and things
 113         look pretty flashy.
 114
 115         Instead, display a countdown that indicates which iteration of the
 116         test we're on.
 117
 118         * resources/TEMPLATE.html: Work around a bug in Firefox by specifying
 119         our style inside the .html file instead of in an external stylesheet.
 120         Under heavy JavaScript load, Firefox neglects to apply external
 121         stylesheets -- probably due to a starved timer.
 122
 123         * resources/driver-TEMPLATE.html: Delay a little between runs to ensure
 124         that our countdown has enough time to paint.
 125
 126 2013-09-26  Geoffrey Garen  <ggaren@apple.com>
 127
 128         Fixed a version number string in SunSpider 1.0.2
 129         https://bugs.webkit.org/show_bug.cgi?id=122004
 130
 131         Reviewed by Gavin Barraclough.
 132
 133         * resources/driver-TEMPLATE.html:
 134         * resources/results-TEMPLATE.html: 1.0.2!
 135
 136 2013-09-26  Geoffrey Garen  <ggaren@apple.com>
 137
 138         Tentative SunSpider 1.0.2
 139         https://bugs.webkit.org/show_bug.cgi?id=122002
 140
 141         Reviewed by Gavin Barraclough.
 142
 143         Be even more aggressive about asking for zero delay between tests by
 144         setting a timeout at the beginning of a test instead of the end. This is
 145         another step toward reducing noise -- particularly noise caused by
 146         power management -- just like the step down from 10ms to 0ms.
 147
 148         Browsers typically put a 4ms floor on timers. The sooner we set
 149         our timer, the sooner it will fire.
 150
 151         * make-hosted:
 152         * resources/driver-TEMPLATE.html: Changed the driver to set the timeout
 153         earlier.
 154         * tests/sunspider-1.0.2: Added. These tests are unchanged from 1.0.1.
 155
 156 2013-09-09  Filip Pizlo  <fpizlo@apple.com>
 157
 158         Stress tests should test the jsc profiler (-p)
 159         https://bugs.webkit.org/show_bug.cgi?id=121043
 160
 161         Reviewed by Mark Hahnenberg.
 162
 163         Add a jsc-stress-test that tries to profile SunSpider.
 164
 165         * profiler-test.yaml: Added.
 166
 167 2013-08-08  Cosmin Truta  <ctruta@blackberry.com>
 168
 169         SunSpider: Move the 3d-morph fix from version 1.0 to version 1.0.1
 170         https://bugs.webkit.org/show_bug.cgi?id=117993
 171
 172         Reviewed by George Staikos.
 173
 174         Bump the SunSpider version to 1.0.1.
 175
 176         Unroll r149548 in order to restore version 1.0 to its original,
 177         and add the 3d-morph fix to version 1.0.1 instead.
 178
 179         * hosted/sunspider.html: Updated to version 1.0.1.
 180         * hosted/versions.html: Updated to version 1.0.1.
 181         * make-hosted: Added support for version 1.0.1.
 182         * resources/driver-TEMPLATE.html: Updated to version 1.0.1.
 183         * resources/results-TEMPLATE.html: Updated to version 1.0.1.
 184         * sunspider: Updated to version 1.0.1.
 185         * sunspider-compare-results: Updated to version 1.0.1.
 186         * tests/sunspider-1.0/3d-morph.js: Rolled back to the original version.
 187         * tests/sunspider-1.0.1/LIST: Added.
 188         * tests/sunspider-1.0.1/3d-cube.js: Added.
 189         * tests/sunspider-1.0.1/3d-morph.js: Added.
 190         * tests/sunspider-1.0.1/3d-raytrace.js: Added.
 191         * tests/sunspider-1.0.1/access-binary-trees.js: Added.
 192         * tests/sunspider-1.0.1/access-fannkuch.js: Added.
 193         * tests/sunspider-1.0.1/access-nbody.js: Added.
 194         * tests/sunspider-1.0.1/access-nsieve.js: Added.
 195         * tests/sunspider-1.0.1/bitops-3bit-bits-in-byte.js: Added.
 196         * tests/sunspider-1.0.1/bitops-bits-in-byte.js: Added.
 197         * tests/sunspider-1.0.1/bitops-bitwise-and.js: Added.
 198         * tests/sunspider-1.0.1/bitops-nsieve-bits.js: Added.
 199         * tests/sunspider-1.0.1/controlflow-recursive.js: Added.
 200         * tests/sunspider-1.0.1/crypto-aes.js: Added.
 201         * tests/sunspider-1.0.1/crypto-md5.js: Added.
 202         * tests/sunspider-1.0.1/crypto-sha1.js: Added.
 203         * tests/sunspider-1.0.1/date-format-tofte.js: Added.
 204         * tests/sunspider-1.0.1/date-format-xparb.js: Added.
 205         * tests/sunspider-1.0.1/math-cordic.js: Added.
 206         * tests/sunspider-1.0.1/math-partial-sums.js: Added.
 207         * tests/sunspider-1.0.1/math-spectral-norm.js: Added.
 208         * tests/sunspider-1.0.1/regexp-dna.js: Added.
 209         * tests/sunspider-1.0.1/string-base64.js: Added.
 210         * tests/sunspider-1.0.1/string-fasta.js: Added.
 211         * tests/sunspider-1.0.1/string-tagcloud.js: Added.
 212         * tests/sunspider-1.0.1/string-unpack-code.js: Added.
 213         * tests/sunspider-1.0.1/string-validate-input.js: Added.
 214
 215 2013-05-03  Cosmin Truta  <ctruta@blackberry.com>
 216
 217         SunSpider 1.0: 3d-morph: use epsilon to check result
 218         https://bugs.webkit.org/show_bug.cgi?id=115553
 219
 220         Reviewed by George Staikos.
 221
 222         SunSpider/3d-morph should be more lenient in validating its results, since they depend on Math.sin, which isn't formally specified.
 223
 224         * tests/sunspider-1.0/3d-morph.js:
 225
 226 2013-04-19  Filip Pizlo  <fpizlo@apple.com>
 227
 228         SunSpider/string-tagcloud should be more lenient in validating its results, since they depend on Math.log, which isn't formally specified
 229         https://bugs.webkit.org/show_bug.cgi?id=114895
 230
 231         Reviewed by Michael Saboff.
 232
 233         * tests/sunspider-1.0/string-tagcloud.js:
 234
 235 2013-04-18  Filip Pizlo  <fpizlo@apple.com>
 236
 237         Whenever it is cheap and non-invasive, SunSpider tests should validate their results to ensure that the browser runs them correctly
 238         https://bugs.webkit.org/show_bug.cgi?id=114852
 239
 240         Reviewed by Geoffrey Garen.
 241
 242         This adds low-overhead checking of the results of each SunSpider tests. This is not
 243         meant to be an exhaustive test that everything was executed correctly, but rather a
 244         simple sanity check that will catch glaring mistakes. The philosophy here is that we're
 245         not trying to prevent all forms of cheating, or that we're trying to prove the
 246         browser's correctness. Moreover, these checks are meant to incur minimal overhead;
 247         they currently clock in at <2% slow-down in SunSpider 1.0.
 248
 249         The test expectations were generated using the LLInt on command-line, and I've verified
 250         that Firefox 20, Chrome 26, IE 10, Safari 6, and ToT with all of the JITs agree. Note
 251         that some tests cannot be precisely validated because they use Math functions, which
 252         ECMAScript chooses to not formally specify - those functions like sin() and friends are
 253         allowed to return implementation-dependent results. Also some tests cannot be validated
 254         at all because their behavior is either intentionally random or is timezone-dependent.
 255         But 23 out of 26 tests now have some kind of validation.
 256
 257         I've updated the harnesses to show erroneous runs when displaying results.
 258
 259         * resources/TEMPLATE.html:
 260         * resources/driver-TEMPLATE.html:
 261         * resources/sunspider-analyze-results.js:
 262         (formatResult):
 263         (resultLine):
 264         (printOutput):
 265         * resources/sunspider-compare-results.js:
 266         (.formatMean):
 267         (.resultLine):
 268         (.printOutput):
 269         (sunspiderCompareResults):
 270         * resources/sunspider-standalone-driver.js:
 271         * tests/sunspider-1.0/3d-cube.js:
 272         (Init):
 273         * tests/sunspider-1.0/3d-morph.js:
 274         * tests/sunspider-1.0/3d-raytrace.js:
 275         * tests/sunspider-1.0/access-binary-trees.js:
 276         * tests/sunspider-1.0/access-fannkuch.js:
 277         * tests/sunspider-1.0/access-nbody.js:
 278         * tests/sunspider-1.0/access-nsieve.js:
 279         (sieve):
 280         * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
 281         * tests/sunspider-1.0/bitops-bits-in-byte.js:
 282         * tests/sunspider-1.0/bitops-bitwise-and.js:
 283         * tests/sunspider-1.0/bitops-nsieve-bits.js:
 284         * tests/sunspider-1.0/controlflow-recursive.js:
 285         * tests/sunspider-1.0/crypto-aes.js:
 286         * tests/sunspider-1.0/crypto-md5.js:
 287         * tests/sunspider-1.0/crypto-sha1.js:
 288         * tests/sunspider-1.0/date-format-tofte.js:
 289         * tests/sunspider-1.0/date-format-xparb.js:
 290         * tests/sunspider-1.0/math-cordic.js:
 291         * tests/sunspider-1.0/math-partial-sums.js:
 292         (partial):
 293         * tests/sunspider-1.0/math-spectral-norm.js:
 294         * tests/sunspider-1.0/regexp-dna.js:
 295         * tests/sunspider-1.0/string-base64.js:
 296         * tests/sunspider-1.0/string-fasta.js:
 297         (fastaRepeat):
 298         (fastaRandom):
 299         * tests/sunspider-1.0/string-tagcloud.js:
 300         * tests/sunspider-1.0/string-unpack-code.js:
 301         * tests/sunspider-1.0/string-validate-input.js:
 302
 303 2013-04-11  Filip Pizlo  <fpizlo@apple.com>
 304
 305         Get rid of the 10ms delay between test executions
 306         https://bugs.webkit.org/show_bug.cgi?id=114458
 307         <rdar://problem/13631112>
 308
 309         Reviewed by Darin Adler and Geoffrey Garen.
 310
 311         More details on this in the bug. Long story short, I don't know why the 10ms delay is
 312         there, and it appears to be the root of some noisyness and unreliability in the results.
 313         Removing it makes results more consistent, which is a good thing.
 314
 315         * resources/driver-TEMPLATE.html:
 316
 317 2013-01-16  Eric Seidel  <eric@webkit.org>
 318
 319         Remove --shark* support from sunspider/run-sunspider now that Shark is dead
 320         https://bugs.webkit.org/show_bug.cgi?id=99512
 321
 322         Reviewed by Darin Adler.
 323
 324         I tried to use --instruments, but that also seems broken (in the same way before/after this patch).
 325
 326         * sunspider:
 327         (runTestsOnce):
 328
 329 2012-08-02  Filip Pizlo  <fpizlo@apple.com>
 330
 331         Unreviewed, revert debugging stuff I landed in http://trac.webkit.org/changeset/124557
 332         by accident.
 333
 334         * tests/v8-v6/v8-crypto.js:
 335         (am1):
 336         (am2):
 337         (am3):
 338         (am4):
 339         (bnpCopyTo):
 340         (bnpFromString):
 341         (bnpClamp):
 342
 343 2012-03-11  Maciej Stachowiak  <mjs@apple.com>
 344
 345         Add --instruments option to sunspider to profile with Instruments
 346         https://bugs.webkit.org/show_bug.cgi?id=80783
 347
 348         Reviewed by Filip Pizlo.
 349
 350         * resources/TimeProfile20us.tracetemplate: Added.
 351         * sunspider:
 352         (runTestsOnce):
 353
 354 2011-11-15  Andy Wingo  <wingo@igalia.com>
 355
 356         Extend sunspider driver to be able to run kraken
 357         https://bugs.webkit.org/show_bug.cgi?id=71799
 358
 359         Reviewed by Filip Pizlo.
 360
 361         * resources/sunspider-standalone-driver.js: Try to load a -data
 362         file.  If that succeeds, we have a kraken-like test, so we time
 363         the test using `load'.  Otherwise fall back to using `run'.
 364
 365 2011-08-11  Mark Rowe  <mrowe@apple.com>
 366
 367         Fix sunspider-compare-results to default to the same suite version as the sunspider script.
 368
 369         Rubber-stamped by Maciej Stachowiak.
 370
 371         * sunspider-compare-results:
 372
 373 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
 374
 375         Update SunSpider versioning to 1.0
 376         https://bugs.webkit.org/show_bug.cgi?id=63870
 377
 378         Reviewed by Dan Bernstein.
 379
 380         * hosted/sunspider.html:
 381         * hosted/versions.html:
 382         * resources/driver-TEMPLATE.html:
 383         * resources/results-TEMPLATE.html:
 384         * sunspider:
 385
 386 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
 387
 388         Sunspider 0.9.1 harness doesn't actually close() its test documents
 389         https://bugs.webkit.org/show_bug.cgi?id=47045
 390
 391         Reviewed by Daniel Bates.
 392
 393         * hosted/versions.html:
 394         * resources/driver-TEMPLATE.html:
 395
 396 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
 397
 398         Not Reviewed.
 399
 400         Fix obviou typo in previous commit.
 401
 402         * tests/sunspider-1.0/bitops-bitwise-and.js:
 403
 404 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
 405
 406         Reviewed by Anders Carlsson.
 407
 408         SunSpider: all four bitops benchmarks can be replaced with NOP
 409         https://bugs.webkit.org/show_bug.cgi?id=38446
 410
 411         * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
 412         (TimeFunc): Save the result.
 413         * tests/sunspider-1.0/bitops-bits-in-byte.js:
 414         (TimeFunc): Save the result.
 415         * tests/sunspider-1.0/bitops-bitwise-and.js: More explicitly
 416         save the results.
 417         * tests/sunspider-1.0/bitops-nsieve-bits.js:
 418         (sieve): Save the result.
 419
 420 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
 421
 422         SunSpider: string-validate-input.js uses an undeclared variable named 'name', which is a DOM API
 423         https://bugs.webkit.org/show_bug.cgi?id=60937
 424
 425         Reviewed by Dan Bernstein.
 426
 427         * tests/sunspider-1.0/string-validate-input.js:
 428         (doTest): Rename the variable to username
 429
 430 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
 431
 432         Reviewed by Darin Adler.
 433
 434         SunSpider: The main function of math-cordic is dead code
 435         https://bugs.webkit.org/show_bug.cgi?id=63863
 436
 437         Try to prevent math-cordic from being eliminated as dead code or
 438         compiled down to a constant.
 439
 440         * tests/sunspider-1.0/math-cordic.js:
 441         (cordicsincos):
 442         (cordic):
 443
 444 2011-06-30  Maciej Stachowiak  <mjs@apple.com>
 445
 446         Reviewed by Adam Barth.
 447
 448         Create sunspider-1.0 directory in preparation for fixing a bunch of bugs
 449         https://bugs.webkit.org/show_bug.cgi?id=63782
 450
 451         * make-hosted: Learn about the new directory.
 452         * tests/sunspider-1.0: Copied from PerformanceTests/SunSpider/tests/sunspider-0.9.1.
 453
 454 2010-11-29  Geoffrey Garen  <ggaren@apple.com>
 455
 456         Reviewed by Gavin Barraclough.
 457
 458         Improved accuracy of command-line SunSpider.
 459
 460         * resources/sunspider-standalone-driver.js: Use the returned number of
 461         elapsed milliseconds from "run" and "checkSyntax" instead of measuring
 462         ourselves, for slightly more accurate numbers.
 463
 464 2010-09-16  Gavin Barraclough  <barraclough@apple.com>
 465
 466         Reviewed by Mark Rowe.
 467         https://bugs.webkit.org/show_bug.cgi?id=45924
 468
 469         Modify the SunSpider harness to allow a path to be passed to the --suite
 470         argument, supress the normal prepend-"tests/" behaviour for suite names
 471         containing a '/'.
 472
 473         Also wrap the code in an anonymous function to move variables out of global
 474         scope (currently the harness cannot handle running tests the overwrite the
 475         variable 'j').
 476
 477         * resources/sunspider-standalone-driver.js:
 478         * sunspider:
 479
 480 2010-08-27  Michael Saboff  <msaboff@apple.com>
 481
 482         Reviewed by Stephanie Lewis.
 483
 484         Added version 5 and version 6 of V8 tests with modifications similar
 485         to the version 4 tests so they'll work with the SunSpider test harness.
 486         Note that these files are external source and therefore don't conform
 487         to WebKit coding standards.
 488         https://bugs.webkit.org/show_bug.cgi?id=44638
 489
 490         * tests/v8-v5: Added.
 491         * tests/v8-v5/LIST: Added.
 492         * tests/v8-v5/v8-crypto.js: Added.
 493         * tests/v8-v5/v8-deltablue.js: Added.
 494         * tests/v8-v5/v8-earley-boyer.js: Added.
 495         * tests/v8-v5/v8-raytrace.js: Added.
 496         * tests/v8-v5/v8-regexp.js: Added.
 497         * tests/v8-v5/v8-richards.js: Added.
 498         * tests/v8-v5/v8-splay.js: Added.
 499         * tests/v8-v6: Added.
 500         * tests/v8-v6/LIST: Added.
 501         * tests/v8-v6/v8-crypto.js: Added.
 502         * tests/v8-v6/v8-deltablue.js: Added.
 503         * tests/v8-v6/v8-earley-boyer.js: Added.
 504         * tests/v8-v6/v8-raytrace.js: Added.
 505         * tests/v8-v6/v8-regexp.js: Added.
 506         * tests/v8-v6/v8-richards.js: Added.
 507         * tests/v8-v6/v8-splay.js: Added.
 508
 509 2010-07-15  Stephanie Lewis  <slewis@apple.com>
 510
 511         Reviewed by Geoff Garen.
 512
 513         http://bugs.webkit.org/show_bug.cgi?id=42406
 514         Output location of results file.
 515
 516         * sunspider:
 517
 518 2010-05-17  Maciej Stachowiak  <mjs@apple.com>
 519
 520         Reviewed by Mark Rowe.
 521
 522         Browser-hosted version of SunSpider runs tests twice as many times as advertised
 523         https://bugs.webkit.org/show_bug.cgi?id=39201
 524
 525         * make-hosted: Make sure to reset test list after every test suite.
 526
 527 2010-05-07  Adam Roben  <aroben@apple.com>
 528
 529         Fix typo in sunspider-compare-results.js that caused incorrect
 530         two-sample t statistics to be computed
 531
 532         Fixes <http://webkit.org/b/38768> SunSpider computes incorrect
 533         two-sample t statistics
 534
 535         Reviewed by Darin Adler.
 536
 537         * resources/sunspider-compare-results.js:
 538         (sunspiderCompareResults.resultLine): Fixed the calculation of the t
 539         variable to match the definition of a two-sample t statistic. Looks
 540         like this was just a simple typo.
 541
 542 2010-04-05  Darin Adler  <darin@apple.com>
 543
 544         Updated the ignore property for changes to the contents here.
 545
 546         * .: Modified property svn:ignore.
 547
 548 2010-02-20  Maciej Stachowiak  <mjs@apple.com>
 549
 550         Reviewed by Adam Barth.
 551
 552         Add sunspider-0.9.1 to the site in preparation for announcing it.
 553         https://bugs.webkit.org/show_bug.cgi?id=35206
 554
 555         * hosted/sunspider.css: Remove tabs, which I noticed where present while
 556         copying to the WebKitSite directory.
 557
 558 2010-01-11  Oliver Hunt  <oliver@apple.com>
 559
 560         Reviewed by Darin Adler.
 561
 562         Sunspider parse-only tests don't run
 563         https://bugs.webkit.org/show_bug.cgi?id=33489
 564
 565         Fix parse-only test list
 566
 567         * tests/parse-only/LIST:
 568
 569 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
 570
 571         Reviewed by Adele Peterson.
 572
 573         Make SunSpider version more prominent in the title
 574         https://bugs.webkit.org/show_bug.cgi?id=32574
 575
 576         * hosted/sunspider.html:
 577         * hosted/versions.html:
 578         * resources/driver-TEMPLATE.html:
 579         * resources/results-TEMPLATE.html:
 580
 581 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
 582
 583         Reviewed by Darin Adler.
 584
 585         Make SunSpider refuse to compare results across content versions
 586         https://bugs.webkit.org/show_bug.cgi?id=32573
 587
 588         The results URL now includes a version indicator, and compare mode
 589         will refuse to compare between versions.
 590
 591         * resources/driver-TEMPLATE.html:
 592         * resources/results-TEMPLATE.html:
 593
 594 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
 595
 596         Reviewed by Darin Adler.
 597
 598         Some Browser-hosted SunSpider files are not valid HTML5
 599         https://bugs.webkit.org/show_bug.cgi?id=32536
 600
 601         Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
 602         close tags and adding a meta charset declaration).
 603
 604         * hosted/sunspider.html:
 605         * hosted/versions.html:
 606         * resources/TEMPLATE.html:
 607         * resources/driver-TEMPLATE.html:
 608         * resources/results-TEMPLATE.html:
 609
 610 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
 611
 612         Reviewed by Eric Seidel.
 613
 614         Make sunspider-0.9.1 the default content set (both command-line and hosted)
 615         https://bugs.webkit.org/show_bug.cgi?id=32537
 616
 617         * hosted/sunspider.html: Change default suite link.
 618         * sunspider: Change default suite.
 619         * sunspider-compare-results: Likewise.
 620
 621 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
 622
 623         Reviewed by Sam Weinig.
 624
 625         in-browser SunSpider suffers excessive penalty under power management
 626         https://bugs.webkit.org/show_bug.cgi?id=32505
 627
 628         I have made a few changes to address this:
 629
 630         1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
 631         document.write to reduce triggering of progress UI.
 632         2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
 633         stepped all the way down the whole time.
 634         3) Run the test cycle an extra time for warmup before the runs that count.
 635
 636         * hosted/sunspider-record-result.js: Removed.
 637         * make-hosted:
 638         * resources/TEMPLATE.html:
 639         * resources/driver-TEMPLATE.html:
 640
 641 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
 642
 643         Reviewed by Gavin Barraclough.
 644
 645         SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
 646         https://bugs.webkit.org/show_bug.cgi?id=16806
 647
 648         Based on a patch by Eric Seidel.
 649
 650         Fix the base64 computation to actually compute correct results. The impact on runtime of
 651         the test is pretty small, but noticeable for some browsers. But at least it's not
 652         doing a wrong and meaningless computation any more.
 653
 654         * tests/sunspider-0.9.1/string-base64.js:
 655         ():
 656         (base64ToString):
 657
 658 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
 659
 660         Fixing commit error...
 661
 662         I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
 663
 664         * resources/driver-TEMPLATE.html:
 665
 666 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
 667
 668         Reviewed by Sam Weinig.
 669
 670         Use JSON.parse instead of eval for Web-hosted SunSpider results processing
 671         https://bugs.webkit.org/show_bug.cgi?id=32490
 672
 673         * hosted/json2.js: Added. Incorporated from json.org
 674         * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
 675         * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
 676         results.
 677
 678 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
 679
 680         Reviewed by Oliver Hunt.
 681
 682         Enable Web-hosted version of SunSpider to handle multiple versions
 683         https://bugs.webkit.org/show_bug.cgi?id=32478
 684
 685         * make-hosted: Now generate the hosted version to be able to run both the
 686         0.9 and the 0.9.1 test suites.
 687         * hosted: Modified property svn:ignore.
 688         * hosted/sunspider-driver.html: Removed.
 689         * hosted/sunspider-results.html: Removed.
 690         * hosted/sunspider.html:
 691         * hosted/versions.html: Added.
 692         * resources/TEMPLATE.html:
 693         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
 694         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
 695
 696 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
 697
 698         Reviewed by Oliver Hunt.
 699
 700         Give command-line sunspider the ability to handle multiple suites and versions
 701         https://bugs.webkit.org/show_bug.cgi?id=32477
 702
 703         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
 704         enumerating the tests. Current suites are:
 705
 706         sunspider-0.9 - riginal SunSpider content
 707         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
 708         ubench - "ubench" microbenchmark suite
 709         v8-v4 - version 4 of the v8 benchmark
 710         parse-only - some parser tests
 711
 712         Sunspider can now be run with the --suite= parameter to select a
 713         suite. The default is --suite=sunspider-0.9.
 714
 715         * .: Modified property svn:ignore to ignore new results directories.
 716         * make-hosted: Generate based on sunspider-0.9 suite.
 717         * resources/TEMPLATE.html: Fix an HTML compliance problem.
 718         * resources/sunspider-standalone-driver.js: Become aware of suite names.
 719         * sunspider: Modified to handle multiple suites as described above.
 720         * sunspider-compare-results: ditto
 721         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
 722         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
 723         * tests/LIST-UBENCH: Removed.
 724         * tests/LIST-V8: Removed.
 725         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
 726         * tests/3d-morph.js: ditto
 727         * tests/3d-raytrace.js: ditto
 728         * tests/access-binary-trees.js: ditto
 729         * tests/access-fannkuch.js: ditto
 730         * tests/access-nbody.js: ditto
 731         * tests/access-nsieve.js: ditto
 732         * tests/bitops-3bit-bits-in-byte.js: ditto
 733         * tests/bitops-bits-in-byte.js: ditto
 734         * tests/bitops-bitwise-and.js: ditto
 735         * tests/bitops-nsieve-bits.js: ditto
 736         * tests/controlflow-recursive.js: ditto
 737         * tests/crypto-aes.js: ditto
 738         * tests/crypto-md5.js: ditto
 739         * tests/crypto-sha1.js: ditto
 740         * tests/date-format-tofte.js: ditto
 741         * tests/date-format-xparb.js: ditto
 742         * tests/math-cordic.js: ditto
 743         * tests/math-partial-sums.js: ditto
 744         * tests/math-spectral-norm.js: ditto
 745         * tests/regexp-dna.js: ditto
 746         * tests/string-base64.js: ditto
 747         * tests/string-fasta.js: ditto
 748         * tests/string-tagcloud.js: ditto
 749         * tests/string-unpack-code.js: ditto
 750         * tests/string-validate-input.js: ditto
 751         * tests/sunspider-0.9/: Added
 752         * tests/sunspider-0.9/*.js: Moved from one level up
 753         * tests/sunspider-0.9.1: Added.
 754         * tests/sunspider-0.9.1/*.js: Moved from one level up.
 755         * tests/v8-crypto.js: Moved to v8-v4/ directory.
 756         * tests/v8-deltablue.js: ditto
 757         * tests/v8-earley-boyer.js: ditto
 758         * tests/v8-raytrace.js: ditto
 759         * tests/v8-regexp.js: ditto
 760         * tests/v8-richards.js: ditto
 761         * tests/v8-splay.js: ditto
 762         * tests/v8-v4: Added.
 763
 764 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
 765
 766         Reviewed by Darin Adler.
 767
 768         Platform specific null device replaced with a platform independent.
 769         https://bugs.webkit.org/show_bug.cgi?id=29544
 770
 771         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
 772
 773 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
 774
 775         Reviewed by Maciej Stachowiak.
 776
 777         [Qt] Make sunspider script work on Windows platform.
 778         https://bugs.webkit.org/show_bug.cgi?id=29544
 779
 780         * sunspider:
 781         Using platform specific null device instead of /dev/null.
 782
 783 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
 784
 785         Reviewed by Oliver Hunt.
 786
 787         https://bugs.webkit.org/show_bug.cgi?id=26540
 788         Currently the SunSpider test driver lacks an option to run a test suite that
 789         will test JavaScriptCore parsing performance only.  This patch adds just such
 790         a test suite and option to SunSpider as well as the jsc test shell.  I've included
 791         three large javascript source files found in the wild: jquery, mootools and prototype.
 792         Combined with the concatenation of all three, these form a new testsuite to measure
 793         and test pure JavaScriptCore parsing performance.
 794
 795         * resources/sunspider-standalone-driver.js:
 796           Load the js with parse-only flag if it is part of the parse-only suite.
 797         * sunspider:
 798           Add the parse-only suite to the list of options.
 799         * sunspider-compare-results:
 800           Add the parse-only suite to the list of options.
 801         * tests/LIST-PARSE-ONLY: Added.
 802         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
 803           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
 804         * tests/parse-only/jquery-1.3.2.js: Added.
 805           Latest version from jquery.com.
 806         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
 807           Latest version from mootools.net.
 808         * tests/parse-only/prototype-1.6.0.3.js: Added.
 809           Latest version from prototypejs.org.
 810
 811 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
 812
 813         Update V8 benchmark to version 4.
 814
 815         Reviewed by Cameron Zwarich.
 816
 817         * tests/LIST-V8:
 818         * tests/v8-crypto.js:
 819         * tests/v8-deltablue.js:
 820         * tests/v8-earley-boyer.js:
 821         * tests/v8-raytrace.js:
 822         * tests/v8-regexp.js: Added.
 823         * tests/v8-richards.js:
 824         * tests/v8-splay.js: Added.
 825
 826 2009-05-26  Olivier DOLE  <odole@pleyo.com>
 827
 828         Reviewed by Maciej.
 829
 830         Store sunspider test suite results in a specific directory for each
 831         different type of benchmark (sunspider, V8 or ubench).
 832
 833         * sunspider:
 834         * sunspider-compare-results:
 835
 836 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
 837
 838         Build fix.
 839
 840         * SunSpider.make:
 841
 842 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
 843
 844         Include SunSpider tests in production builds.
 845
 846         Reviewed by Adam Roben.
 847
 848         * SunSpider.make: Added.
 849         * xcopy.excludes: Added.
 850
 851 2009-03-04  Oliver Hunt  <oliver@apple.com>
 852
 853         Added a few more tests to ubench
 854
 855         RS=Gavin Barraclough
 856
 857         Adding a couple of test to cover too few and too many arguments.  This
 858         is needed to prevent regressions in all call paths from my current refactoring.
 859
 860         * tests/LIST-UBENCH:
 861         * tests/ubench/function-correct-args.js: Added.
 862         (f):
 863         * tests/ubench/function-excess-args.js: Added.
 864         (f):
 865
 866 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
 867
 868         Fix a bug where when V8's or UBENCH's test suite list file is missing
 869         the error message is suggesting that SunSpider's list file is missing.
 870
 871         Reviewed by Sam Weinig.
 872
 873         * sunspider:
 874         (loadTestsList): Fix the error message to use the right list name.
 875
 876 2008-10-07  David Hyatt  <hyatt@apple.com>
 877
 878         https://bugs.webkit.org/show_bug.cgi?id=18989
 879
 880         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
 881         we want all browsers to be doing the same thing on the test.
 882
 883         Reviewed by Maciej
 884
 885         * tests/regexp-dna.js:
 886
 887 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 888
 889         Rubber-stamped by Maciej Stachowiak.
 890
 891         The bleeding edge branch of V8 now supports passing fileanmes after
 892         '-f', so the SunSpider script should be changed to do this again.
 893         This allows all 3 major JavaScript engines to be used with the
 894         unmodified SunSpider script.
 895
 896         * sunspider:
 897
 898 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 899
 900         Reviewed by Maciej Stachowiak.
 901
 902         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
 903         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
 904         correct the punctuation in its documentation.
 905
 906         * sunspider:
 907
 908 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 909
 910         Reviewed by Oliver Hunt.
 911
 912         Bug 20616: Incorporate V8 benchmarks in testing
 913         <https://bugs.webkit.org/show_bug.cgi?id=20616>
 914
 915         This is a first attempt at running the V8 benchmarks within SunSpider,
 916         with the --v8 option. The number of iterations is fixed for each
 917         benchmark, and was chosen to make each of the benchmarks run between
 918         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
 919
 920         * sunspider:
 921         * tests/LIST-V8: Added.
 922         * tests/v8-crypto.js: Added.
 923         * tests/v8-deltablue.js: Added.
 924         * tests/v8-earley-boyer.js: Added.
 925         * tests/v8-raytrace.js: Added.
 926         * tests/v8-richards.js: Added.
 927
 928 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 929
 930         Not reviewed.
 931
 932         Remove a local change that was accidentally committed in r36034.
 933
 934         * resources/sunspider-standalone-driver.js:
 935
 936 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 937
 938         Rubber-stamped by Oliver Hunt.
 939
 940         Make SunSpider work with JavaScript shells that don't take filenames
 941         after a '-f' argument.
 942
 943         * sunspider:
 944
 945 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 946
 947         Reviewed by Timothy.
 948
 949         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
 950
 951         * sunspider:
 952         * tests/LIST-SQUIRRELFISH: Removed.
 953
 954 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
 955
 956         Rubber-stamped by Mark Rowe.
 957
 958         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
 959
 960         * sunspider:
 961
 962 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
 963
 964         Rubber stamped by Oliver.
 965
 966         - add newly working tests to squirrelfish list
 967
 968         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
 969
 970 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
 971
 972         Rubber stamped by Oliver.
 973
 974         - move ubench tests properly.
 975
 976         * tests/ubench: Added.
 977         * tests/ubench/function-closure.js: Added.
 978         * tests/ubench/function-empty.js: Added.
 979         * tests/ubench/function-missing-args.js: Added.
 980         * tests/ubench/function-sum.js: Added.
 981         * tests/ubench/loop-empty-resolve.js: Added.
 982         * tests/ubench/loop-empty.js: Added.
 983         * tests/ubench/loop-sum.js: Added.
 984
 985 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
 986
 987         Rubber stamped by Oliver.
 988
 989         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
 990
 991 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
 992
 993         Reviewed by Oliver.
 994
 995         - make --squirrelfish mode use the real harness, but a reduced set of tests
 996
 997         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
 998
 999         * sunspider:
1000         * tests/LIST-SQUIRRELFISH:
1001         * tests/LIST-UBENCH: Added.
1002         * tests/squirrelfish: Moved to tests/ubench
1003
1004 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
1005
1006         Reviewed by Maciej Stachowiak.
1007
1008         Removed an unused test.
1009
1010         * tests/squirrelfish/loop-resolve.js: Removed.
1011
1012 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
1013
1014         Reviewed by Oliver.
1015
1016         Add newly runnable tests to --squirrelfish mode.
1017
1018         * tests/LIST-SQUIRRELFISH:
1019
1020 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
1021
1022         Reviewed by Maciej Stachowiak.
1023
1024         A little more love for --squirrelfish mode:
1025
1026         Fixed a misplaced sort. Results properly sort now.
1027
1028         Added a test list just for squirrelfish, and updated it to include
1029         all currently passing SunSpider tests.
1030
1031         Fixed the pruning regexp to match 3d-morph.js.
1032
1033         * sunspider:
1034
1035 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
1036
1037         Reviewed by Oliver Hunt.
1038
1039         --squirrelfish mode: pared down tests for squirrelfish to chew on.
1040
1041         * sunspider:
1042         * tests/LIST:
1043         * tests/squirrelfish: Added.
1044         * tests/squirrelfish/function-closure.js: Added.
1045         * tests/squirrelfish/function-empty.js: Added.
1046         * tests/squirrelfish/function-missing-args.js: Added.
1047         * tests/squirrelfish/function-sum.js: Added.
1048         * tests/squirrelfish/loop-empty-resolve.js: Added.
1049         * tests/squirrelfish/loop-empty.js: Added.
1050         * tests/squirrelfish/loop-resolve.js: Added.
1051         * tests/squirrelfish/loop-sum.js: Added.
1052
1053 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
1054
1055         Reviewed by Geoff.
1056
1057         * sunspider: Pause update daemon on Darwin to make results reliable
1058         again (workaround for <rdar://problem/5811127>).
1059
1060 2008-02-07  Eric Seidel  <eric@webkit.org>
1061
1062         Reviewed by Sam.
1063
1064         * sunspider: add an error message when ./tests/LIST is not found
1065
1066 2007-12-29  Alp Toker  <alp@atoker.com>
1067
1068         Fix typo.
1069
1070         * hosted/sunspider.html:
1071
1072 2007-12-29  Alp Toker  <alp@atoker.com>
1073
1074         Reviewed by Maciej.
1075
1076         Provide the current URL in the SunSpider results page. Useful for
1077         copying benchmark results out of browsers without a location bar or
1078         where the location bar can't handle long URLs.
1079
1080         * hosted/sunspider-results.html:
1081
1082 2007-12-20  Eric Seidel  <eric@webkit.org>
1083
1084         Reviewed by mjs.
1085
1086         * sunspider: increase default number of runs for more accurate numbers
1087
1088 2007-12-18  Mark Rowe  <mrowe@apple.com>
1089
1090         Fix typo.
1091
1092         * hosted/sunspider.html:
1093
1094 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
1095
1096         Reviewed by Eric.
1097
1098         - give an extra digit of precision for small differences.
1099
1100         * resources/sunspider-compare-results.js:
1101
1102 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
1103
1104         Reviewed by Eric.
1105
1106         - work around IE layout bugs
1107
1108         * hosted/sunspider.css:
1109
1110 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
1111
1112         Reviewed by Mark.
1113
1114         - fix compare feature
1115
1116         * hosted/sunspider-results.html:
1117
1118 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
1119
1120         Reviewed by Mark.
1121
1122         - Change the format of the output URL to be shorter, to meet IE constraints
1123
1124         * hosted/sunspider-driver.html:
1125         * hosted/sunspider-results.html:
1126
1127 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
1128
1129         Reviewed by Oliver.
1130
1131         - Add some explanatory test and a dumb logo to SunSpider web hosted version
1132
1133         * hosted/sunspider-driver.html:
1134         * hosted/sunspider-results.html:
1135         * hosted/sunspider.css:
1136         * hosted/sunspider.html:
1137
1138 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
1139
1140         Reviewed by Oliver.
1141
1142         - Make the web-hosted version of SunSpider look a little bit less drab.
1143
1144         * hosted/sunspider-driver.html:
1145         * hosted/sunspider-results.html:
1146         * hosted/sunspider.css: Added.
1147         * hosted/sunspider.html:
1148         * resources/TEMPLATE.html:
1149
1150 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
1151
1152         Reviewed by Sam.
1153
1154         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
1155
1156         A sample of what new-style results would look like:
1157
1158         TEST                   COMPARISON            FROM                 TO             DETAILS
1159         ...
1160         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
1161         ...
1162             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
1163
1164         * resources/sunspider-compare-results.js:
1165
1166 2007-11-14  Eric Seidel  <eric@webkit.org>
1167
1168         Reviewed by Sam.
1169
1170         * sunspider: add --shark-cache for L2 Cache Miss profiling
1171
1172 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
1173
1174         Reviewed by Darin Adler.
1175
1176         http://bugs.webkit.org/show_bug.cgi?id=15925
1177         SunSpider should check for Shark being installed
1178
1179         * sunspider: Explicitly test that shark command line tool is installed.
1180
1181 2007-11-04  Darin Adler  <darin@apple.com>
1182
1183         * hosted/sunspider.html: Tweaked formatting a bit.
1184
1185 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
1186
1187         Reviewed by Darin Adler.
1188
1189         - Add compare mode and a description of the benchmark to the hosted version.
1190
1191         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
1192         * hosted/sunspider.html: Added intro text.
1193         * make-hosted: Copy new file.
1194         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
1195         a function.
1196         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
1197         expects.
1198         * sunspider-compare-results: Use sunspider-standalone-compare.js.
1199
1200 2007-11-02  Darin Adler  <darin@apple.com>
1201
1202         * sunspider: Changed "--base" to "--set-baseline".
1203
1204 2007-11-02  Darin Adler  <darin@apple.com>
1205
1206         Reviewed by Maciej.
1207
1208         * sunspider: Added a "--base" option that records the current test as a baseline.
1209         Moved the code to find the newest .mshark file into a function (since I needed to use
1210         it in the other script and I prefer to copy/paste a function).
1211
1212         * sunspider-compare-results: Added logic to compare the baseline with the last
1213         run when you don't pass any parameters.
1214
1215 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
1216
1217         Reviewed by Eric.
1218
1219         - Added list of ECMAScript3 features that SunSpider does not cover currently.
1220
1221         * UNCOVERED: Added.
1222
1223 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
1224
1225         Reviewed by Oliver.
1226
1227         - Give better error messages when sunspider-compare-results fails
1228
1229         * sunspider-compare-results:
1230
1231 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
1232
1233         Reviewed by Darin Adler.
1234
1235         - fix Date tests to work in command-line SpiderMonkey
1236
1237         * tests/date-format-tofte.js:
1238         * tests/date-format-xparb.js:
1239
1240 2007-10-29  Darin Adler  <darin@apple.com>
1241
1242         Reviewed by Maciej.
1243
1244         * sunspider: Added --shark20 option, to run Shark at its highest sample
1245         resolution instead of its default.
1246
1247 2007-10-28  Adam Roben  <aroben@apple.com>
1248
1249         Fix SunSpider on Windows
1250
1251         Reviewed by Eric.
1252
1253         * sunspider: Don't use colons in filenames.
1254
1255 2007-10-28  Darin Adler  <darin@apple.com>
1256
1257         Reviewed by Adam.
1258
1259         * resources/sunspider-analyze-results.js: Tweak the output format a little.
1260         Change so that when there's only one run we don't write out confidence
1261         intervals at all rather than writing "NaN" over and over again.
1262
1263         * sunspider: Use the time and date as a suffix on the results file, that way
1264         all the results are left behind in the tmp directory. This will make it easy
1265         for us to add features that compare past results in the future. Also rename
1266         the .mshark files using the same scheme.
1267
1268         * sunspider-compare-results: Relax the parsing rules so we can parse the
1269         results file exactly as it's emitted from sunspider.
1270
1271 2007-10-25  Adam Roben  <aroben@apple.com>
1272
1273         Strip carriage returns from results
1274
1275         Carriage returns were screwing up sunspider-compare-results on
1276         Windows.
1277
1278         Rubberstamped by Oliver.
1279
1280         * sunspider:
1281
1282 2007-10-24  Darin Adler  <darin@apple.com>
1283
1284         Reviewed by Eric.
1285
1286         * sunspider: Open the profile in shark after completing a run.
1287         * sunspider-compare-results: Tiny perl tweaks.
1288
1289 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
1290
1291         Reviewed by Darin Adler.
1292
1293         - Don't hardcode my path to testkjs
1294
1295         * sunspider-compare-results:
1296
1297 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
1298
1299         Reviewed by Darin Adler.
1300
1301         - Add a compare script to compare results, and improve formatting of standard results a bit.
1302
1303         * resources/sunspider-analyze-results.js:
1304         * resources/sunspider-compare-results.js: Added.
1305         * sunspider-compare-results: Added.
1306
1307 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
1308
1309         Reviewed by Darin Adler.
1310
1311         * resources/sunspider-analyze-results.js: Do the error range
1312         properly, using the t-distribution instead of 1.96 (which was
1313         based on the normal distribution).
1314
1315         * sunspider: Print results in a way that will be friendlier to the
1316         soon-to-come compare mode.
1317
1318 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
1319
1320         Reviewed by Review.
1321
1322         - add more tests (probably the final set for now)
1323
1324         Already balanced. Added date, regexp, control flow, and a few
1325         more string and object/array access tests.
1326
1327         * tests/LIST:
1328         * tests/access-fannkuch.js: Added.
1329         * tests/access-nbody.js: Added.
1330         * tests/controlflow-recursive.js: Added.
1331         * tests/date-format-tofte.js: Added.
1332         * tests/date-format-xparb.js: Added.
1333         * tests/regexp-dna.js: Added.
1334         * tests/string-validate-input.js: Added.
1335
1336 2007-10-21  Eric Seidel  <eric@webkit.org>
1337
1338         Reviewed by mjs.
1339
1340         * sunspider: make --shark only sample from testkjs
1341
1342 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
1343
1344         Rubber stamped by Eric.
1345
1346         - added crypto tests (already balanced)
1347
1348         * tests/LIST:
1349         * tests/crypto-aes.js: Added.
1350         * tests/crypto-md5.js: Added.
1351         * tests/crypto-sha1.js: Added.
1352
1353 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
1354
1355         Reviewed by Adam.
1356
1357         - rebalance new tests
1358
1359         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
1360
1361         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1362
1363         This balances them with the existing tests.
1364
1365         * tests/access-binary-trees.js:
1366         * tests/access-nsieve.js:
1367         * tests/bitops-nsieve-bits.js:
1368         * tests/math-partial-sums.js:
1369         * tests/math-spectral-norm.js:
1370         * tests/string-fasta.js:
1371
1372 2007-10-20  Eric Seidel  <eric@webkit.org>
1373
1374         Reviewed by darin.
1375
1376         Add improved argument handling to sunspider, including
1377         --runs=<number>, --tests=<pattern, --shark, and --help
1378         Also re-factor code into subroutines
1379
1380         * sunspider: improved argument handling
1381
1382 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1383
1384         Reviewed by Mark.
1385
1386         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
1387
1388         * TODO:
1389         * tests/LIST:
1390         * tests/access-binary-trees.js: Added.
1391         * tests/access-nsieve.js: Added.
1392         * tests/bitops-nsieve-bits.js: Added.
1393         * tests/math-partial-sums.js: Added.
1394         * tests/math-spectral-norm.js: Added.
1395         * tests/string-fasta.js: Added.
1396
1397 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1398
1399         Reviewed by Darin Adler.
1400
1401         * TODO: Updated to note areas that are now well-covered.
1402
1403         - Change the tests and the driver to avoid leaving large object
1404         graphs hanging around, since that throws off the subsequent tests.
1405
1406         * resources/sunspider-standalone-driver.js:
1407         * tests/3d-cube.js:
1408         * tests/3d-morph.js:
1409         * tests/3d-raytrace.js:
1410         * tests/bitops-bitwise-and.js:
1411         * tests/string-base64.js:
1412         * tests/string-tagcloud.js:
1413
1414 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1415
1416         Reviewed by Nikolas.
1417
1418         - Rebalance test complexity.
1419
1420         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
1421
1422         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1423
1424         However, for some tests, IE7 was a huge outlier compared to other
1425         major browsers. For those tests, I used Firefox 2.0.0.8 on the
1426         same OS and hardware as a normalization basis.
1427
1428         * tests/3d-morph.js:
1429         * tests/3d-raytrace.js:
1430         * tests/bitops-3bit-bits-in-byte.js:
1431         * tests/bitops-bits-in-byte.js:
1432         * tests/bitops-bitwise-and.js:
1433         * tests/math-cordic.js:
1434         * tests/string-base64.js:
1435         * tests/string-tagcloud.js:
1436         * tests/string-unpack-code.js:
1437
1438 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1439
1440         Reviewed by Oliver.
1441
1442         - Fix the browser-hosted driver to be compatible with Safari 2
1443
1444         * hosted/sunspider-driver.html:
1445
1446 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1447
1448         Rubber stamped by Hyatt.
1449
1450         - Add BSD license notice to cordic test, the original author said to do so.
1451
1452         * tests/math-cordic.js:
1453
1454 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1455
1456         Reviewed by Eric.
1457
1458         - Support Windows and Windows browsers.
1459
1460         * hosted/sunspider-results.html: Display results correctly in IE.
1461         * sunspider: tweaks to work on Windows, and with spaces in the path.
1462
1463 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1464
1465         Rubber stamped by Oliver.
1466
1467         * tests/string-tagcloud.js: Remove stray print()
1468
1469 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
1470
1471         Reviewed by Oliver.
1472
1473         Add some string test cases.
1474
1475         * tests/string-base64.js: Added. Does base64 encoding and decoding.
1476         * tests/string-tagcloud.js: Added. Parses a JSON string and
1477         generates tagcloud HTML markup based on the data.
1478         * tests/string-unpack-code.js: Added. Unpack compressed versions of
1479         four of the most popular JavaScript libraries.
1480         * tests/LIST: Added the new tests.
1481
1482 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
1483
1484         Reviewed by Anders.
1485
1486         * TODO: Updated with info on how to do the stats properly.
1487
1488         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
1489         avoid overly slowing down the benchmark.
1490
1491         Added some 3D-related test cases.
1492
1493         * tests/3d-cube.js: Added.
1494         * tests/3d-morph.js: Added.
1495         * tests/3d-raytrace.js: Added.
1496         * tests/LIST:
1497
1498 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
1499
1500         Reviewed by Darin Adler.
1501
1502         - New JavaScript benchmark
1503         http://bugs.webkit.org/show_bug.cgi?id=15515
1504
1505         A start on a new JavaScript benchmark, with standalone and
1506         browser-hosted drivers.
1507
1508         The test content is still incomplete and the drivers could use
1509         some more features but there is enough here to make a good start.
1510
1511         The drivers compute a 95% confidence interval on the mean for the
1512         whole test, each category, and each individual test to make it
1513         easier to tell whether differences are statistically
1514         significant. The confidence interval can be narrowed by running
1515         the test more times.
1516
1517         * TODO: Added. Remaining things that need to be done.
1518         * sunspider: Added. Perl script that acts as the standalone test driver.
1519         * resources/sunspider-standalone-driver.js: Added. JavaScript part
1520         of standalone test driver.
1521         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
1522         analysis code.
1523         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
1524         * make-hosted: Added. Script to generate browser-hosted tests.
1525         * hosted/sunspider.html: Added. Start page for browser-hosted test.
1526         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
1527         * hosted/sunspider-results.html: Added. Results page for browser-hosted
1528         tests.
1529         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
1530         tests.
1531         * tests/LIST: Added. List of tests to use.
1532         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
1533         * tests/bitops-bits-in-byte.js: Added. Ditto.
1534         * tests/bitops-bitwise-and.js: Added. Ditto.
1535         * tests/math-cordic.js: Added. Ditto.
1536