PerformanceTests/SunSpider/ChangeLog

   1 2014-05-19  Filip Pizlo  <fpizlo@apple.com>
   2
   3         Take care of some ARM64 test failures
   4         https://bugs.webkit.org/show_bug.cgi?id=133090
   5
   6         Reviewed by Geoffrey Garen.
   7
   8         * profiler-test.yaml: Skip these on not-x86.
   9
  10 2014-03-14  Maciej Stachowiak  <mjs@apple.com>
  11
  12         Replace "Apple Computer, Inc." with "Apple Inc." in copyright headers
  13         https://bugs.webkit.org/show_bug.cgi?id=130276
  14         <rdar://problem/16266927>
  15
  16         Reviewed by Simon Fraser.
  17
  18         * hosted/sunspider.html:
  19         * hosted/versions.html:
  20         * make-hosted:
  21         * resources/TEMPLATE.html:
  22         * resources/driver-TEMPLATE.html:
  23         * resources/results-TEMPLATE.html:
  24         * resources/sunspider-analyze-results.js:
  25         * resources/sunspider-compare-results.js:
  26         * resources/sunspider-standalone-compare.js:
  27         * resources/sunspider-standalone-driver.js:
  28         * sunspider:
  29         * sunspider-compare-results:
  30         * tests/sunspider-0.9.1/3d-morph.js:
  31         * tests/sunspider-0.9.1/3d-raytrace.js:
  32         * tests/sunspider-0.9.1/bitops-bitwise-and.js:
  33         * tests/sunspider-0.9.1/math-cordic.js:
  34         * tests/sunspider-0.9.1/string-tagcloud.js:
  35         * tests/sunspider-0.9/3d-morph.js:
  36         * tests/sunspider-0.9/3d-raytrace.js:
  37         * tests/sunspider-0.9/bitops-bitwise-and.js:
  38         * tests/sunspider-0.9/math-cordic.js:
  39         * tests/sunspider-0.9/string-tagcloud.js:
  40         * tests/sunspider-1.0.1/3d-morph.js:
  41         * tests/sunspider-1.0.1/3d-raytrace.js:
  42         * tests/sunspider-1.0.1/bitops-bitwise-and.js:
  43         * tests/sunspider-1.0.1/math-cordic.js:
  44         * tests/sunspider-1.0.1/string-tagcloud.js:
  45         * tests/sunspider-1.0.2/3d-morph.js:
  46         * tests/sunspider-1.0.2/3d-raytrace.js:
  47         * tests/sunspider-1.0.2/bitops-bitwise-and.js:
  48         * tests/sunspider-1.0.2/math-cordic.js:
  49         * tests/sunspider-1.0.2/string-tagcloud.js:
  50         * tests/sunspider-1.0/3d-morph.js:
  51         * tests/sunspider-1.0/3d-raytrace.js:
  52         * tests/sunspider-1.0/bitops-bitwise-and.js:
  53         * tests/sunspider-1.0/math-cordic.js:
  54         * tests/sunspider-1.0/string-tagcloud.js:
  55
  56 2014-02-10  Filip Pizlo  <fpizlo@apple.com>
  57
  58         Unreviewed, revert the accidental change from r163789.
  59
  60         * tests/v8-v6/v8-deltablue.js:
  61
  62 2013-12-10  Filip Pizlo  <fpizlo@apple.com>
  63
  64         Get rid of forward exit on DoubleAsInt32
  65         https://bugs.webkit.org/show_bug.cgi?id=125552
  66
  67         Reviewed by Oliver Hunt.
  68
  69         Use SunSpider as a kind of spot-check for the
  70         no-architecture-specific-optimization paths in the compiler.
  71
  72         * no-architecture-specific-optimizations.yaml: Added.
  73
  74 2013-10-08  Geoffrey Garen  <ggaren@apple.com>
  75
  76         Refined power management in SunSpider 1.0.2
  77         https://bugs.webkit.org/show_bug.cgi?id=122525
  78
  79         Reviewed by Gavin Barraclough.
  80
  81         * hosted/sunspider.css:
  82         (#frameparent):
  83         (#countdown):
  84         * resources/TEMPLATE.html: Provide an explicit countdown element in the
  85         parent, instead of hacking it into each child frame.
  86
  87         * resources/driver-TEMPLATE.html: Be more explicit about how we respond
  88         to power management. Just moving setTimeout(0) inside the loop is a
  89         little too subtle, and may not work across browsers.
  90
  91         Also, leave a little extra time after changing the countdown graphic,
  92         to give the browser some time to paint.
  93
  94 2013-10-02  Geoffrey Garen  <ggaren@apple.com>
  95
  96         Improved painting behavior in SunSpider 1.0.2
  97         https://bugs.webkit.org/show_bug.cgi?id=122257
  98
  99         Reviewed by Oliver Hunt.
 100
 101         Individual test names with times was a good painting strategy years
 102         back, when each benchmark took hundreds of milliseconds to run. Now,
 103         the whole suite can finish in a a few hundres milliseconds, and things
 104         look pretty flashy.
 105
 106         Instead, display a countdown that indicates which iteration of the
 107         test we're on.
 108
 109         * resources/TEMPLATE.html: Work around a bug in Firefox by specifying
 110         our style inside the .html file instead of in an external stylesheet.
 111         Under heavy JavaScript load, Firefox neglects to apply external
 112         stylesheets -- probably due to a starved timer.
 113
 114         * resources/driver-TEMPLATE.html: Delay a little between runs to ensure
 115         that our countdown has enough time to paint.
 116
 117 2013-09-26  Geoffrey Garen  <ggaren@apple.com>
 118
 119         Fixed a version number string in SunSpider 1.0.2
 120         https://bugs.webkit.org/show_bug.cgi?id=122004
 121
 122         Reviewed by Gavin Barraclough.
 123
 124         * resources/driver-TEMPLATE.html:
 125         * resources/results-TEMPLATE.html: 1.0.2!
 126
 127 2013-09-26  Geoffrey Garen  <ggaren@apple.com>
 128
 129         Tentative SunSpider 1.0.2
 130         https://bugs.webkit.org/show_bug.cgi?id=122002
 131
 132         Reviewed by Gavin Barraclough.
 133
 134         Be even more aggressive about asking for zero delay between tests by
 135         setting a timeout at the beginning of a test instead of the end. This is
 136         another step toward reducing noise -- particularly noise caused by
 137         power management -- just like the step down from 10ms to 0ms.
 138
 139         Browsers typically put a 4ms floor on timers. The sooner we set
 140         our timer, the sooner it will fire.
 141
 142         * make-hosted:
 143         * resources/driver-TEMPLATE.html: Changed the driver to set the timeout
 144         earlier.
 145         * tests/sunspider-1.0.2: Added. These tests are unchanged from 1.0.1.
 146
 147 2013-09-09  Filip Pizlo  <fpizlo@apple.com>
 148
 149         Stress tests should test the jsc profiler (-p)
 150         https://bugs.webkit.org/show_bug.cgi?id=121043
 151
 152         Reviewed by Mark Hahnenberg.
 153
 154         Add a jsc-stress-test that tries to profile SunSpider.
 155
 156         * profiler-test.yaml: Added.
 157
 158 2013-08-08  Cosmin Truta  <ctruta@blackberry.com>
 159
 160         SunSpider: Move the 3d-morph fix from version 1.0 to version 1.0.1
 161         https://bugs.webkit.org/show_bug.cgi?id=117993
 162
 163         Reviewed by George Staikos.
 164
 165         Bump the SunSpider version to 1.0.1.
 166
 167         Unroll r149548 in order to restore version 1.0 to its original,
 168         and add the 3d-morph fix to version 1.0.1 instead.
 169
 170         * hosted/sunspider.html: Updated to version 1.0.1.
 171         * hosted/versions.html: Updated to version 1.0.1.
 172         * make-hosted: Added support for version 1.0.1.
 173         * resources/driver-TEMPLATE.html: Updated to version 1.0.1.
 174         * resources/results-TEMPLATE.html: Updated to version 1.0.1.
 175         * sunspider: Updated to version 1.0.1.
 176         * sunspider-compare-results: Updated to version 1.0.1.
 177         * tests/sunspider-1.0/3d-morph.js: Rolled back to the original version.
 178         * tests/sunspider-1.0.1/LIST: Added.
 179         * tests/sunspider-1.0.1/3d-cube.js: Added.
 180         * tests/sunspider-1.0.1/3d-morph.js: Added.
 181         * tests/sunspider-1.0.1/3d-raytrace.js: Added.
 182         * tests/sunspider-1.0.1/access-binary-trees.js: Added.
 183         * tests/sunspider-1.0.1/access-fannkuch.js: Added.
 184         * tests/sunspider-1.0.1/access-nbody.js: Added.
 185         * tests/sunspider-1.0.1/access-nsieve.js: Added.
 186         * tests/sunspider-1.0.1/bitops-3bit-bits-in-byte.js: Added.
 187         * tests/sunspider-1.0.1/bitops-bits-in-byte.js: Added.
 188         * tests/sunspider-1.0.1/bitops-bitwise-and.js: Added.
 189         * tests/sunspider-1.0.1/bitops-nsieve-bits.js: Added.
 190         * tests/sunspider-1.0.1/controlflow-recursive.js: Added.
 191         * tests/sunspider-1.0.1/crypto-aes.js: Added.
 192         * tests/sunspider-1.0.1/crypto-md5.js: Added.
 193         * tests/sunspider-1.0.1/crypto-sha1.js: Added.
 194         * tests/sunspider-1.0.1/date-format-tofte.js: Added.
 195         * tests/sunspider-1.0.1/date-format-xparb.js: Added.
 196         * tests/sunspider-1.0.1/math-cordic.js: Added.
 197         * tests/sunspider-1.0.1/math-partial-sums.js: Added.
 198         * tests/sunspider-1.0.1/math-spectral-norm.js: Added.
 199         * tests/sunspider-1.0.1/regexp-dna.js: Added.
 200         * tests/sunspider-1.0.1/string-base64.js: Added.
 201         * tests/sunspider-1.0.1/string-fasta.js: Added.
 202         * tests/sunspider-1.0.1/string-tagcloud.js: Added.
 203         * tests/sunspider-1.0.1/string-unpack-code.js: Added.
 204         * tests/sunspider-1.0.1/string-validate-input.js: Added.
 205
 206 2013-05-03  Cosmin Truta  <ctruta@blackberry.com>
 207
 208         SunSpider 1.0: 3d-morph: use epsilon to check result
 209         https://bugs.webkit.org/show_bug.cgi?id=115553
 210
 211         Reviewed by George Staikos.
 212
 213         SunSpider/3d-morph should be more lenient in validating its results, since they depend on Math.sin, which isn't formally specified.
 214
 215         * tests/sunspider-1.0/3d-morph.js:
 216
 217 2013-04-19  Filip Pizlo  <fpizlo@apple.com>
 218
 219         SunSpider/string-tagcloud should be more lenient in validating its results, since they depend on Math.log, which isn't formally specified
 220         https://bugs.webkit.org/show_bug.cgi?id=114895
 221
 222         Reviewed by Michael Saboff.
 223
 224         * tests/sunspider-1.0/string-tagcloud.js:
 225
 226 2013-04-18  Filip Pizlo  <fpizlo@apple.com>
 227
 228         Whenever it is cheap and non-invasive, SunSpider tests should validate their results to ensure that the browser runs them correctly
 229         https://bugs.webkit.org/show_bug.cgi?id=114852
 230
 231         Reviewed by Geoffrey Garen.
 232
 233         This adds low-overhead checking of the results of each SunSpider tests. This is not
 234         meant to be an exhaustive test that everything was executed correctly, but rather a
 235         simple sanity check that will catch glaring mistakes. The philosophy here is that we're
 236         not trying to prevent all forms of cheating, or that we're trying to prove the
 237         browser's correctness. Moreover, these checks are meant to incur minimal overhead;
 238         they currently clock in at <2% slow-down in SunSpider 1.0.
 239
 240         The test expectations were generated using the LLInt on command-line, and I've verified
 241         that Firefox 20, Chrome 26, IE 10, Safari 6, and ToT with all of the JITs agree. Note
 242         that some tests cannot be precisely validated because they use Math functions, which
 243         ECMAScript chooses to not formally specify - those functions like sin() and friends are
 244         allowed to return implementation-dependent results. Also some tests cannot be validated
 245         at all because their behavior is either intentionally random or is timezone-dependent.
 246         But 23 out of 26 tests now have some kind of validation.
 247
 248         I've updated the harnesses to show erroneous runs when displaying results.
 249
 250         * resources/TEMPLATE.html:
 251         * resources/driver-TEMPLATE.html:
 252         * resources/sunspider-analyze-results.js:
 253         (formatResult):
 254         (resultLine):
 255         (printOutput):
 256         * resources/sunspider-compare-results.js:
 257         (.formatMean):
 258         (.resultLine):
 259         (.printOutput):
 260         (sunspiderCompareResults):
 261         * resources/sunspider-standalone-driver.js:
 262         * tests/sunspider-1.0/3d-cube.js:
 263         (Init):
 264         * tests/sunspider-1.0/3d-morph.js:
 265         * tests/sunspider-1.0/3d-raytrace.js:
 266         * tests/sunspider-1.0/access-binary-trees.js:
 267         * tests/sunspider-1.0/access-fannkuch.js:
 268         * tests/sunspider-1.0/access-nbody.js:
 269         * tests/sunspider-1.0/access-nsieve.js:
 270         (sieve):
 271         * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
 272         * tests/sunspider-1.0/bitops-bits-in-byte.js:
 273         * tests/sunspider-1.0/bitops-bitwise-and.js:
 274         * tests/sunspider-1.0/bitops-nsieve-bits.js:
 275         * tests/sunspider-1.0/controlflow-recursive.js:
 276         * tests/sunspider-1.0/crypto-aes.js:
 277         * tests/sunspider-1.0/crypto-md5.js:
 278         * tests/sunspider-1.0/crypto-sha1.js:
 279         * tests/sunspider-1.0/date-format-tofte.js:
 280         * tests/sunspider-1.0/date-format-xparb.js:
 281         * tests/sunspider-1.0/math-cordic.js:
 282         * tests/sunspider-1.0/math-partial-sums.js:
 283         (partial):
 284         * tests/sunspider-1.0/math-spectral-norm.js:
 285         * tests/sunspider-1.0/regexp-dna.js:
 286         * tests/sunspider-1.0/string-base64.js:
 287         * tests/sunspider-1.0/string-fasta.js:
 288         (fastaRepeat):
 289         (fastaRandom):
 290         * tests/sunspider-1.0/string-tagcloud.js:
 291         * tests/sunspider-1.0/string-unpack-code.js:
 292         * tests/sunspider-1.0/string-validate-input.js:
 293
 294 2013-04-11  Filip Pizlo  <fpizlo@apple.com>
 295
 296         Get rid of the 10ms delay between test executions
 297         https://bugs.webkit.org/show_bug.cgi?id=114458
 298         <rdar://problem/13631112>
 299
 300         Reviewed by Darin Adler and Geoffrey Garen.
 301
 302         More details on this in the bug. Long story short, I don't know why the 10ms delay is
 303         there, and it appears to be the root of some noisyness and unreliability in the results.
 304         Removing it makes results more consistent, which is a good thing.
 305
 306         * resources/driver-TEMPLATE.html:
 307
 308 2013-01-16  Eric Seidel  <eric@webkit.org>
 309
 310         Remove --shark* support from sunspider/run-sunspider now that Shark is dead
 311         https://bugs.webkit.org/show_bug.cgi?id=99512
 312
 313         Reviewed by Darin Adler.
 314
 315         I tried to use --instruments, but that also seems broken (in the same way before/after this patch).
 316
 317         * sunspider:
 318         (runTestsOnce):
 319
 320 2012-08-02  Filip Pizlo  <fpizlo@apple.com>
 321
 322         Unreviewed, revert debugging stuff I landed in http://trac.webkit.org/changeset/124557
 323         by accident.
 324
 325         * tests/v8-v6/v8-crypto.js:
 326         (am1):
 327         (am2):
 328         (am3):
 329         (am4):
 330         (bnpCopyTo):
 331         (bnpFromString):
 332         (bnpClamp):
 333
 334 2012-03-11  Maciej Stachowiak  <mjs@apple.com>
 335
 336         Add --instruments option to sunspider to profile with Instruments
 337         https://bugs.webkit.org/show_bug.cgi?id=80783
 338
 339         Reviewed by Filip Pizlo.
 340
 341         * resources/TimeProfile20us.tracetemplate: Added.
 342         * sunspider:
 343         (runTestsOnce):
 344
 345 2011-11-15  Andy Wingo  <wingo@igalia.com>
 346
 347         Extend sunspider driver to be able to run kraken
 348         https://bugs.webkit.org/show_bug.cgi?id=71799
 349
 350         Reviewed by Filip Pizlo.
 351
 352         * resources/sunspider-standalone-driver.js: Try to load a -data
 353         file.  If that succeeds, we have a kraken-like test, so we time
 354         the test using `load'.  Otherwise fall back to using `run'.
 355
 356 2011-08-11  Mark Rowe  <mrowe@apple.com>
 357
 358         Fix sunspider-compare-results to default to the same suite version as the sunspider script.
 359
 360         Rubber-stamped by Maciej Stachowiak.
 361
 362         * sunspider-compare-results:
 363
 364 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
 365
 366         Update SunSpider versioning to 1.0
 367         https://bugs.webkit.org/show_bug.cgi?id=63870
 368
 369         Reviewed by Dan Bernstein.
 370
 371         * hosted/sunspider.html:
 372         * hosted/versions.html:
 373         * resources/driver-TEMPLATE.html:
 374         * resources/results-TEMPLATE.html:
 375         * sunspider:
 376
 377 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
 378
 379         Sunspider 0.9.1 harness doesn't actually close() its test documents
 380         https://bugs.webkit.org/show_bug.cgi?id=47045
 381
 382         Reviewed by Daniel Bates.
 383
 384         * hosted/versions.html:
 385         * resources/driver-TEMPLATE.html:
 386
 387 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
 388
 389         Not Reviewed.
 390
 391         Fix obviou typo in previous commit.
 392
 393         * tests/sunspider-1.0/bitops-bitwise-and.js:
 394
 395 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
 396
 397         Reviewed by Anders Carlsson.
 398
 399         SunSpider: all four bitops benchmarks can be replaced with NOP
 400         https://bugs.webkit.org/show_bug.cgi?id=38446
 401
 402         * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
 403         (TimeFunc): Save the result.
 404         * tests/sunspider-1.0/bitops-bits-in-byte.js:
 405         (TimeFunc): Save the result.
 406         * tests/sunspider-1.0/bitops-bitwise-and.js: More explicitly
 407         save the results.
 408         * tests/sunspider-1.0/bitops-nsieve-bits.js:
 409         (sieve): Save the result.
 410
 411 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
 412
 413         SunSpider: string-validate-input.js uses an undeclared variable named 'name', which is a DOM API
 414         https://bugs.webkit.org/show_bug.cgi?id=60937
 415
 416         Reviewed by Dan Bernstein.
 417
 418         * tests/sunspider-1.0/string-validate-input.js:
 419         (doTest): Rename the variable to username
 420
 421 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
 422
 423         Reviewed by Darin Adler.
 424
 425         SunSpider: The main function of math-cordic is dead code
 426         https://bugs.webkit.org/show_bug.cgi?id=63863
 427
 428         Try to prevent math-cordic from being eliminated as dead code or
 429         compiled down to a constant.
 430
 431         * tests/sunspider-1.0/math-cordic.js:
 432         (cordicsincos):
 433         (cordic):
 434
 435 2011-06-30  Maciej Stachowiak  <mjs@apple.com>
 436
 437         Reviewed by Adam Barth.
 438
 439         Create sunspider-1.0 directory in preparation for fixing a bunch of bugs
 440         https://bugs.webkit.org/show_bug.cgi?id=63782
 441
 442         * make-hosted: Learn about the new directory.
 443         * tests/sunspider-1.0: Copied from PerformanceTests/SunSpider/tests/sunspider-0.9.1.
 444
 445 2010-11-29  Geoffrey Garen  <ggaren@apple.com>
 446
 447         Reviewed by Gavin Barraclough.
 448
 449         Improved accuracy of command-line SunSpider.
 450
 451         * resources/sunspider-standalone-driver.js: Use the returned number of
 452         elapsed milliseconds from "run" and "checkSyntax" instead of measuring
 453         ourselves, for slightly more accurate numbers.
 454
 455 2010-09-16  Gavin Barraclough  <barraclough@apple.com>
 456
 457         Reviewed by Mark Rowe.
 458         https://bugs.webkit.org/show_bug.cgi?id=45924
 459
 460         Modify the SunSpider harness to allow a path to be passed to the --suite
 461         argument, supress the normal prepend-"tests/" behaviour for suite names
 462         containing a '/'.
 463
 464         Also wrap the code in an anonymous function to move variables out of global
 465         scope (currently the harness cannot handle running tests the overwrite the
 466         variable 'j').
 467
 468         * resources/sunspider-standalone-driver.js:
 469         * sunspider:
 470
 471 2010-08-27  Michael Saboff  <msaboff@apple.com>
 472
 473         Reviewed by Stephanie Lewis.
 474
 475         Added version 5 and version 6 of V8 tests with modifications similar
 476         to the version 4 tests so they'll work with the SunSpider test harness.
 477         Note that these files are external source and therefore don't conform
 478         to WebKit coding standards.
 479         https://bugs.webkit.org/show_bug.cgi?id=44638
 480
 481         * tests/v8-v5: Added.
 482         * tests/v8-v5/LIST: Added.
 483         * tests/v8-v5/v8-crypto.js: Added.
 484         * tests/v8-v5/v8-deltablue.js: Added.
 485         * tests/v8-v5/v8-earley-boyer.js: Added.
 486         * tests/v8-v5/v8-raytrace.js: Added.
 487         * tests/v8-v5/v8-regexp.js: Added.
 488         * tests/v8-v5/v8-richards.js: Added.
 489         * tests/v8-v5/v8-splay.js: Added.
 490         * tests/v8-v6: Added.
 491         * tests/v8-v6/LIST: Added.
 492         * tests/v8-v6/v8-crypto.js: Added.
 493         * tests/v8-v6/v8-deltablue.js: Added.
 494         * tests/v8-v6/v8-earley-boyer.js: Added.
 495         * tests/v8-v6/v8-raytrace.js: Added.
 496         * tests/v8-v6/v8-regexp.js: Added.
 497         * tests/v8-v6/v8-richards.js: Added.
 498         * tests/v8-v6/v8-splay.js: Added.
 499
 500 2010-07-15  Stephanie Lewis  <slewis@apple.com>
 501
 502         Reviewed by Geoff Garen.
 503
 504         http://bugs.webkit.org/show_bug.cgi?id=42406
 505         Output location of results file.
 506
 507         * sunspider:
 508
 509 2010-05-17  Maciej Stachowiak  <mjs@apple.com>
 510
 511         Reviewed by Mark Rowe.
 512
 513         Browser-hosted version of SunSpider runs tests twice as many times as advertised
 514         https://bugs.webkit.org/show_bug.cgi?id=39201
 515
 516         * make-hosted: Make sure to reset test list after every test suite.
 517
 518 2010-05-07  Adam Roben  <aroben@apple.com>
 519
 520         Fix typo in sunspider-compare-results.js that caused incorrect
 521         two-sample t statistics to be computed
 522
 523         Fixes <http://webkit.org/b/38768> SunSpider computes incorrect
 524         two-sample t statistics
 525
 526         Reviewed by Darin Adler.
 527
 528         * resources/sunspider-compare-results.js:
 529         (sunspiderCompareResults.resultLine): Fixed the calculation of the t
 530         variable to match the definition of a two-sample t statistic. Looks
 531         like this was just a simple typo.
 532
 533 2010-04-05  Darin Adler  <darin@apple.com>
 534
 535         Updated the ignore property for changes to the contents here.
 536
 537         * .: Modified property svn:ignore.
 538
 539 2010-02-20  Maciej Stachowiak  <mjs@apple.com>
 540
 541         Reviewed by Adam Barth.
 542
 543         Add sunspider-0.9.1 to the site in preparation for announcing it.
 544         https://bugs.webkit.org/show_bug.cgi?id=35206
 545
 546         * hosted/sunspider.css: Remove tabs, which I noticed where present while
 547         copying to the WebKitSite directory.
 548
 549 2010-01-11  Oliver Hunt  <oliver@apple.com>
 550
 551         Reviewed by Darin Adler.
 552
 553         Sunspider parse-only tests don't run
 554         https://bugs.webkit.org/show_bug.cgi?id=33489
 555
 556         Fix parse-only test list
 557
 558         * tests/parse-only/LIST:
 559
 560 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
 561
 562         Reviewed by Adele Peterson.
 563
 564         Make SunSpider version more prominent in the title
 565         https://bugs.webkit.org/show_bug.cgi?id=32574
 566
 567         * hosted/sunspider.html:
 568         * hosted/versions.html:
 569         * resources/driver-TEMPLATE.html:
 570         * resources/results-TEMPLATE.html:
 571
 572 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
 573
 574         Reviewed by Darin Adler.
 575
 576         Make SunSpider refuse to compare results across content versions
 577         https://bugs.webkit.org/show_bug.cgi?id=32573
 578
 579         The results URL now includes a version indicator, and compare mode
 580         will refuse to compare between versions.
 581
 582         * resources/driver-TEMPLATE.html:
 583         * resources/results-TEMPLATE.html:
 584
 585 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
 586
 587         Reviewed by Darin Adler.
 588
 589         Some Browser-hosted SunSpider files are not valid HTML5
 590         https://bugs.webkit.org/show_bug.cgi?id=32536
 591
 592         Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
 593         close tags and adding a meta charset declaration).
 594
 595         * hosted/sunspider.html:
 596         * hosted/versions.html:
 597         * resources/TEMPLATE.html:
 598         * resources/driver-TEMPLATE.html:
 599         * resources/results-TEMPLATE.html:
 600
 601 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
 602
 603         Reviewed by Eric Seidel.
 604
 605         Make sunspider-0.9.1 the default content set (both command-line and hosted)
 606         https://bugs.webkit.org/show_bug.cgi?id=32537
 607
 608         * hosted/sunspider.html: Change default suite link.
 609         * sunspider: Change default suite.
 610         * sunspider-compare-results: Likewise.
 611
 612 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
 613
 614         Reviewed by Sam Weinig.
 615
 616         in-browser SunSpider suffers excessive penalty under power management
 617         https://bugs.webkit.org/show_bug.cgi?id=32505
 618
 619         I have made a few changes to address this:
 620
 621         1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
 622         document.write to reduce triggering of progress UI.
 623         2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
 624         stepped all the way down the whole time.
 625         3) Run the test cycle an extra time for warmup before the runs that count.
 626
 627         * hosted/sunspider-record-result.js: Removed.
 628         * make-hosted:
 629         * resources/TEMPLATE.html:
 630         * resources/driver-TEMPLATE.html:
 631
 632 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
 633
 634         Reviewed by Gavin Barraclough.
 635
 636         SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
 637         https://bugs.webkit.org/show_bug.cgi?id=16806
 638
 639         Based on a patch by Eric Seidel.
 640
 641         Fix the base64 computation to actually compute correct results. The impact on runtime of
 642         the test is pretty small, but noticeable for some browsers. But at least it's not
 643         doing a wrong and meaningless computation any more.
 644
 645         * tests/sunspider-0.9.1/string-base64.js:
 646         ():
 647         (base64ToString):
 648
 649 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
 650
 651         Fixing commit error...
 652
 653         I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
 654
 655         * resources/driver-TEMPLATE.html:
 656
 657 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
 658
 659         Reviewed by Sam Weinig.
 660
 661         Use JSON.parse instead of eval for Web-hosted SunSpider results processing
 662         https://bugs.webkit.org/show_bug.cgi?id=32490
 663
 664         * hosted/json2.js: Added. Incorporated from json.org
 665         * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
 666         * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
 667         results.
 668
 669 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
 670
 671         Reviewed by Oliver Hunt.
 672
 673         Enable Web-hosted version of SunSpider to handle multiple versions
 674         https://bugs.webkit.org/show_bug.cgi?id=32478
 675
 676         * make-hosted: Now generate the hosted version to be able to run both the
 677         0.9 and the 0.9.1 test suites.
 678         * hosted: Modified property svn:ignore.
 679         * hosted/sunspider-driver.html: Removed.
 680         * hosted/sunspider-results.html: Removed.
 681         * hosted/sunspider.html:
 682         * hosted/versions.html: Added.
 683         * resources/TEMPLATE.html:
 684         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
 685         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
 686
 687 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
 688
 689         Reviewed by Oliver Hunt.
 690
 691         Give command-line sunspider the ability to handle multiple suites and versions
 692         https://bugs.webkit.org/show_bug.cgi?id=32477
 693
 694         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
 695         enumerating the tests. Current suites are:
 696
 697         sunspider-0.9 - riginal SunSpider content
 698         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
 699         ubench - "ubench" microbenchmark suite
 700         v8-v4 - version 4 of the v8 benchmark
 701         parse-only - some parser tests
 702
 703         Sunspider can now be run with the --suite= parameter to select a
 704         suite. The default is --suite=sunspider-0.9.
 705
 706         * .: Modified property svn:ignore to ignore new results directories.
 707         * make-hosted: Generate based on sunspider-0.9 suite.
 708         * resources/TEMPLATE.html: Fix an HTML compliance problem.
 709         * resources/sunspider-standalone-driver.js: Become aware of suite names.
 710         * sunspider: Modified to handle multiple suites as described above.
 711         * sunspider-compare-results: ditto
 712         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
 713         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
 714         * tests/LIST-UBENCH: Removed.
 715         * tests/LIST-V8: Removed.
 716         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
 717         * tests/3d-morph.js: ditto
 718         * tests/3d-raytrace.js: ditto
 719         * tests/access-binary-trees.js: ditto
 720         * tests/access-fannkuch.js: ditto
 721         * tests/access-nbody.js: ditto
 722         * tests/access-nsieve.js: ditto
 723         * tests/bitops-3bit-bits-in-byte.js: ditto
 724         * tests/bitops-bits-in-byte.js: ditto
 725         * tests/bitops-bitwise-and.js: ditto
 726         * tests/bitops-nsieve-bits.js: ditto
 727         * tests/controlflow-recursive.js: ditto
 728         * tests/crypto-aes.js: ditto
 729         * tests/crypto-md5.js: ditto
 730         * tests/crypto-sha1.js: ditto
 731         * tests/date-format-tofte.js: ditto
 732         * tests/date-format-xparb.js: ditto
 733         * tests/math-cordic.js: ditto
 734         * tests/math-partial-sums.js: ditto
 735         * tests/math-spectral-norm.js: ditto
 736         * tests/regexp-dna.js: ditto
 737         * tests/string-base64.js: ditto
 738         * tests/string-fasta.js: ditto
 739         * tests/string-tagcloud.js: ditto
 740         * tests/string-unpack-code.js: ditto
 741         * tests/string-validate-input.js: ditto
 742         * tests/sunspider-0.9/: Added
 743         * tests/sunspider-0.9/*.js: Moved from one level up
 744         * tests/sunspider-0.9.1: Added.
 745         * tests/sunspider-0.9.1/*.js: Moved from one level up.
 746         * tests/v8-crypto.js: Moved to v8-v4/ directory.
 747         * tests/v8-deltablue.js: ditto
 748         * tests/v8-earley-boyer.js: ditto
 749         * tests/v8-raytrace.js: ditto
 750         * tests/v8-regexp.js: ditto
 751         * tests/v8-richards.js: ditto
 752         * tests/v8-splay.js: ditto
 753         * tests/v8-v4: Added.
 754
 755 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
 756
 757         Reviewed by Darin Adler.
 758
 759         Platform specific null device replaced with a platform independent.
 760         https://bugs.webkit.org/show_bug.cgi?id=29544
 761
 762         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
 763
 764 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
 765
 766         Reviewed by Maciej Stachowiak.
 767
 768         [Qt] Make sunspider script work on Windows platform.
 769         https://bugs.webkit.org/show_bug.cgi?id=29544
 770
 771         * sunspider:
 772         Using platform specific null device instead of /dev/null.
 773
 774 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
 775
 776         Reviewed by Oliver Hunt.
 777
 778         https://bugs.webkit.org/show_bug.cgi?id=26540
 779         Currently the SunSpider test driver lacks an option to run a test suite that
 780         will test JavaScriptCore parsing performance only.  This patch adds just such
 781         a test suite and option to SunSpider as well as the jsc test shell.  I've included
 782         three large javascript source files found in the wild: jquery, mootools and prototype.
 783         Combined with the concatenation of all three, these form a new testsuite to measure
 784         and test pure JavaScriptCore parsing performance.
 785
 786         * resources/sunspider-standalone-driver.js:
 787           Load the js with parse-only flag if it is part of the parse-only suite.
 788         * sunspider:
 789           Add the parse-only suite to the list of options.
 790         * sunspider-compare-results:
 791           Add the parse-only suite to the list of options.
 792         * tests/LIST-PARSE-ONLY: Added.
 793         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
 794           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
 795         * tests/parse-only/jquery-1.3.2.js: Added.
 796           Latest version from jquery.com.
 797         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
 798           Latest version from mootools.net.
 799         * tests/parse-only/prototype-1.6.0.3.js: Added.
 800           Latest version from prototypejs.org.
 801
 802 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
 803
 804         Update V8 benchmark to version 4.
 805
 806         Reviewed by Cameron Zwarich.
 807
 808         * tests/LIST-V8:
 809         * tests/v8-crypto.js:
 810         * tests/v8-deltablue.js:
 811         * tests/v8-earley-boyer.js:
 812         * tests/v8-raytrace.js:
 813         * tests/v8-regexp.js: Added.
 814         * tests/v8-richards.js:
 815         * tests/v8-splay.js: Added.
 816
 817 2009-05-26  Olivier DOLE  <odole@pleyo.com>
 818
 819         Reviewed by Maciej.
 820
 821         Store sunspider test suite results in a specific directory for each
 822         different type of benchmark (sunspider, V8 or ubench).
 823
 824         * sunspider:
 825         * sunspider-compare-results:
 826
 827 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
 828
 829         Build fix.
 830
 831         * SunSpider.make:
 832
 833 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
 834
 835         Include SunSpider tests in production builds.
 836
 837         Reviewed by Adam Roben.
 838
 839         * SunSpider.make: Added.
 840         * xcopy.excludes: Added.
 841
 842 2009-03-04  Oliver Hunt  <oliver@apple.com>
 843
 844         Added a few more tests to ubench
 845
 846         RS=Gavin Barraclough
 847
 848         Adding a couple of test to cover too few and too many arguments.  This
 849         is needed to prevent regressions in all call paths from my current refactoring.
 850
 851         * tests/LIST-UBENCH:
 852         * tests/ubench/function-correct-args.js: Added.
 853         (f):
 854         * tests/ubench/function-excess-args.js: Added.
 855         (f):
 856
 857 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
 858
 859         Fix a bug where when V8's or UBENCH's test suite list file is missing
 860         the error message is suggesting that SunSpider's list file is missing.
 861
 862         Reviewed by Sam Weinig.
 863
 864         * sunspider:
 865         (loadTestsList): Fix the error message to use the right list name.
 866
 867 2008-10-07  David Hyatt  <hyatt@apple.com>
 868
 869         https://bugs.webkit.org/show_bug.cgi?id=18989
 870
 871         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
 872         we want all browsers to be doing the same thing on the test.
 873
 874         Reviewed by Maciej
 875
 876         * tests/regexp-dna.js:
 877
 878 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 879
 880         Rubber-stamped by Maciej Stachowiak.
 881
 882         The bleeding edge branch of V8 now supports passing fileanmes after
 883         '-f', so the SunSpider script should be changed to do this again.
 884         This allows all 3 major JavaScript engines to be used with the
 885         unmodified SunSpider script.
 886
 887         * sunspider:
 888
 889 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 890
 891         Reviewed by Maciej Stachowiak.
 892
 893         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
 894         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
 895         correct the punctuation in its documentation.
 896
 897         * sunspider:
 898
 899 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 900
 901         Reviewed by Oliver Hunt.
 902
 903         Bug 20616: Incorporate V8 benchmarks in testing
 904         <https://bugs.webkit.org/show_bug.cgi?id=20616>
 905
 906         This is a first attempt at running the V8 benchmarks within SunSpider,
 907         with the --v8 option. The number of iterations is fixed for each
 908         benchmark, and was chosen to make each of the benchmarks run between
 909         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
 910
 911         * sunspider:
 912         * tests/LIST-V8: Added.
 913         * tests/v8-crypto.js: Added.
 914         * tests/v8-deltablue.js: Added.
 915         * tests/v8-earley-boyer.js: Added.
 916         * tests/v8-raytrace.js: Added.
 917         * tests/v8-richards.js: Added.
 918
 919 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 920
 921         Not reviewed.
 922
 923         Remove a local change that was accidentally committed in r36034.
 924
 925         * resources/sunspider-standalone-driver.js:
 926
 927 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 928
 929         Rubber-stamped by Oliver Hunt.
 930
 931         Make SunSpider work with JavaScript shells that don't take filenames
 932         after a '-f' argument.
 933
 934         * sunspider:
 935
 936 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 937
 938         Reviewed by Timothy.
 939
 940         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
 941
 942         * sunspider:
 943         * tests/LIST-SQUIRRELFISH: Removed.
 944
 945 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
 946
 947         Rubber-stamped by Mark Rowe.
 948
 949         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
 950
 951         * sunspider:
 952
 953 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
 954
 955         Rubber stamped by Oliver.
 956
 957         - add newly working tests to squirrelfish list
 958
 959         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
 960
 961 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
 962
 963         Rubber stamped by Oliver.
 964
 965         - move ubench tests properly.
 966
 967         * tests/ubench: Added.
 968         * tests/ubench/function-closure.js: Added.
 969         * tests/ubench/function-empty.js: Added.
 970         * tests/ubench/function-missing-args.js: Added.
 971         * tests/ubench/function-sum.js: Added.
 972         * tests/ubench/loop-empty-resolve.js: Added.
 973         * tests/ubench/loop-empty.js: Added.
 974         * tests/ubench/loop-sum.js: Added.
 975
 976 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
 977
 978         Rubber stamped by Oliver.
 979
 980         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
 981
 982 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
 983
 984         Reviewed by Oliver.
 985
 986         - make --squirrelfish mode use the real harness, but a reduced set of tests
 987
 988         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
 989
 990         * sunspider:
 991         * tests/LIST-SQUIRRELFISH:
 992         * tests/LIST-UBENCH: Added.
 993         * tests/squirrelfish: Moved to tests/ubench
 994
 995 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
 996
 997         Reviewed by Maciej Stachowiak.
 998
 999         Removed an unused test.
1000
1001         * tests/squirrelfish/loop-resolve.js: Removed.
1002
1003 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
1004
1005         Reviewed by Oliver.
1006
1007         Add newly runnable tests to --squirrelfish mode.
1008
1009         * tests/LIST-SQUIRRELFISH:
1010
1011 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
1012
1013         Reviewed by Maciej Stachowiak.
1014
1015         A little more love for --squirrelfish mode:
1016
1017         Fixed a misplaced sort. Results properly sort now.
1018
1019         Added a test list just for squirrelfish, and updated it to include
1020         all currently passing SunSpider tests.
1021
1022         Fixed the pruning regexp to match 3d-morph.js.
1023
1024         * sunspider:
1025
1026 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
1027
1028         Reviewed by Oliver Hunt.
1029
1030         --squirrelfish mode: pared down tests for squirrelfish to chew on.
1031
1032         * sunspider:
1033         * tests/LIST:
1034         * tests/squirrelfish: Added.
1035         * tests/squirrelfish/function-closure.js: Added.
1036         * tests/squirrelfish/function-empty.js: Added.
1037         * tests/squirrelfish/function-missing-args.js: Added.
1038         * tests/squirrelfish/function-sum.js: Added.
1039         * tests/squirrelfish/loop-empty-resolve.js: Added.
1040         * tests/squirrelfish/loop-empty.js: Added.
1041         * tests/squirrelfish/loop-resolve.js: Added.
1042         * tests/squirrelfish/loop-sum.js: Added.
1043
1044 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
1045
1046         Reviewed by Geoff.
1047
1048         * sunspider: Pause update daemon on Darwin to make results reliable
1049         again (workaround for <rdar://problem/5811127>).
1050
1051 2008-02-07  Eric Seidel  <eric@webkit.org>
1052
1053         Reviewed by Sam.
1054
1055         * sunspider: add an error message when ./tests/LIST is not found
1056
1057 2007-12-29  Alp Toker  <alp@atoker.com>
1058
1059         Fix typo.
1060
1061         * hosted/sunspider.html:
1062
1063 2007-12-29  Alp Toker  <alp@atoker.com>
1064
1065         Reviewed by Maciej.
1066
1067         Provide the current URL in the SunSpider results page. Useful for
1068         copying benchmark results out of browsers without a location bar or
1069         where the location bar can't handle long URLs.
1070
1071         * hosted/sunspider-results.html:
1072
1073 2007-12-20  Eric Seidel  <eric@webkit.org>
1074
1075         Reviewed by mjs.
1076
1077         * sunspider: increase default number of runs for more accurate numbers
1078
1079 2007-12-18  Mark Rowe  <mrowe@apple.com>
1080
1081         Fix typo.
1082
1083         * hosted/sunspider.html:
1084
1085 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
1086
1087         Reviewed by Eric.
1088
1089         - give an extra digit of precision for small differences.
1090
1091         * resources/sunspider-compare-results.js:
1092
1093 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
1094
1095         Reviewed by Eric.
1096
1097         - work around IE layout bugs
1098
1099         * hosted/sunspider.css:
1100
1101 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
1102
1103         Reviewed by Mark.
1104
1105         - fix compare feature
1106
1107         * hosted/sunspider-results.html:
1108
1109 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
1110
1111         Reviewed by Mark.
1112
1113         - Change the format of the output URL to be shorter, to meet IE constraints
1114
1115         * hosted/sunspider-driver.html:
1116         * hosted/sunspider-results.html:
1117
1118 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
1119
1120         Reviewed by Oliver.
1121
1122         - Add some explanatory test and a dumb logo to SunSpider web hosted version
1123
1124         * hosted/sunspider-driver.html:
1125         * hosted/sunspider-results.html:
1126         * hosted/sunspider.css:
1127         * hosted/sunspider.html:
1128
1129 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
1130
1131         Reviewed by Oliver.
1132
1133         - Make the web-hosted version of SunSpider look a little bit less drab.
1134
1135         * hosted/sunspider-driver.html:
1136         * hosted/sunspider-results.html:
1137         * hosted/sunspider.css: Added.
1138         * hosted/sunspider.html:
1139         * resources/TEMPLATE.html:
1140
1141 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
1142
1143         Reviewed by Sam.
1144
1145         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
1146
1147         A sample of what new-style results would look like:
1148
1149         TEST                   COMPARISON            FROM                 TO             DETAILS
1150         ...
1151         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
1152         ...
1153             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
1154
1155         * resources/sunspider-compare-results.js:
1156
1157 2007-11-14  Eric Seidel  <eric@webkit.org>
1158
1159         Reviewed by Sam.
1160
1161         * sunspider: add --shark-cache for L2 Cache Miss profiling
1162
1163 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
1164
1165         Reviewed by Darin Adler.
1166
1167         http://bugs.webkit.org/show_bug.cgi?id=15925
1168         SunSpider should check for Shark being installed
1169
1170         * sunspider: Explicitly test that shark command line tool is installed.
1171
1172 2007-11-04  Darin Adler  <darin@apple.com>
1173
1174         * hosted/sunspider.html: Tweaked formatting a bit.
1175
1176 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
1177
1178         Reviewed by Darin Adler.
1179
1180         - Add compare mode and a description of the benchmark to the hosted version.
1181
1182         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
1183         * hosted/sunspider.html: Added intro text.
1184         * make-hosted: Copy new file.
1185         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
1186         a function.
1187         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
1188         expects.
1189         * sunspider-compare-results: Use sunspider-standalone-compare.js.
1190
1191 2007-11-02  Darin Adler  <darin@apple.com>
1192
1193         * sunspider: Changed "--base" to "--set-baseline".
1194
1195 2007-11-02  Darin Adler  <darin@apple.com>
1196
1197         Reviewed by Maciej.
1198
1199         * sunspider: Added a "--base" option that records the current test as a baseline.
1200         Moved the code to find the newest .mshark file into a function (since I needed to use
1201         it in the other script and I prefer to copy/paste a function).
1202
1203         * sunspider-compare-results: Added logic to compare the baseline with the last
1204         run when you don't pass any parameters.
1205
1206 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
1207
1208         Reviewed by Eric.
1209
1210         - Added list of ECMAScript3 features that SunSpider does not cover currently.
1211
1212         * UNCOVERED: Added.
1213
1214 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
1215
1216         Reviewed by Oliver.
1217
1218         - Give better error messages when sunspider-compare-results fails
1219
1220         * sunspider-compare-results:
1221
1222 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
1223
1224         Reviewed by Darin Adler.
1225
1226         - fix Date tests to work in command-line SpiderMonkey
1227
1228         * tests/date-format-tofte.js:
1229         * tests/date-format-xparb.js:
1230
1231 2007-10-29  Darin Adler  <darin@apple.com>
1232
1233         Reviewed by Maciej.
1234
1235         * sunspider: Added --shark20 option, to run Shark at its highest sample
1236         resolution instead of its default.
1237
1238 2007-10-28  Adam Roben  <aroben@apple.com>
1239
1240         Fix SunSpider on Windows
1241
1242         Reviewed by Eric.
1243
1244         * sunspider: Don't use colons in filenames.
1245
1246 2007-10-28  Darin Adler  <darin@apple.com>
1247
1248         Reviewed by Adam.
1249
1250         * resources/sunspider-analyze-results.js: Tweak the output format a little.
1251         Change so that when there's only one run we don't write out confidence
1252         intervals at all rather than writing "NaN" over and over again.
1253
1254         * sunspider: Use the time and date as a suffix on the results file, that way
1255         all the results are left behind in the tmp directory. This will make it easy
1256         for us to add features that compare past results in the future. Also rename
1257         the .mshark files using the same scheme.
1258
1259         * sunspider-compare-results: Relax the parsing rules so we can parse the
1260         results file exactly as it's emitted from sunspider.
1261
1262 2007-10-25  Adam Roben  <aroben@apple.com>
1263
1264         Strip carriage returns from results
1265
1266         Carriage returns were screwing up sunspider-compare-results on
1267         Windows.
1268
1269         Rubberstamped by Oliver.
1270
1271         * sunspider:
1272
1273 2007-10-24  Darin Adler  <darin@apple.com>
1274
1275         Reviewed by Eric.
1276
1277         * sunspider: Open the profile in shark after completing a run.
1278         * sunspider-compare-results: Tiny perl tweaks.
1279
1280 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
1281
1282         Reviewed by Darin Adler.
1283
1284         - Don't hardcode my path to testkjs
1285
1286         * sunspider-compare-results:
1287
1288 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
1289
1290         Reviewed by Darin Adler.
1291
1292         - Add a compare script to compare results, and improve formatting of standard results a bit.
1293
1294         * resources/sunspider-analyze-results.js:
1295         * resources/sunspider-compare-results.js: Added.
1296         * sunspider-compare-results: Added.
1297
1298 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
1299
1300         Reviewed by Darin Adler.
1301
1302         * resources/sunspider-analyze-results.js: Do the error range
1303         properly, using the t-distribution instead of 1.96 (which was
1304         based on the normal distribution).
1305
1306         * sunspider: Print results in a way that will be friendlier to the
1307         soon-to-come compare mode.
1308
1309 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
1310
1311         Reviewed by Review.
1312
1313         - add more tests (probably the final set for now)
1314
1315         Already balanced. Added date, regexp, control flow, and a few
1316         more string and object/array access tests.
1317
1318         * tests/LIST:
1319         * tests/access-fannkuch.js: Added.
1320         * tests/access-nbody.js: Added.
1321         * tests/controlflow-recursive.js: Added.
1322         * tests/date-format-tofte.js: Added.
1323         * tests/date-format-xparb.js: Added.
1324         * tests/regexp-dna.js: Added.
1325         * tests/string-validate-input.js: Added.
1326
1327 2007-10-21  Eric Seidel  <eric@webkit.org>
1328
1329         Reviewed by mjs.
1330
1331         * sunspider: make --shark only sample from testkjs
1332
1333 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
1334
1335         Rubber stamped by Eric.
1336
1337         - added crypto tests (already balanced)
1338
1339         * tests/LIST:
1340         * tests/crypto-aes.js: Added.
1341         * tests/crypto-md5.js: Added.
1342         * tests/crypto-sha1.js: Added.
1343
1344 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
1345
1346         Reviewed by Adam.
1347
1348         - rebalance new tests
1349
1350         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
1351
1352         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1353
1354         This balances them with the existing tests.
1355
1356         * tests/access-binary-trees.js:
1357         * tests/access-nsieve.js:
1358         * tests/bitops-nsieve-bits.js:
1359         * tests/math-partial-sums.js:
1360         * tests/math-spectral-norm.js:
1361         * tests/string-fasta.js:
1362
1363 2007-10-20  Eric Seidel  <eric@webkit.org>
1364
1365         Reviewed by darin.
1366
1367         Add improved argument handling to sunspider, including
1368         --runs=<number>, --tests=<pattern, --shark, and --help
1369         Also re-factor code into subroutines
1370
1371         * sunspider: improved argument handling
1372
1373 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1374
1375         Reviewed by Mark.
1376
1377         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
1378
1379         * TODO:
1380         * tests/LIST:
1381         * tests/access-binary-trees.js: Added.
1382         * tests/access-nsieve.js: Added.
1383         * tests/bitops-nsieve-bits.js: Added.
1384         * tests/math-partial-sums.js: Added.
1385         * tests/math-spectral-norm.js: Added.
1386         * tests/string-fasta.js: Added.
1387
1388 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1389
1390         Reviewed by Darin Adler.
1391
1392         * TODO: Updated to note areas that are now well-covered.
1393
1394         - Change the tests and the driver to avoid leaving large object
1395         graphs hanging around, since that throws off the subsequent tests.
1396
1397         * resources/sunspider-standalone-driver.js:
1398         * tests/3d-cube.js:
1399         * tests/3d-morph.js:
1400         * tests/3d-raytrace.js:
1401         * tests/bitops-bitwise-and.js:
1402         * tests/string-base64.js:
1403         * tests/string-tagcloud.js:
1404
1405 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1406
1407         Reviewed by Nikolas.
1408
1409         - Rebalance test complexity.
1410
1411         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
1412
1413         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1414
1415         However, for some tests, IE7 was a huge outlier compared to other
1416         major browsers. For those tests, I used Firefox 2.0.0.8 on the
1417         same OS and hardware as a normalization basis.
1418
1419         * tests/3d-morph.js:
1420         * tests/3d-raytrace.js:
1421         * tests/bitops-3bit-bits-in-byte.js:
1422         * tests/bitops-bits-in-byte.js:
1423         * tests/bitops-bitwise-and.js:
1424         * tests/math-cordic.js:
1425         * tests/string-base64.js:
1426         * tests/string-tagcloud.js:
1427         * tests/string-unpack-code.js:
1428
1429 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1430
1431         Reviewed by Oliver.
1432
1433         - Fix the browser-hosted driver to be compatible with Safari 2
1434
1435         * hosted/sunspider-driver.html:
1436
1437 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1438
1439         Rubber stamped by Hyatt.
1440
1441         - Add BSD license notice to cordic test, the original author said to do so.
1442
1443         * tests/math-cordic.js:
1444
1445 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1446
1447         Reviewed by Eric.
1448
1449         - Support Windows and Windows browsers.
1450
1451         * hosted/sunspider-results.html: Display results correctly in IE.
1452         * sunspider: tweaks to work on Windows, and with spaces in the path.
1453
1454 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1455
1456         Rubber stamped by Oliver.
1457
1458         * tests/string-tagcloud.js: Remove stray print()
1459
1460 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
1461
1462         Reviewed by Oliver.
1463
1464         Add some string test cases.
1465
1466         * tests/string-base64.js: Added. Does base64 encoding and decoding.
1467         * tests/string-tagcloud.js: Added. Parses a JSON string and
1468         generates tagcloud HTML markup based on the data.
1469         * tests/string-unpack-code.js: Added. Unpack compressed versions of
1470         four of the most popular JavaScript libraries.
1471         * tests/LIST: Added the new tests.
1472
1473 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
1474
1475         Reviewed by Anders.
1476
1477         * TODO: Updated with info on how to do the stats properly.
1478
1479         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
1480         avoid overly slowing down the benchmark.
1481
1482         Added some 3D-related test cases.
1483
1484         * tests/3d-cube.js: Added.
1485         * tests/3d-morph.js: Added.
1486         * tests/3d-raytrace.js: Added.
1487         * tests/LIST:
1488
1489 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
1490
1491         Reviewed by Darin Adler.
1492
1493         - New JavaScript benchmark
1494         http://bugs.webkit.org/show_bug.cgi?id=15515
1495
1496         A start on a new JavaScript benchmark, with standalone and
1497         browser-hosted drivers.
1498
1499         The test content is still incomplete and the drivers could use
1500         some more features but there is enough here to make a good start.
1501
1502         The drivers compute a 95% confidence interval on the mean for the
1503         whole test, each category, and each individual test to make it
1504         easier to tell whether differences are statistically
1505         significant. The confidence interval can be narrowed by running
1506         the test more times.
1507
1508         * TODO: Added. Remaining things that need to be done.
1509         * sunspider: Added. Perl script that acts as the standalone test driver.
1510         * resources/sunspider-standalone-driver.js: Added. JavaScript part
1511         of standalone test driver.
1512         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
1513         analysis code.
1514         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
1515         * make-hosted: Added. Script to generate browser-hosted tests.
1516         * hosted/sunspider.html: Added. Start page for browser-hosted test.
1517         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
1518         * hosted/sunspider-results.html: Added. Results page for browser-hosted
1519         tests.
1520         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
1521         tests.
1522         * tests/LIST: Added. List of tests to use.
1523         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
1524         * tests/bitops-bits-in-byte.js: Added. Ditto.
1525         * tests/bitops-bitwise-and.js: Added. Ditto.
1526         * tests/math-cordic.js: Added. Ditto.
1527