PerformanceTests/SunSpider/ChangeLog

   1 2010-11-29  Geoffrey Garen  <ggaren@apple.com>
   2
   3         Reviewed by Gavin Barraclough.
   4
   5         Improved accuracy of command-line SunSpider.
   6
   7         * resources/sunspider-standalone-driver.js: Use the returned number of
   8         elapsed milliseconds from "run" and "checkSyntax" instead of measuring
   9         ourselves, for slightly more accurate numbers.
  10
  11 2010-09-16  Gavin Barraclough  <barraclough@apple.com>
  12
  13         Reviewed by Mark Rowe.
  14         https://bugs.webkit.org/show_bug.cgi?id=45924
  15
  16         Modify the SunSpider harness to allow a path to be passed to the --suite
  17         argument, supress the normal prepend-"tests/" behaviour for suite names
  18         containing a '/'.
  19
  20         Also wrap the code in an anonymous function to move variables out of global
  21         scope (currently the harness cannot handle running tests the overwrite the
  22         variable 'j').
  23
  24         * resources/sunspider-standalone-driver.js:
  25         * sunspider:
  26
  27 2010-08-27  Michael Saboff  <msaboff@apple.com>
  28
  29         Reviewed by Stephanie Lewis.
  30
  31         Added version 5 and version 6 of V8 tests with modifications similar
  32         to the version 4 tests so they'll work with the SunSpider test harness.
  33         Note that these files are external source and therefore don't conform
  34         to WebKit coding standards.
  35         https://bugs.webkit.org/show_bug.cgi?id=44638
  36
  37         * tests/v8-v5: Added.
  38         * tests/v8-v5/LIST: Added.
  39         * tests/v8-v5/v8-crypto.js: Added.
  40         * tests/v8-v5/v8-deltablue.js: Added.
  41         * tests/v8-v5/v8-earley-boyer.js: Added.
  42         * tests/v8-v5/v8-raytrace.js: Added.
  43         * tests/v8-v5/v8-regexp.js: Added.
  44         * tests/v8-v5/v8-richards.js: Added.
  45         * tests/v8-v5/v8-splay.js: Added.
  46         * tests/v8-v6: Added.
  47         * tests/v8-v6/LIST: Added.
  48         * tests/v8-v6/v8-crypto.js: Added.
  49         * tests/v8-v6/v8-deltablue.js: Added.
  50         * tests/v8-v6/v8-earley-boyer.js: Added.
  51         * tests/v8-v6/v8-raytrace.js: Added.
  52         * tests/v8-v6/v8-regexp.js: Added.
  53         * tests/v8-v6/v8-richards.js: Added.
  54         * tests/v8-v6/v8-splay.js: Added.
  55
  56 2010-07-15  Stephanie Lewis  <slewis@apple.com>
  57
  58         Reviewed by Geoff Garen.
  59
  60         http://bugs.webkit.org/show_bug.cgi?id=42406
  61         Output location of results file.
  62
  63         * sunspider:
  64
  65 2010-05-17  Maciej Stachowiak  <mjs@apple.com>
  66
  67         Reviewed by Mark Rowe.
  68
  69         Browser-hosted version of SunSpider runs tests twice as many times as advertised
  70         https://bugs.webkit.org/show_bug.cgi?id=39201
  71
  72         * make-hosted: Make sure to reset test list after every test suite.
  73
  74 2010-05-07  Adam Roben  <aroben@apple.com>
  75
  76         Fix typo in sunspider-compare-results.js that caused incorrect
  77         two-sample t statistics to be computed
  78
  79         Fixes <http://webkit.org/b/38768> SunSpider computes incorrect
  80         two-sample t statistics
  81
  82         Reviewed by Darin Adler.
  83
  84         * resources/sunspider-compare-results.js:
  85         (sunspiderCompareResults.resultLine): Fixed the calculation of the t
  86         variable to match the definition of a two-sample t statistic. Looks
  87         like this was just a simple typo.
  88
  89 2010-04-05  Darin Adler  <darin@apple.com>
  90
  91         Updated the ignore property for changes to the contents here.
  92
  93         * .: Modified property svn:ignore.
  94
  95 2010-02-20  Maciej Stachowiak  <mjs@apple.com>
  96
  97         Reviewed by Adam Barth.
  98
  99         Add sunspider-0.9.1 to the site in preparation for announcing it.
 100         https://bugs.webkit.org/show_bug.cgi?id=35206
 101
 102         * hosted/sunspider.css: Remove tabs, which I noticed where present while
 103         copying to the WebKitSite directory.
 104
 105 2010-01-11  Oliver Hunt  <oliver@apple.com>
 106
 107         Reviewed by Darin Adler.
 108
 109         Sunspider parse-only tests don't run
 110         https://bugs.webkit.org/show_bug.cgi?id=33489
 111
 112         Fix parse-only test list
 113
 114         * tests/parse-only/LIST:
 115
 116 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
 117
 118         Reviewed by Adele Peterson.
 119
 120         Make SunSpider version more prominent in the title
 121         https://bugs.webkit.org/show_bug.cgi?id=32574
 122
 123         * hosted/sunspider.html:
 124         * hosted/versions.html:
 125         * resources/driver-TEMPLATE.html:
 126         * resources/results-TEMPLATE.html:
 127
 128 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
 129
 130         Reviewed by Darin Adler.
 131
 132         Make SunSpider refuse to compare results across content versions
 133         https://bugs.webkit.org/show_bug.cgi?id=32573
 134
 135         The results URL now includes a version indicator, and compare mode
 136         will refuse to compare between versions.
 137
 138         * resources/driver-TEMPLATE.html:
 139         * resources/results-TEMPLATE.html:
 140
 141 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
 142
 143         Reviewed by Darin Adler.
 144
 145         Some Browser-hosted SunSpider files are not valid HTML5
 146         https://bugs.webkit.org/show_bug.cgi?id=32536
 147
 148         Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
 149         close tags and adding a meta charset declaration).
 150
 151         * hosted/sunspider.html:
 152         * hosted/versions.html:
 153         * resources/TEMPLATE.html:
 154         * resources/driver-TEMPLATE.html:
 155         * resources/results-TEMPLATE.html:
 156
 157 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
 158
 159         Reviewed by Eric Seidel.
 160
 161         Make sunspider-0.9.1 the default content set (both command-line and hosted)
 162         https://bugs.webkit.org/show_bug.cgi?id=32537
 163
 164         * hosted/sunspider.html: Change default suite link.
 165         * sunspider: Change default suite.
 166         * sunspider-compare-results: Likewise.
 167
 168 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
 169
 170         Reviewed by Sam Weinig.
 171
 172         in-browser SunSpider suffers excessive penalty under power management
 173         https://bugs.webkit.org/show_bug.cgi?id=32505
 174
 175         I have made a few changes to address this:
 176
 177         1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
 178         document.write to reduce triggering of progress UI.
 179         2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
 180         stepped all the way down the whole time.
 181         3) Run the test cycle an extra time for warmup before the runs that count.
 182
 183         * hosted/sunspider-record-result.js: Removed.
 184         * make-hosted:
 185         * resources/TEMPLATE.html:
 186         * resources/driver-TEMPLATE.html:
 187
 188 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
 189
 190         Reviewed by Gavin Barraclough.
 191
 192         SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
 193         https://bugs.webkit.org/show_bug.cgi?id=16806
 194
 195         Based on a patch by Eric Seidel.
 196
 197         Fix the base64 computation to actually compute correct results. The impact on runtime of
 198         the test is pretty small, but noticeable for some browsers. But at least it's not
 199         doing a wrong and meaningless computation any more.
 200
 201         * tests/sunspider-0.9.1/string-base64.js:
 202         ():
 203         (base64ToString):
 204
 205 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
 206
 207         Fixing commit error...
 208
 209         I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
 210
 211         * resources/driver-TEMPLATE.html:
 212
 213 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
 214
 215         Reviewed by Sam Weinig.
 216
 217         Use JSON.parse instead of eval for Web-hosted SunSpider results processing
 218         https://bugs.webkit.org/show_bug.cgi?id=32490
 219
 220         * hosted/json2.js: Added. Incorporated from json.org
 221         * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
 222         * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
 223         results.
 224
 225 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
 226
 227         Reviewed by Oliver Hunt.
 228
 229         Enable Web-hosted version of SunSpider to handle multiple versions
 230         https://bugs.webkit.org/show_bug.cgi?id=32478
 231
 232         * make-hosted: Now generate the hosted version to be able to run both the
 233         0.9 and the 0.9.1 test suites.
 234         * hosted: Modified property svn:ignore.
 235         * hosted/sunspider-driver.html: Removed.
 236         * hosted/sunspider-results.html: Removed.
 237         * hosted/sunspider.html:
 238         * hosted/versions.html: Added.
 239         * resources/TEMPLATE.html:
 240         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
 241         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
 242
 243 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
 244
 245         Reviewed by Oliver Hunt.
 246
 247         Give command-line sunspider the ability to handle multiple suites and versions
 248         https://bugs.webkit.org/show_bug.cgi?id=32477
 249
 250         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
 251         enumerating the tests. Current suites are:
 252
 253         sunspider-0.9 - riginal SunSpider content
 254         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
 255         ubench - "ubench" microbenchmark suite
 256         v8-v4 - version 4 of the v8 benchmark
 257         parse-only - some parser tests
 258
 259         Sunspider can now be run with the --suite= parameter to select a
 260         suite. The default is --suite=sunspider-0.9.
 261
 262         * .: Modified property svn:ignore to ignore new results directories.
 263         * make-hosted: Generate based on sunspider-0.9 suite.
 264         * resources/TEMPLATE.html: Fix an HTML compliance problem.
 265         * resources/sunspider-standalone-driver.js: Become aware of suite names.
 266         * sunspider: Modified to handle multiple suites as described above.
 267         * sunspider-compare-results: ditto
 268         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
 269         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
 270         * tests/LIST-UBENCH: Removed.
 271         * tests/LIST-V8: Removed.
 272         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
 273         * tests/3d-morph.js: ditto
 274         * tests/3d-raytrace.js: ditto
 275         * tests/access-binary-trees.js: ditto
 276         * tests/access-fannkuch.js: ditto
 277         * tests/access-nbody.js: ditto
 278         * tests/access-nsieve.js: ditto
 279         * tests/bitops-3bit-bits-in-byte.js: ditto
 280         * tests/bitops-bits-in-byte.js: ditto
 281         * tests/bitops-bitwise-and.js: ditto
 282         * tests/bitops-nsieve-bits.js: ditto
 283         * tests/controlflow-recursive.js: ditto
 284         * tests/crypto-aes.js: ditto
 285         * tests/crypto-md5.js: ditto
 286         * tests/crypto-sha1.js: ditto
 287         * tests/date-format-tofte.js: ditto
 288         * tests/date-format-xparb.js: ditto
 289         * tests/math-cordic.js: ditto
 290         * tests/math-partial-sums.js: ditto
 291         * tests/math-spectral-norm.js: ditto
 292         * tests/regexp-dna.js: ditto
 293         * tests/string-base64.js: ditto
 294         * tests/string-fasta.js: ditto
 295         * tests/string-tagcloud.js: ditto
 296         * tests/string-unpack-code.js: ditto
 297         * tests/string-validate-input.js: ditto
 298         * tests/sunspider-0.9/: Added
 299         * tests/sunspider-0.9/*.js: Moved from one level up
 300         * tests/sunspider-0.9.1: Added.
 301         * tests/sunspider-0.9.1/*.js: Moved from one level up.
 302         * tests/v8-crypto.js: Moved to v8-v4/ directory.
 303         * tests/v8-deltablue.js: ditto
 304         * tests/v8-earley-boyer.js: ditto
 305         * tests/v8-raytrace.js: ditto
 306         * tests/v8-regexp.js: ditto
 307         * tests/v8-richards.js: ditto
 308         * tests/v8-splay.js: ditto
 309         * tests/v8-v4: Added.
 310
 311 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
 312
 313         Reviewed by Darin Adler.
 314
 315         Platform specific null device replaced with a platform independent.
 316         https://bugs.webkit.org/show_bug.cgi?id=29544
 317
 318         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
 319
 320 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
 321
 322         Reviewed by Maciej Stachowiak.
 323
 324         [Qt] Make sunspider script work on Windows platform.
 325         https://bugs.webkit.org/show_bug.cgi?id=29544
 326
 327         * sunspider:
 328         Using platform specific null device instead of /dev/null.
 329
 330 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
 331
 332         Reviewed by Oliver Hunt.
 333
 334         https://bugs.webkit.org/show_bug.cgi?id=26540
 335         Currently the SunSpider test driver lacks an option to run a test suite that
 336         will test JavaScriptCore parsing performance only.  This patch adds just such
 337         a test suite and option to SunSpider as well as the jsc test shell.  I've included
 338         three large javascript source files found in the wild: jquery, mootools and prototype.
 339         Combined with the concatenation of all three, these form a new testsuite to measure
 340         and test pure JavaScriptCore parsing performance.
 341
 342         * resources/sunspider-standalone-driver.js:
 343           Load the js with parse-only flag if it is part of the parse-only suite.
 344         * sunspider:
 345           Add the parse-only suite to the list of options.
 346         * sunspider-compare-results:
 347           Add the parse-only suite to the list of options.
 348         * tests/LIST-PARSE-ONLY: Added.
 349         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
 350           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
 351         * tests/parse-only/jquery-1.3.2.js: Added.
 352           Latest version from jquery.com.
 353         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
 354           Latest version from mootools.net.
 355         * tests/parse-only/prototype-1.6.0.3.js: Added.
 356           Latest version from prototypejs.org.
 357
 358 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
 359
 360         Update V8 benchmark to version 4.
 361
 362         Reviewed by Cameron Zwarich.
 363
 364         * tests/LIST-V8:
 365         * tests/v8-crypto.js:
 366         * tests/v8-deltablue.js:
 367         * tests/v8-earley-boyer.js:
 368         * tests/v8-raytrace.js:
 369         * tests/v8-regexp.js: Added.
 370         * tests/v8-richards.js:
 371         * tests/v8-splay.js: Added.
 372
 373 2009-05-26  Olivier DOLE  <odole@pleyo.com>
 374
 375         Reviewed by Maciej.
 376
 377         Store sunspider test suite results in a specific directory for each
 378         different type of benchmark (sunspider, V8 or ubench).
 379
 380         * sunspider:
 381         * sunspider-compare-results:
 382
 383 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
 384
 385         Build fix.
 386
 387         * SunSpider.make:
 388
 389 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
 390
 391         Include SunSpider tests in production builds.
 392
 393         Reviewed by Adam Roben.
 394
 395         * SunSpider.make: Added.
 396         * xcopy.excludes: Added.
 397
 398 2009-03-04  Oliver Hunt  <oliver@apple.com>
 399
 400         Added a few more tests to ubench
 401
 402         RS=Gavin Barraclough
 403
 404         Adding a couple of test to cover too few and too many arguments.  This
 405         is needed to prevent regressions in all call paths from my current refactoring.
 406
 407         * tests/LIST-UBENCH:
 408         * tests/ubench/function-correct-args.js: Added.
 409         (f):
 410         * tests/ubench/function-excess-args.js: Added.
 411         (f):
 412
 413 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
 414
 415         Fix a bug where when V8's or UBENCH's test suite list file is missing
 416         the error message is suggesting that SunSpider's list file is missing.
 417
 418         Reviewed by Sam Weinig.
 419
 420         * sunspider:
 421         (loadTestsList): Fix the error message to use the right list name.
 422
 423 2008-10-07  David Hyatt  <hyatt@apple.com>
 424
 425         https://bugs.webkit.org/show_bug.cgi?id=18989
 426
 427         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
 428         we want all browsers to be doing the same thing on the test.
 429
 430         Reviewed by Maciej
 431
 432         * tests/regexp-dna.js:
 433
 434 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 435
 436         Rubber-stamped by Maciej Stachowiak.
 437
 438         The bleeding edge branch of V8 now supports passing fileanmes after
 439         '-f', so the SunSpider script should be changed to do this again.
 440         This allows all 3 major JavaScript engines to be used with the
 441         unmodified SunSpider script.
 442
 443         * sunspider:
 444
 445 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 446
 447         Reviewed by Maciej Stachowiak.
 448
 449         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
 450         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
 451         correct the punctuation in its documentation.
 452
 453         * sunspider:
 454
 455 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 456
 457         Reviewed by Oliver Hunt.
 458
 459         Bug 20616: Incorporate V8 benchmarks in testing
 460         <https://bugs.webkit.org/show_bug.cgi?id=20616>
 461
 462         This is a first attempt at running the V8 benchmarks within SunSpider,
 463         with the --v8 option. The number of iterations is fixed for each
 464         benchmark, and was chosen to make each of the benchmarks run between
 465         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
 466
 467         * sunspider:
 468         * tests/LIST-V8: Added.
 469         * tests/v8-crypto.js: Added.
 470         * tests/v8-deltablue.js: Added.
 471         * tests/v8-earley-boyer.js: Added.
 472         * tests/v8-raytrace.js: Added.
 473         * tests/v8-richards.js: Added.
 474
 475 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 476
 477         Not reviewed.
 478
 479         Remove a local change that was accidentally committed in r36034.
 480
 481         * resources/sunspider-standalone-driver.js:
 482
 483 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 484
 485         Rubber-stamped by Oliver Hunt.
 486
 487         Make SunSpider work with JavaScript shells that don't take filenames
 488         after a '-f' argument.
 489
 490         * sunspider:
 491
 492 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 493
 494         Reviewed by Timothy.
 495
 496         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
 497
 498         * sunspider:
 499         * tests/LIST-SQUIRRELFISH: Removed.
 500
 501 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
 502
 503         Rubber-stamped by Mark Rowe.
 504
 505         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
 506
 507         * sunspider:
 508
 509 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
 510
 511         Rubber stamped by Oliver.
 512
 513         - add newly working tests to squirrelfish list
 514
 515         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
 516
 517 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
 518
 519         Rubber stamped by Oliver.
 520
 521         - move ubench tests properly.
 522
 523         * tests/ubench: Added.
 524         * tests/ubench/function-closure.js: Added.
 525         * tests/ubench/function-empty.js: Added.
 526         * tests/ubench/function-missing-args.js: Added.
 527         * tests/ubench/function-sum.js: Added.
 528         * tests/ubench/loop-empty-resolve.js: Added.
 529         * tests/ubench/loop-empty.js: Added.
 530         * tests/ubench/loop-sum.js: Added.
 531
 532 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
 533
 534         Rubber stamped by Oliver.
 535
 536         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
 537
 538 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
 539
 540         Reviewed by Oliver.
 541
 542         - make --squirrelfish mode use the real harness, but a reduced set of tests
 543
 544         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
 545
 546         * sunspider:
 547         * tests/LIST-SQUIRRELFISH:
 548         * tests/LIST-UBENCH: Added.
 549         * tests/squirrelfish: Moved to tests/ubench
 550
 551 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
 552
 553         Reviewed by Maciej Stachowiak.
 554
 555         Removed an unused test.
 556
 557         * tests/squirrelfish/loop-resolve.js: Removed.
 558
 559 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
 560
 561         Reviewed by Oliver.
 562
 563         Add newly runnable tests to --squirrelfish mode.
 564
 565         * tests/LIST-SQUIRRELFISH:
 566
 567 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
 568
 569         Reviewed by Maciej Stachowiak.
 570
 571         A little more love for --squirrelfish mode:
 572
 573         Fixed a misplaced sort. Results properly sort now.
 574
 575         Added a test list just for squirrelfish, and updated it to include
 576         all currently passing SunSpider tests.
 577
 578         Fixed the pruning regexp to match 3d-morph.js.
 579
 580         * sunspider:
 581
 582 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
 583
 584         Reviewed by Oliver Hunt.
 585
 586         --squirrelfish mode: pared down tests for squirrelfish to chew on.
 587
 588         * sunspider:
 589         * tests/LIST:
 590         * tests/squirrelfish: Added.
 591         * tests/squirrelfish/function-closure.js: Added.
 592         * tests/squirrelfish/function-empty.js: Added.
 593         * tests/squirrelfish/function-missing-args.js: Added.
 594         * tests/squirrelfish/function-sum.js: Added.
 595         * tests/squirrelfish/loop-empty-resolve.js: Added.
 596         * tests/squirrelfish/loop-empty.js: Added.
 597         * tests/squirrelfish/loop-resolve.js: Added.
 598         * tests/squirrelfish/loop-sum.js: Added.
 599
 600 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
 601
 602         Reviewed by Geoff.
 603
 604         * sunspider: Pause update daemon on Darwin to make results reliable
 605         again (workaround for <rdar://problem/5811127>).
 606
 607 2008-02-07  Eric Seidel  <eric@webkit.org>
 608
 609         Reviewed by Sam.
 610
 611         * sunspider: add an error message when ./tests/LIST is not found
 612
 613 2007-12-29  Alp Toker  <alp@atoker.com>
 614
 615         Fix typo.
 616
 617         * hosted/sunspider.html:
 618
 619 2007-12-29  Alp Toker  <alp@atoker.com>
 620
 621         Reviewed by Maciej.
 622
 623         Provide the current URL in the SunSpider results page. Useful for
 624         copying benchmark results out of browsers without a location bar or
 625         where the location bar can't handle long URLs.
 626
 627         * hosted/sunspider-results.html:
 628
 629 2007-12-20  Eric Seidel  <eric@webkit.org>
 630
 631         Reviewed by mjs.
 632
 633         * sunspider: increase default number of runs for more accurate numbers
 634
 635 2007-12-18  Mark Rowe  <mrowe@apple.com>
 636
 637         Fix typo.
 638
 639         * hosted/sunspider.html:
 640
 641 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
 642
 643         Reviewed by Eric.
 644
 645         - give an extra digit of precision for small differences.
 646
 647         * resources/sunspider-compare-results.js:
 648
 649 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
 650
 651         Reviewed by Eric.
 652
 653         - work around IE layout bugs
 654
 655         * hosted/sunspider.css:
 656
 657 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
 658
 659         Reviewed by Mark.
 660
 661         - fix compare feature
 662
 663         * hosted/sunspider-results.html:
 664
 665 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
 666
 667         Reviewed by Mark.
 668
 669         - Change the format of the output URL to be shorter, to meet IE constraints
 670
 671         * hosted/sunspider-driver.html:
 672         * hosted/sunspider-results.html:
 673
 674 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
 675
 676         Reviewed by Oliver.
 677
 678         - Add some explanatory test and a dumb logo to SunSpider web hosted version
 679
 680         * hosted/sunspider-driver.html:
 681         * hosted/sunspider-results.html:
 682         * hosted/sunspider.css:
 683         * hosted/sunspider.html:
 684
 685 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
 686
 687         Reviewed by Oliver.
 688
 689         - Make the web-hosted version of SunSpider look a little bit less drab.
 690
 691         * hosted/sunspider-driver.html:
 692         * hosted/sunspider-results.html:
 693         * hosted/sunspider.css: Added.
 694         * hosted/sunspider.html:
 695         * resources/TEMPLATE.html:
 696
 697 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
 698
 699         Reviewed by Sam.
 700
 701         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
 702
 703         A sample of what new-style results would look like:
 704
 705         TEST                   COMPARISON            FROM                 TO             DETAILS
 706         ...
 707         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
 708         ...
 709             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
 710
 711         * resources/sunspider-compare-results.js:
 712
 713 2007-11-14  Eric Seidel  <eric@webkit.org>
 714
 715         Reviewed by Sam.
 716
 717         * sunspider: add --shark-cache for L2 Cache Miss profiling
 718
 719 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
 720
 721         Reviewed by Darin Adler.
 722
 723         http://bugs.webkit.org/show_bug.cgi?id=15925
 724         SunSpider should check for Shark being installed
 725
 726         * sunspider: Explicitly test that shark command line tool is installed.
 727
 728 2007-11-04  Darin Adler  <darin@apple.com>
 729
 730         * hosted/sunspider.html: Tweaked formatting a bit.
 731
 732 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
 733
 734         Reviewed by Darin Adler.
 735
 736         - Add compare mode and a description of the benchmark to the hosted version.
 737
 738         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
 739         * hosted/sunspider.html: Added intro text.
 740         * make-hosted: Copy new file.
 741         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
 742         a function.
 743         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
 744         expects.
 745         * sunspider-compare-results: Use sunspider-standalone-compare.js.
 746
 747 2007-11-02  Darin Adler  <darin@apple.com>
 748
 749         * sunspider: Changed "--base" to "--set-baseline".
 750
 751 2007-11-02  Darin Adler  <darin@apple.com>
 752
 753         Reviewed by Maciej.
 754
 755         * sunspider: Added a "--base" option that records the current test as a baseline.
 756         Moved the code to find the newest .mshark file into a function (since I needed to use
 757         it in the other script and I prefer to copy/paste a function).
 758
 759         * sunspider-compare-results: Added logic to compare the baseline with the last
 760         run when you don't pass any parameters.
 761
 762 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 763
 764         Reviewed by Eric.
 765
 766         - Added list of ECMAScript3 features that SunSpider does not cover currently.
 767
 768         * UNCOVERED: Added.
 769
 770 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 771
 772         Reviewed by Oliver.
 773
 774         - Give better error messages when sunspider-compare-results fails
 775
 776         * sunspider-compare-results:
 777
 778 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 779
 780         Reviewed by Darin Adler.
 781
 782         - fix Date tests to work in command-line SpiderMonkey
 783
 784         * tests/date-format-tofte.js:
 785         * tests/date-format-xparb.js:
 786
 787 2007-10-29  Darin Adler  <darin@apple.com>
 788
 789         Reviewed by Maciej.
 790
 791         * sunspider: Added --shark20 option, to run Shark at its highest sample
 792         resolution instead of its default.
 793
 794 2007-10-28  Adam Roben  <aroben@apple.com>
 795
 796         Fix SunSpider on Windows
 797
 798         Reviewed by Eric.
 799
 800         * sunspider: Don't use colons in filenames.
 801
 802 2007-10-28  Darin Adler  <darin@apple.com>
 803
 804         Reviewed by Adam.
 805
 806         * resources/sunspider-analyze-results.js: Tweak the output format a little.
 807         Change so that when there's only one run we don't write out confidence
 808         intervals at all rather than writing "NaN" over and over again.
 809
 810         * sunspider: Use the time and date as a suffix on the results file, that way
 811         all the results are left behind in the tmp directory. This will make it easy
 812         for us to add features that compare past results in the future. Also rename
 813         the .mshark files using the same scheme.
 814
 815         * sunspider-compare-results: Relax the parsing rules so we can parse the
 816         results file exactly as it's emitted from sunspider.
 817
 818 2007-10-25  Adam Roben  <aroben@apple.com>
 819
 820         Strip carriage returns from results
 821
 822         Carriage returns were screwing up sunspider-compare-results on
 823         Windows.
 824
 825         Rubberstamped by Oliver.
 826
 827         * sunspider:
 828
 829 2007-10-24  Darin Adler  <darin@apple.com>
 830
 831         Reviewed by Eric.
 832
 833         * sunspider: Open the profile in shark after completing a run.
 834         * sunspider-compare-results: Tiny perl tweaks.
 835
 836 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 837
 838         Reviewed by Darin Adler.
 839
 840         - Don't hardcode my path to testkjs
 841
 842         * sunspider-compare-results:
 843
 844 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 845
 846         Reviewed by Darin Adler.
 847
 848         - Add a compare script to compare results, and improve formatting of standard results a bit.
 849
 850         * resources/sunspider-analyze-results.js:
 851         * resources/sunspider-compare-results.js: Added.
 852         * sunspider-compare-results: Added.
 853
 854 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 855
 856         Reviewed by Darin Adler.
 857
 858         * resources/sunspider-analyze-results.js: Do the error range
 859         properly, using the t-distribution instead of 1.96 (which was
 860         based on the normal distribution).
 861
 862         * sunspider: Print results in a way that will be friendlier to the
 863         soon-to-come compare mode.
 864
 865 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
 866
 867         Reviewed by Review.
 868
 869         - add more tests (probably the final set for now)
 870
 871         Already balanced. Added date, regexp, control flow, and a few
 872         more string and object/array access tests.
 873
 874         * tests/LIST:
 875         * tests/access-fannkuch.js: Added.
 876         * tests/access-nbody.js: Added.
 877         * tests/controlflow-recursive.js: Added.
 878         * tests/date-format-tofte.js: Added.
 879         * tests/date-format-xparb.js: Added.
 880         * tests/regexp-dna.js: Added.
 881         * tests/string-validate-input.js: Added.
 882
 883 2007-10-21  Eric Seidel  <eric@webkit.org>
 884
 885         Reviewed by mjs.
 886
 887         * sunspider: make --shark only sample from testkjs
 888
 889 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 890
 891         Rubber stamped by Eric.
 892
 893         - added crypto tests (already balanced)
 894
 895         * tests/LIST:
 896         * tests/crypto-aes.js: Added.
 897         * tests/crypto-md5.js: Added.
 898         * tests/crypto-sha1.js: Added.
 899
 900 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 901
 902         Reviewed by Adam.
 903
 904         - rebalance new tests
 905
 906         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
 907
 908         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 909
 910         This balances them with the existing tests.
 911
 912         * tests/access-binary-trees.js:
 913         * tests/access-nsieve.js:
 914         * tests/bitops-nsieve-bits.js:
 915         * tests/math-partial-sums.js:
 916         * tests/math-spectral-norm.js:
 917         * tests/string-fasta.js:
 918
 919 2007-10-20  Eric Seidel  <eric@webkit.org>
 920
 921         Reviewed by darin.
 922
 923         Add improved argument handling to sunspider, including
 924         --runs=<number>, --tests=<pattern, --shark, and --help
 925         Also re-factor code into subroutines
 926
 927         * sunspider: improved argument handling
 928
 929 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 930
 931         Reviewed by Mark.
 932
 933         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
 934
 935         * TODO:
 936         * tests/LIST:
 937         * tests/access-binary-trees.js: Added.
 938         * tests/access-nsieve.js: Added.
 939         * tests/bitops-nsieve-bits.js: Added.
 940         * tests/math-partial-sums.js: Added.
 941         * tests/math-spectral-norm.js: Added.
 942         * tests/string-fasta.js: Added.
 943
 944 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 945
 946         Reviewed by Darin Adler.
 947
 948         * TODO: Updated to note areas that are now well-covered.
 949
 950         - Change the tests and the driver to avoid leaving large object
 951         graphs hanging around, since that throws off the subsequent tests.
 952
 953         * resources/sunspider-standalone-driver.js:
 954         * tests/3d-cube.js:
 955         * tests/3d-morph.js:
 956         * tests/3d-raytrace.js:
 957         * tests/bitops-bitwise-and.js:
 958         * tests/string-base64.js:
 959         * tests/string-tagcloud.js:
 960
 961 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 962
 963         Reviewed by Nikolas.
 964
 965         - Rebalance test complexity.
 966
 967         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
 968
 969         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 970
 971         However, for some tests, IE7 was a huge outlier compared to other
 972         major browsers. For those tests, I used Firefox 2.0.0.8 on the
 973         same OS and hardware as a normalization basis.
 974
 975         * tests/3d-morph.js:
 976         * tests/3d-raytrace.js:
 977         * tests/bitops-3bit-bits-in-byte.js:
 978         * tests/bitops-bits-in-byte.js:
 979         * tests/bitops-bitwise-and.js:
 980         * tests/math-cordic.js:
 981         * tests/string-base64.js:
 982         * tests/string-tagcloud.js:
 983         * tests/string-unpack-code.js:
 984
 985 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 986
 987         Reviewed by Oliver.
 988
 989         - Fix the browser-hosted driver to be compatible with Safari 2
 990
 991         * hosted/sunspider-driver.html:
 992
 993 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 994
 995         Rubber stamped by Hyatt.
 996
 997         - Add BSD license notice to cordic test, the original author said to do so.
 998
 999         * tests/math-cordic.js:
1000
1001 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1002
1003         Reviewed by Eric.
1004
1005         - Support Windows and Windows browsers.
1006
1007         * hosted/sunspider-results.html: Display results correctly in IE.
1008         * sunspider: tweaks to work on Windows, and with spaces in the path.
1009
1010 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1011
1012         Rubber stamped by Oliver.
1013
1014         * tests/string-tagcloud.js: Remove stray print()
1015
1016 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
1017
1018         Reviewed by Oliver.
1019
1020         Add some string test cases.
1021
1022         * tests/string-base64.js: Added. Does base64 encoding and decoding.
1023         * tests/string-tagcloud.js: Added. Parses a JSON string and
1024         generates tagcloud HTML markup based on the data.
1025         * tests/string-unpack-code.js: Added. Unpack compressed versions of
1026         four of the most popular JavaScript libraries.
1027         * tests/LIST: Added the new tests.
1028
1029 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
1030
1031         Reviewed by Anders.
1032
1033         * TODO: Updated with info on how to do the stats properly.
1034
1035         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
1036         avoid overly slowing down the benchmark.
1037
1038         Added some 3D-related test cases.
1039
1040         * tests/3d-cube.js: Added.
1041         * tests/3d-morph.js: Added.
1042         * tests/3d-raytrace.js: Added.
1043         * tests/LIST:
1044
1045 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
1046
1047         Reviewed by Darin Adler.
1048
1049         - New JavaScript benchmark
1050         http://bugs.webkit.org/show_bug.cgi?id=15515
1051
1052         A start on a new JavaScript benchmark, with standalone and
1053         browser-hosted drivers.
1054
1055         The test content is still incomplete and the drivers could use
1056         some more features but there is enough here to make a good start.
1057
1058         The drivers compute a 95% confidence interval on the mean for the
1059         whole test, each category, and each individual test to make it
1060         easier to tell whether differences are statistically
1061         significant. The confidence interval can be narrowed by running
1062         the test more times.
1063
1064         * TODO: Added. Remaining things that need to be done.
1065         * sunspider: Added. Perl script that acts as the standalone test driver.
1066         * resources/sunspider-standalone-driver.js: Added. JavaScript part
1067         of standalone test driver.
1068         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
1069         analysis code.
1070         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
1071         * make-hosted: Added. Script to generate browser-hosted tests.
1072         * hosted/sunspider.html: Added. Start page for browser-hosted test.
1073         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
1074         * hosted/sunspider-results.html: Added. Results page for browser-hosted
1075         tests.
1076         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
1077         tests.
1078         * tests/LIST: Added. List of tests to use.
1079         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
1080         * tests/bitops-bits-in-byte.js: Added. Ditto.
1081         * tests/bitops-bitwise-and.js: Added. Ditto.
1082         * tests/math-cordic.js: Added. Ditto.
1083