PerformanceTests/SunSpider/ChangeLog

   1 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
   2
   3         Update SunSpider versioning to 1.0
   4         https://bugs.webkit.org/show_bug.cgi?id=63870
   5
   6         Reviewed by Dan Bernstein.
   7
   8         * hosted/sunspider.html:
   9         * hosted/versions.html:
  10         * resources/driver-TEMPLATE.html:
  11         * resources/results-TEMPLATE.html:
  12         * sunspider:
  13
  14 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
  15
  16         Sunspider 0.9.1 harness doesn't actually close() its test documents
  17         https://bugs.webkit.org/show_bug.cgi?id=47045
  18
  19         Reviewed by Daniel Bates.
  20
  21         * hosted/versions.html:
  22         * resources/driver-TEMPLATE.html:
  23
  24 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
  25
  26         Not Reviewed.
  27
  28         Fix obviou typo in previous commit.
  29
  30         * tests/sunspider-1.0/bitops-bitwise-and.js:
  31
  32 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
  33
  34         Reviewed by Anders Carlsson.
  35
  36         SunSpider: all four bitops benchmarks can be replaced with NOP
  37         https://bugs.webkit.org/show_bug.cgi?id=38446
  38
  39         * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
  40         (TimeFunc): Save the result.
  41         * tests/sunspider-1.0/bitops-bits-in-byte.js:
  42         (TimeFunc): Save the result.
  43         * tests/sunspider-1.0/bitops-bitwise-and.js: More explicitly
  44         save the results.
  45         * tests/sunspider-1.0/bitops-nsieve-bits.js:
  46         (sieve): Save the result.
  47
  48 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
  49
  50         SunSpider: string-validate-input.js uses an undeclared variable named 'name', which is a DOM API
  51         https://bugs.webkit.org/show_bug.cgi?id=60937
  52
  53         Reviewed by Dan Bernstein.
  54
  55         * tests/sunspider-1.0/string-validate-input.js:
  56         (doTest): Rename the variable to username
  57
  58 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
  59
  60         Reviewed by Darin Adler.
  61
  62         SunSpider: The main function of math-cordic is dead code
  63         https://bugs.webkit.org/show_bug.cgi?id=63863
  64
  65         Try to prevent math-cordic from being eliminated as dead code or
  66         compiled down to a constant.
  67
  68         * tests/sunspider-1.0/math-cordic.js:
  69         (cordicsincos):
  70         (cordic):
  71
  72 2011-06-30  Maciej Stachowiak  <mjs@apple.com>
  73
  74         Reviewed by Adam Barth.
  75
  76         Create sunspider-1.0 directory in preparation for fixing a bunch of bugs
  77         https://bugs.webkit.org/show_bug.cgi?id=63782
  78
  79         * make-hosted: Learn about the new directory.
  80         * tests/sunspider-1.0: Copied from PerformanceTests/SunSpider/tests/sunspider-0.9.1.
  81
  82 2010-11-29  Geoffrey Garen  <ggaren@apple.com>
  83
  84         Reviewed by Gavin Barraclough.
  85
  86         Improved accuracy of command-line SunSpider.
  87
  88         * resources/sunspider-standalone-driver.js: Use the returned number of
  89         elapsed milliseconds from "run" and "checkSyntax" instead of measuring
  90         ourselves, for slightly more accurate numbers.
  91
  92 2010-09-16  Gavin Barraclough  <barraclough@apple.com>
  93
  94         Reviewed by Mark Rowe.
  95         https://bugs.webkit.org/show_bug.cgi?id=45924
  96
  97         Modify the SunSpider harness to allow a path to be passed to the --suite
  98         argument, supress the normal prepend-"tests/" behaviour for suite names
  99         containing a '/'.
 100
 101         Also wrap the code in an anonymous function to move variables out of global
 102         scope (currently the harness cannot handle running tests the overwrite the
 103         variable 'j').
 104
 105         * resources/sunspider-standalone-driver.js:
 106         * sunspider:
 107
 108 2010-08-27  Michael Saboff  <msaboff@apple.com>
 109
 110         Reviewed by Stephanie Lewis.
 111
 112         Added version 5 and version 6 of V8 tests with modifications similar
 113         to the version 4 tests so they'll work with the SunSpider test harness.
 114         Note that these files are external source and therefore don't conform
 115         to WebKit coding standards.
 116         https://bugs.webkit.org/show_bug.cgi?id=44638
 117
 118         * tests/v8-v5: Added.
 119         * tests/v8-v5/LIST: Added.
 120         * tests/v8-v5/v8-crypto.js: Added.
 121         * tests/v8-v5/v8-deltablue.js: Added.
 122         * tests/v8-v5/v8-earley-boyer.js: Added.
 123         * tests/v8-v5/v8-raytrace.js: Added.
 124         * tests/v8-v5/v8-regexp.js: Added.
 125         * tests/v8-v5/v8-richards.js: Added.
 126         * tests/v8-v5/v8-splay.js: Added.
 127         * tests/v8-v6: Added.
 128         * tests/v8-v6/LIST: Added.
 129         * tests/v8-v6/v8-crypto.js: Added.
 130         * tests/v8-v6/v8-deltablue.js: Added.
 131         * tests/v8-v6/v8-earley-boyer.js: Added.
 132         * tests/v8-v6/v8-raytrace.js: Added.
 133         * tests/v8-v6/v8-regexp.js: Added.
 134         * tests/v8-v6/v8-richards.js: Added.
 135         * tests/v8-v6/v8-splay.js: Added.
 136
 137 2010-07-15  Stephanie Lewis  <slewis@apple.com>
 138
 139         Reviewed by Geoff Garen.
 140
 141         http://bugs.webkit.org/show_bug.cgi?id=42406
 142         Output location of results file.
 143
 144         * sunspider:
 145
 146 2010-05-17  Maciej Stachowiak  <mjs@apple.com>
 147
 148         Reviewed by Mark Rowe.
 149
 150         Browser-hosted version of SunSpider runs tests twice as many times as advertised
 151         https://bugs.webkit.org/show_bug.cgi?id=39201
 152
 153         * make-hosted: Make sure to reset test list after every test suite.
 154
 155 2010-05-07  Adam Roben  <aroben@apple.com>
 156
 157         Fix typo in sunspider-compare-results.js that caused incorrect
 158         two-sample t statistics to be computed
 159
 160         Fixes <http://webkit.org/b/38768> SunSpider computes incorrect
 161         two-sample t statistics
 162
 163         Reviewed by Darin Adler.
 164
 165         * resources/sunspider-compare-results.js:
 166         (sunspiderCompareResults.resultLine): Fixed the calculation of the t
 167         variable to match the definition of a two-sample t statistic. Looks
 168         like this was just a simple typo.
 169
 170 2010-04-05  Darin Adler  <darin@apple.com>
 171
 172         Updated the ignore property for changes to the contents here.
 173
 174         * .: Modified property svn:ignore.
 175
 176 2010-02-20  Maciej Stachowiak  <mjs@apple.com>
 177
 178         Reviewed by Adam Barth.
 179
 180         Add sunspider-0.9.1 to the site in preparation for announcing it.
 181         https://bugs.webkit.org/show_bug.cgi?id=35206
 182
 183         * hosted/sunspider.css: Remove tabs, which I noticed where present while
 184         copying to the WebKitSite directory.
 185
 186 2010-01-11  Oliver Hunt  <oliver@apple.com>
 187
 188         Reviewed by Darin Adler.
 189
 190         Sunspider parse-only tests don't run
 191         https://bugs.webkit.org/show_bug.cgi?id=33489
 192
 193         Fix parse-only test list
 194
 195         * tests/parse-only/LIST:
 196
 197 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
 198
 199         Reviewed by Adele Peterson.
 200
 201         Make SunSpider version more prominent in the title
 202         https://bugs.webkit.org/show_bug.cgi?id=32574
 203
 204         * hosted/sunspider.html:
 205         * hosted/versions.html:
 206         * resources/driver-TEMPLATE.html:
 207         * resources/results-TEMPLATE.html:
 208
 209 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
 210
 211         Reviewed by Darin Adler.
 212
 213         Make SunSpider refuse to compare results across content versions
 214         https://bugs.webkit.org/show_bug.cgi?id=32573
 215
 216         The results URL now includes a version indicator, and compare mode
 217         will refuse to compare between versions.
 218
 219         * resources/driver-TEMPLATE.html:
 220         * resources/results-TEMPLATE.html:
 221
 222 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
 223
 224         Reviewed by Darin Adler.
 225
 226         Some Browser-hosted SunSpider files are not valid HTML5
 227         https://bugs.webkit.org/show_bug.cgi?id=32536
 228
 229         Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
 230         close tags and adding a meta charset declaration).
 231
 232         * hosted/sunspider.html:
 233         * hosted/versions.html:
 234         * resources/TEMPLATE.html:
 235         * resources/driver-TEMPLATE.html:
 236         * resources/results-TEMPLATE.html:
 237
 238 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
 239
 240         Reviewed by Eric Seidel.
 241
 242         Make sunspider-0.9.1 the default content set (both command-line and hosted)
 243         https://bugs.webkit.org/show_bug.cgi?id=32537
 244
 245         * hosted/sunspider.html: Change default suite link.
 246         * sunspider: Change default suite.
 247         * sunspider-compare-results: Likewise.
 248
 249 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
 250
 251         Reviewed by Sam Weinig.
 252
 253         in-browser SunSpider suffers excessive penalty under power management
 254         https://bugs.webkit.org/show_bug.cgi?id=32505
 255
 256         I have made a few changes to address this:
 257
 258         1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
 259         document.write to reduce triggering of progress UI.
 260         2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
 261         stepped all the way down the whole time.
 262         3) Run the test cycle an extra time for warmup before the runs that count.
 263
 264         * hosted/sunspider-record-result.js: Removed.
 265         * make-hosted:
 266         * resources/TEMPLATE.html:
 267         * resources/driver-TEMPLATE.html:
 268
 269 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
 270
 271         Reviewed by Gavin Barraclough.
 272
 273         SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
 274         https://bugs.webkit.org/show_bug.cgi?id=16806
 275
 276         Based on a patch by Eric Seidel.
 277
 278         Fix the base64 computation to actually compute correct results. The impact on runtime of
 279         the test is pretty small, but noticeable for some browsers. But at least it's not
 280         doing a wrong and meaningless computation any more.
 281
 282         * tests/sunspider-0.9.1/string-base64.js:
 283         ():
 284         (base64ToString):
 285
 286 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
 287
 288         Fixing commit error...
 289
 290         I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
 291
 292         * resources/driver-TEMPLATE.html:
 293
 294 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
 295
 296         Reviewed by Sam Weinig.
 297
 298         Use JSON.parse instead of eval for Web-hosted SunSpider results processing
 299         https://bugs.webkit.org/show_bug.cgi?id=32490
 300
 301         * hosted/json2.js: Added. Incorporated from json.org
 302         * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
 303         * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
 304         results.
 305
 306 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
 307
 308         Reviewed by Oliver Hunt.
 309
 310         Enable Web-hosted version of SunSpider to handle multiple versions
 311         https://bugs.webkit.org/show_bug.cgi?id=32478
 312
 313         * make-hosted: Now generate the hosted version to be able to run both the
 314         0.9 and the 0.9.1 test suites.
 315         * hosted: Modified property svn:ignore.
 316         * hosted/sunspider-driver.html: Removed.
 317         * hosted/sunspider-results.html: Removed.
 318         * hosted/sunspider.html:
 319         * hosted/versions.html: Added.
 320         * resources/TEMPLATE.html:
 321         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
 322         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
 323
 324 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
 325
 326         Reviewed by Oliver Hunt.
 327
 328         Give command-line sunspider the ability to handle multiple suites and versions
 329         https://bugs.webkit.org/show_bug.cgi?id=32477
 330
 331         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
 332         enumerating the tests. Current suites are:
 333
 334         sunspider-0.9 - riginal SunSpider content
 335         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
 336         ubench - "ubench" microbenchmark suite
 337         v8-v4 - version 4 of the v8 benchmark
 338         parse-only - some parser tests
 339
 340         Sunspider can now be run with the --suite= parameter to select a
 341         suite. The default is --suite=sunspider-0.9.
 342
 343         * .: Modified property svn:ignore to ignore new results directories.
 344         * make-hosted: Generate based on sunspider-0.9 suite.
 345         * resources/TEMPLATE.html: Fix an HTML compliance problem.
 346         * resources/sunspider-standalone-driver.js: Become aware of suite names.
 347         * sunspider: Modified to handle multiple suites as described above.
 348         * sunspider-compare-results: ditto
 349         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
 350         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
 351         * tests/LIST-UBENCH: Removed.
 352         * tests/LIST-V8: Removed.
 353         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
 354         * tests/3d-morph.js: ditto
 355         * tests/3d-raytrace.js: ditto
 356         * tests/access-binary-trees.js: ditto
 357         * tests/access-fannkuch.js: ditto
 358         * tests/access-nbody.js: ditto
 359         * tests/access-nsieve.js: ditto
 360         * tests/bitops-3bit-bits-in-byte.js: ditto
 361         * tests/bitops-bits-in-byte.js: ditto
 362         * tests/bitops-bitwise-and.js: ditto
 363         * tests/bitops-nsieve-bits.js: ditto
 364         * tests/controlflow-recursive.js: ditto
 365         * tests/crypto-aes.js: ditto
 366         * tests/crypto-md5.js: ditto
 367         * tests/crypto-sha1.js: ditto
 368         * tests/date-format-tofte.js: ditto
 369         * tests/date-format-xparb.js: ditto
 370         * tests/math-cordic.js: ditto
 371         * tests/math-partial-sums.js: ditto
 372         * tests/math-spectral-norm.js: ditto
 373         * tests/regexp-dna.js: ditto
 374         * tests/string-base64.js: ditto
 375         * tests/string-fasta.js: ditto
 376         * tests/string-tagcloud.js: ditto
 377         * tests/string-unpack-code.js: ditto
 378         * tests/string-validate-input.js: ditto
 379         * tests/sunspider-0.9/: Added
 380         * tests/sunspider-0.9/*.js: Moved from one level up
 381         * tests/sunspider-0.9.1: Added.
 382         * tests/sunspider-0.9.1/*.js: Moved from one level up.
 383         * tests/v8-crypto.js: Moved to v8-v4/ directory.
 384         * tests/v8-deltablue.js: ditto
 385         * tests/v8-earley-boyer.js: ditto
 386         * tests/v8-raytrace.js: ditto
 387         * tests/v8-regexp.js: ditto
 388         * tests/v8-richards.js: ditto
 389         * tests/v8-splay.js: ditto
 390         * tests/v8-v4: Added.
 391
 392 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
 393
 394         Reviewed by Darin Adler.
 395
 396         Platform specific null device replaced with a platform independent.
 397         https://bugs.webkit.org/show_bug.cgi?id=29544
 398
 399         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
 400
 401 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
 402
 403         Reviewed by Maciej Stachowiak.
 404
 405         [Qt] Make sunspider script work on Windows platform.
 406         https://bugs.webkit.org/show_bug.cgi?id=29544
 407
 408         * sunspider:
 409         Using platform specific null device instead of /dev/null.
 410
 411 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
 412
 413         Reviewed by Oliver Hunt.
 414
 415         https://bugs.webkit.org/show_bug.cgi?id=26540
 416         Currently the SunSpider test driver lacks an option to run a test suite that
 417         will test JavaScriptCore parsing performance only.  This patch adds just such
 418         a test suite and option to SunSpider as well as the jsc test shell.  I've included
 419         three large javascript source files found in the wild: jquery, mootools and prototype.
 420         Combined with the concatenation of all three, these form a new testsuite to measure
 421         and test pure JavaScriptCore parsing performance.
 422
 423         * resources/sunspider-standalone-driver.js:
 424           Load the js with parse-only flag if it is part of the parse-only suite.
 425         * sunspider:
 426           Add the parse-only suite to the list of options.
 427         * sunspider-compare-results:
 428           Add the parse-only suite to the list of options.
 429         * tests/LIST-PARSE-ONLY: Added.
 430         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
 431           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
 432         * tests/parse-only/jquery-1.3.2.js: Added.
 433           Latest version from jquery.com.
 434         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
 435           Latest version from mootools.net.
 436         * tests/parse-only/prototype-1.6.0.3.js: Added.
 437           Latest version from prototypejs.org.
 438
 439 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
 440
 441         Update V8 benchmark to version 4.
 442
 443         Reviewed by Cameron Zwarich.
 444
 445         * tests/LIST-V8:
 446         * tests/v8-crypto.js:
 447         * tests/v8-deltablue.js:
 448         * tests/v8-earley-boyer.js:
 449         * tests/v8-raytrace.js:
 450         * tests/v8-regexp.js: Added.
 451         * tests/v8-richards.js:
 452         * tests/v8-splay.js: Added.
 453
 454 2009-05-26  Olivier DOLE  <odole@pleyo.com>
 455
 456         Reviewed by Maciej.
 457
 458         Store sunspider test suite results in a specific directory for each
 459         different type of benchmark (sunspider, V8 or ubench).
 460
 461         * sunspider:
 462         * sunspider-compare-results:
 463
 464 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
 465
 466         Build fix.
 467
 468         * SunSpider.make:
 469
 470 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
 471
 472         Include SunSpider tests in production builds.
 473
 474         Reviewed by Adam Roben.
 475
 476         * SunSpider.make: Added.
 477         * xcopy.excludes: Added.
 478
 479 2009-03-04  Oliver Hunt  <oliver@apple.com>
 480
 481         Added a few more tests to ubench
 482
 483         RS=Gavin Barraclough
 484
 485         Adding a couple of test to cover too few and too many arguments.  This
 486         is needed to prevent regressions in all call paths from my current refactoring.
 487
 488         * tests/LIST-UBENCH:
 489         * tests/ubench/function-correct-args.js: Added.
 490         (f):
 491         * tests/ubench/function-excess-args.js: Added.
 492         (f):
 493
 494 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
 495
 496         Fix a bug where when V8's or UBENCH's test suite list file is missing
 497         the error message is suggesting that SunSpider's list file is missing.
 498
 499         Reviewed by Sam Weinig.
 500
 501         * sunspider:
 502         (loadTestsList): Fix the error message to use the right list name.
 503
 504 2008-10-07  David Hyatt  <hyatt@apple.com>
 505
 506         https://bugs.webkit.org/show_bug.cgi?id=18989
 507
 508         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
 509         we want all browsers to be doing the same thing on the test.
 510
 511         Reviewed by Maciej
 512
 513         * tests/regexp-dna.js:
 514
 515 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 516
 517         Rubber-stamped by Maciej Stachowiak.
 518
 519         The bleeding edge branch of V8 now supports passing fileanmes after
 520         '-f', so the SunSpider script should be changed to do this again.
 521         This allows all 3 major JavaScript engines to be used with the
 522         unmodified SunSpider script.
 523
 524         * sunspider:
 525
 526 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 527
 528         Reviewed by Maciej Stachowiak.
 529
 530         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
 531         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
 532         correct the punctuation in its documentation.
 533
 534         * sunspider:
 535
 536 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 537
 538         Reviewed by Oliver Hunt.
 539
 540         Bug 20616: Incorporate V8 benchmarks in testing
 541         <https://bugs.webkit.org/show_bug.cgi?id=20616>
 542
 543         This is a first attempt at running the V8 benchmarks within SunSpider,
 544         with the --v8 option. The number of iterations is fixed for each
 545         benchmark, and was chosen to make each of the benchmarks run between
 546         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
 547
 548         * sunspider:
 549         * tests/LIST-V8: Added.
 550         * tests/v8-crypto.js: Added.
 551         * tests/v8-deltablue.js: Added.
 552         * tests/v8-earley-boyer.js: Added.
 553         * tests/v8-raytrace.js: Added.
 554         * tests/v8-richards.js: Added.
 555
 556 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 557
 558         Not reviewed.
 559
 560         Remove a local change that was accidentally committed in r36034.
 561
 562         * resources/sunspider-standalone-driver.js:
 563
 564 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 565
 566         Rubber-stamped by Oliver Hunt.
 567
 568         Make SunSpider work with JavaScript shells that don't take filenames
 569         after a '-f' argument.
 570
 571         * sunspider:
 572
 573 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 574
 575         Reviewed by Timothy.
 576
 577         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
 578
 579         * sunspider:
 580         * tests/LIST-SQUIRRELFISH: Removed.
 581
 582 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
 583
 584         Rubber-stamped by Mark Rowe.
 585
 586         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
 587
 588         * sunspider:
 589
 590 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
 591
 592         Rubber stamped by Oliver.
 593
 594         - add newly working tests to squirrelfish list
 595
 596         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
 597
 598 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
 599
 600         Rubber stamped by Oliver.
 601
 602         - move ubench tests properly.
 603
 604         * tests/ubench: Added.
 605         * tests/ubench/function-closure.js: Added.
 606         * tests/ubench/function-empty.js: Added.
 607         * tests/ubench/function-missing-args.js: Added.
 608         * tests/ubench/function-sum.js: Added.
 609         * tests/ubench/loop-empty-resolve.js: Added.
 610         * tests/ubench/loop-empty.js: Added.
 611         * tests/ubench/loop-sum.js: Added.
 612
 613 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
 614
 615         Rubber stamped by Oliver.
 616
 617         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
 618
 619 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
 620
 621         Reviewed by Oliver.
 622
 623         - make --squirrelfish mode use the real harness, but a reduced set of tests
 624
 625         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
 626
 627         * sunspider:
 628         * tests/LIST-SQUIRRELFISH:
 629         * tests/LIST-UBENCH: Added.
 630         * tests/squirrelfish: Moved to tests/ubench
 631
 632 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
 633
 634         Reviewed by Maciej Stachowiak.
 635
 636         Removed an unused test.
 637
 638         * tests/squirrelfish/loop-resolve.js: Removed.
 639
 640 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
 641
 642         Reviewed by Oliver.
 643
 644         Add newly runnable tests to --squirrelfish mode.
 645
 646         * tests/LIST-SQUIRRELFISH:
 647
 648 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
 649
 650         Reviewed by Maciej Stachowiak.
 651
 652         A little more love for --squirrelfish mode:
 653
 654         Fixed a misplaced sort. Results properly sort now.
 655
 656         Added a test list just for squirrelfish, and updated it to include
 657         all currently passing SunSpider tests.
 658
 659         Fixed the pruning regexp to match 3d-morph.js.
 660
 661         * sunspider:
 662
 663 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
 664
 665         Reviewed by Oliver Hunt.
 666
 667         --squirrelfish mode: pared down tests for squirrelfish to chew on.
 668
 669         * sunspider:
 670         * tests/LIST:
 671         * tests/squirrelfish: Added.
 672         * tests/squirrelfish/function-closure.js: Added.
 673         * tests/squirrelfish/function-empty.js: Added.
 674         * tests/squirrelfish/function-missing-args.js: Added.
 675         * tests/squirrelfish/function-sum.js: Added.
 676         * tests/squirrelfish/loop-empty-resolve.js: Added.
 677         * tests/squirrelfish/loop-empty.js: Added.
 678         * tests/squirrelfish/loop-resolve.js: Added.
 679         * tests/squirrelfish/loop-sum.js: Added.
 680
 681 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
 682
 683         Reviewed by Geoff.
 684
 685         * sunspider: Pause update daemon on Darwin to make results reliable
 686         again (workaround for <rdar://problem/5811127>).
 687
 688 2008-02-07  Eric Seidel  <eric@webkit.org>
 689
 690         Reviewed by Sam.
 691
 692         * sunspider: add an error message when ./tests/LIST is not found
 693
 694 2007-12-29  Alp Toker  <alp@atoker.com>
 695
 696         Fix typo.
 697
 698         * hosted/sunspider.html:
 699
 700 2007-12-29  Alp Toker  <alp@atoker.com>
 701
 702         Reviewed by Maciej.
 703
 704         Provide the current URL in the SunSpider results page. Useful for
 705         copying benchmark results out of browsers without a location bar or
 706         where the location bar can't handle long URLs.
 707
 708         * hosted/sunspider-results.html:
 709
 710 2007-12-20  Eric Seidel  <eric@webkit.org>
 711
 712         Reviewed by mjs.
 713
 714         * sunspider: increase default number of runs for more accurate numbers
 715
 716 2007-12-18  Mark Rowe  <mrowe@apple.com>
 717
 718         Fix typo.
 719
 720         * hosted/sunspider.html:
 721
 722 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
 723
 724         Reviewed by Eric.
 725
 726         - give an extra digit of precision for small differences.
 727
 728         * resources/sunspider-compare-results.js:
 729
 730 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
 731
 732         Reviewed by Eric.
 733
 734         - work around IE layout bugs
 735
 736         * hosted/sunspider.css:
 737
 738 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
 739
 740         Reviewed by Mark.
 741
 742         - fix compare feature
 743
 744         * hosted/sunspider-results.html:
 745
 746 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
 747
 748         Reviewed by Mark.
 749
 750         - Change the format of the output URL to be shorter, to meet IE constraints
 751
 752         * hosted/sunspider-driver.html:
 753         * hosted/sunspider-results.html:
 754
 755 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
 756
 757         Reviewed by Oliver.
 758
 759         - Add some explanatory test and a dumb logo to SunSpider web hosted version
 760
 761         * hosted/sunspider-driver.html:
 762         * hosted/sunspider-results.html:
 763         * hosted/sunspider.css:
 764         * hosted/sunspider.html:
 765
 766 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
 767
 768         Reviewed by Oliver.
 769
 770         - Make the web-hosted version of SunSpider look a little bit less drab.
 771
 772         * hosted/sunspider-driver.html:
 773         * hosted/sunspider-results.html:
 774         * hosted/sunspider.css: Added.
 775         * hosted/sunspider.html:
 776         * resources/TEMPLATE.html:
 777
 778 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
 779
 780         Reviewed by Sam.
 781
 782         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
 783
 784         A sample of what new-style results would look like:
 785
 786         TEST                   COMPARISON            FROM                 TO             DETAILS
 787         ...
 788         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
 789         ...
 790             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
 791
 792         * resources/sunspider-compare-results.js:
 793
 794 2007-11-14  Eric Seidel  <eric@webkit.org>
 795
 796         Reviewed by Sam.
 797
 798         * sunspider: add --shark-cache for L2 Cache Miss profiling
 799
 800 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
 801
 802         Reviewed by Darin Adler.
 803
 804         http://bugs.webkit.org/show_bug.cgi?id=15925
 805         SunSpider should check for Shark being installed
 806
 807         * sunspider: Explicitly test that shark command line tool is installed.
 808
 809 2007-11-04  Darin Adler  <darin@apple.com>
 810
 811         * hosted/sunspider.html: Tweaked formatting a bit.
 812
 813 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
 814
 815         Reviewed by Darin Adler.
 816
 817         - Add compare mode and a description of the benchmark to the hosted version.
 818
 819         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
 820         * hosted/sunspider.html: Added intro text.
 821         * make-hosted: Copy new file.
 822         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
 823         a function.
 824         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
 825         expects.
 826         * sunspider-compare-results: Use sunspider-standalone-compare.js.
 827
 828 2007-11-02  Darin Adler  <darin@apple.com>
 829
 830         * sunspider: Changed "--base" to "--set-baseline".
 831
 832 2007-11-02  Darin Adler  <darin@apple.com>
 833
 834         Reviewed by Maciej.
 835
 836         * sunspider: Added a "--base" option that records the current test as a baseline.
 837         Moved the code to find the newest .mshark file into a function (since I needed to use
 838         it in the other script and I prefer to copy/paste a function).
 839
 840         * sunspider-compare-results: Added logic to compare the baseline with the last
 841         run when you don't pass any parameters.
 842
 843 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 844
 845         Reviewed by Eric.
 846
 847         - Added list of ECMAScript3 features that SunSpider does not cover currently.
 848
 849         * UNCOVERED: Added.
 850
 851 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 852
 853         Reviewed by Oliver.
 854
 855         - Give better error messages when sunspider-compare-results fails
 856
 857         * sunspider-compare-results:
 858
 859 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 860
 861         Reviewed by Darin Adler.
 862
 863         - fix Date tests to work in command-line SpiderMonkey
 864
 865         * tests/date-format-tofte.js:
 866         * tests/date-format-xparb.js:
 867
 868 2007-10-29  Darin Adler  <darin@apple.com>
 869
 870         Reviewed by Maciej.
 871
 872         * sunspider: Added --shark20 option, to run Shark at its highest sample
 873         resolution instead of its default.
 874
 875 2007-10-28  Adam Roben  <aroben@apple.com>
 876
 877         Fix SunSpider on Windows
 878
 879         Reviewed by Eric.
 880
 881         * sunspider: Don't use colons in filenames.
 882
 883 2007-10-28  Darin Adler  <darin@apple.com>
 884
 885         Reviewed by Adam.
 886
 887         * resources/sunspider-analyze-results.js: Tweak the output format a little.
 888         Change so that when there's only one run we don't write out confidence
 889         intervals at all rather than writing "NaN" over and over again.
 890
 891         * sunspider: Use the time and date as a suffix on the results file, that way
 892         all the results are left behind in the tmp directory. This will make it easy
 893         for us to add features that compare past results in the future. Also rename
 894         the .mshark files using the same scheme.
 895
 896         * sunspider-compare-results: Relax the parsing rules so we can parse the
 897         results file exactly as it's emitted from sunspider.
 898
 899 2007-10-25  Adam Roben  <aroben@apple.com>
 900
 901         Strip carriage returns from results
 902
 903         Carriage returns were screwing up sunspider-compare-results on
 904         Windows.
 905
 906         Rubberstamped by Oliver.
 907
 908         * sunspider:
 909
 910 2007-10-24  Darin Adler  <darin@apple.com>
 911
 912         Reviewed by Eric.
 913
 914         * sunspider: Open the profile in shark after completing a run.
 915         * sunspider-compare-results: Tiny perl tweaks.
 916
 917 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 918
 919         Reviewed by Darin Adler.
 920
 921         - Don't hardcode my path to testkjs
 922
 923         * sunspider-compare-results:
 924
 925 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 926
 927         Reviewed by Darin Adler.
 928
 929         - Add a compare script to compare results, and improve formatting of standard results a bit.
 930
 931         * resources/sunspider-analyze-results.js:
 932         * resources/sunspider-compare-results.js: Added.
 933         * sunspider-compare-results: Added.
 934
 935 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 936
 937         Reviewed by Darin Adler.
 938
 939         * resources/sunspider-analyze-results.js: Do the error range
 940         properly, using the t-distribution instead of 1.96 (which was
 941         based on the normal distribution).
 942
 943         * sunspider: Print results in a way that will be friendlier to the
 944         soon-to-come compare mode.
 945
 946 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
 947
 948         Reviewed by Review.
 949
 950         - add more tests (probably the final set for now)
 951
 952         Already balanced. Added date, regexp, control flow, and a few
 953         more string and object/array access tests.
 954
 955         * tests/LIST:
 956         * tests/access-fannkuch.js: Added.
 957         * tests/access-nbody.js: Added.
 958         * tests/controlflow-recursive.js: Added.
 959         * tests/date-format-tofte.js: Added.
 960         * tests/date-format-xparb.js: Added.
 961         * tests/regexp-dna.js: Added.
 962         * tests/string-validate-input.js: Added.
 963
 964 2007-10-21  Eric Seidel  <eric@webkit.org>
 965
 966         Reviewed by mjs.
 967
 968         * sunspider: make --shark only sample from testkjs
 969
 970 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 971
 972         Rubber stamped by Eric.
 973
 974         - added crypto tests (already balanced)
 975
 976         * tests/LIST:
 977         * tests/crypto-aes.js: Added.
 978         * tests/crypto-md5.js: Added.
 979         * tests/crypto-sha1.js: Added.
 980
 981 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 982
 983         Reviewed by Adam.
 984
 985         - rebalance new tests
 986
 987         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
 988
 989         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 990
 991         This balances them with the existing tests.
 992
 993         * tests/access-binary-trees.js:
 994         * tests/access-nsieve.js:
 995         * tests/bitops-nsieve-bits.js:
 996         * tests/math-partial-sums.js:
 997         * tests/math-spectral-norm.js:
 998         * tests/string-fasta.js:
 999
1000 2007-10-20  Eric Seidel  <eric@webkit.org>
1001
1002         Reviewed by darin.
1003
1004         Add improved argument handling to sunspider, including
1005         --runs=<number>, --tests=<pattern, --shark, and --help
1006         Also re-factor code into subroutines
1007
1008         * sunspider: improved argument handling
1009
1010 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1011
1012         Reviewed by Mark.
1013
1014         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
1015
1016         * TODO:
1017         * tests/LIST:
1018         * tests/access-binary-trees.js: Added.
1019         * tests/access-nsieve.js: Added.
1020         * tests/bitops-nsieve-bits.js: Added.
1021         * tests/math-partial-sums.js: Added.
1022         * tests/math-spectral-norm.js: Added.
1023         * tests/string-fasta.js: Added.
1024
1025 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1026
1027         Reviewed by Darin Adler.
1028
1029         * TODO: Updated to note areas that are now well-covered.
1030
1031         - Change the tests and the driver to avoid leaving large object
1032         graphs hanging around, since that throws off the subsequent tests.
1033
1034         * resources/sunspider-standalone-driver.js:
1035         * tests/3d-cube.js:
1036         * tests/3d-morph.js:
1037         * tests/3d-raytrace.js:
1038         * tests/bitops-bitwise-and.js:
1039         * tests/string-base64.js:
1040         * tests/string-tagcloud.js:
1041
1042 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1043
1044         Reviewed by Nikolas.
1045
1046         - Rebalance test complexity.
1047
1048         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
1049
1050         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1051
1052         However, for some tests, IE7 was a huge outlier compared to other
1053         major browsers. For those tests, I used Firefox 2.0.0.8 on the
1054         same OS and hardware as a normalization basis.
1055
1056         * tests/3d-morph.js:
1057         * tests/3d-raytrace.js:
1058         * tests/bitops-3bit-bits-in-byte.js:
1059         * tests/bitops-bits-in-byte.js:
1060         * tests/bitops-bitwise-and.js:
1061         * tests/math-cordic.js:
1062         * tests/string-base64.js:
1063         * tests/string-tagcloud.js:
1064         * tests/string-unpack-code.js:
1065
1066 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1067
1068         Reviewed by Oliver.
1069
1070         - Fix the browser-hosted driver to be compatible with Safari 2
1071
1072         * hosted/sunspider-driver.html:
1073
1074 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1075
1076         Rubber stamped by Hyatt.
1077
1078         - Add BSD license notice to cordic test, the original author said to do so.
1079
1080         * tests/math-cordic.js:
1081
1082 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1083
1084         Reviewed by Eric.
1085
1086         - Support Windows and Windows browsers.
1087
1088         * hosted/sunspider-results.html: Display results correctly in IE.
1089         * sunspider: tweaks to work on Windows, and with spaces in the path.
1090
1091 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1092
1093         Rubber stamped by Oliver.
1094
1095         * tests/string-tagcloud.js: Remove stray print()
1096
1097 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
1098
1099         Reviewed by Oliver.
1100
1101         Add some string test cases.
1102
1103         * tests/string-base64.js: Added. Does base64 encoding and decoding.
1104         * tests/string-tagcloud.js: Added. Parses a JSON string and
1105         generates tagcloud HTML markup based on the data.
1106         * tests/string-unpack-code.js: Added. Unpack compressed versions of
1107         four of the most popular JavaScript libraries.
1108         * tests/LIST: Added the new tests.
1109
1110 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
1111
1112         Reviewed by Anders.
1113
1114         * TODO: Updated with info on how to do the stats properly.
1115
1116         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
1117         avoid overly slowing down the benchmark.
1118
1119         Added some 3D-related test cases.
1120
1121         * tests/3d-cube.js: Added.
1122         * tests/3d-morph.js: Added.
1123         * tests/3d-raytrace.js: Added.
1124         * tests/LIST:
1125
1126 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
1127
1128         Reviewed by Darin Adler.
1129
1130         - New JavaScript benchmark
1131         http://bugs.webkit.org/show_bug.cgi?id=15515
1132
1133         A start on a new JavaScript benchmark, with standalone and
1134         browser-hosted drivers.
1135
1136         The test content is still incomplete and the drivers could use
1137         some more features but there is enough here to make a good start.
1138
1139         The drivers compute a 95% confidence interval on the mean for the
1140         whole test, each category, and each individual test to make it
1141         easier to tell whether differences are statistically
1142         significant. The confidence interval can be narrowed by running
1143         the test more times.
1144
1145         * TODO: Added. Remaining things that need to be done.
1146         * sunspider: Added. Perl script that acts as the standalone test driver.
1147         * resources/sunspider-standalone-driver.js: Added. JavaScript part
1148         of standalone test driver.
1149         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
1150         analysis code.
1151         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
1152         * make-hosted: Added. Script to generate browser-hosted tests.
1153         * hosted/sunspider.html: Added. Start page for browser-hosted test.
1154         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
1155         * hosted/sunspider-results.html: Added. Results page for browser-hosted
1156         tests.
1157         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
1158         tests.
1159         * tests/LIST: Added. List of tests to use.
1160         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
1161         * tests/bitops-bits-in-byte.js: Added. Ditto.
1162         * tests/bitops-bitwise-and.js: Added. Ditto.
1163         * tests/math-cordic.js: Added. Ditto.
1164