PerformanceTests/SunSpider/ChangeLog

   1 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
   2
   3         SunSpider: string-validate-input.js uses an undeclared variable named 'name', which is a DOM API
   4         https://bugs.webkit.org/show_bug.cgi?id=60937
   5
   6         Reviewed by Dan Bernstein.
   7
   8         * tests/sunspider-1.0/string-validate-input.js:
   9         (doTest): Rename the variable to username
  10
  11 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
  12
  13         Reviewed by Darin Adler.
  14
  15         SunSpider: The main function of math-cordic is dead code
  16         https://bugs.webkit.org/show_bug.cgi?id=63863
  17
  18         Try to prevent math-cordic from being eliminated as dead code or
  19         compiled down to a constant.
  20
  21         * tests/sunspider-1.0/math-cordic.js:
  22         (cordicsincos):
  23         (cordic):
  24
  25 2011-06-30  Maciej Stachowiak  <mjs@apple.com>
  26
  27         Reviewed by Adam Barth.
  28
  29         Create sunspider-1.0 directory in preparation for fixing a bunch of bugs
  30         https://bugs.webkit.org/show_bug.cgi?id=63782
  31
  32         * make-hosted: Learn about the new directory.
  33         * tests/sunspider-1.0: Copied from PerformanceTests/SunSpider/tests/sunspider-0.9.1.
  34
  35 2010-11-29  Geoffrey Garen  <ggaren@apple.com>
  36
  37         Reviewed by Gavin Barraclough.
  38
  39         Improved accuracy of command-line SunSpider.
  40
  41         * resources/sunspider-standalone-driver.js: Use the returned number of
  42         elapsed milliseconds from "run" and "checkSyntax" instead of measuring
  43         ourselves, for slightly more accurate numbers.
  44
  45 2010-09-16  Gavin Barraclough  <barraclough@apple.com>
  46
  47         Reviewed by Mark Rowe.
  48         https://bugs.webkit.org/show_bug.cgi?id=45924
  49
  50         Modify the SunSpider harness to allow a path to be passed to the --suite
  51         argument, supress the normal prepend-"tests/" behaviour for suite names
  52         containing a '/'.
  53
  54         Also wrap the code in an anonymous function to move variables out of global
  55         scope (currently the harness cannot handle running tests the overwrite the
  56         variable 'j').
  57
  58         * resources/sunspider-standalone-driver.js:
  59         * sunspider:
  60
  61 2010-08-27  Michael Saboff  <msaboff@apple.com>
  62
  63         Reviewed by Stephanie Lewis.
  64
  65         Added version 5 and version 6 of V8 tests with modifications similar
  66         to the version 4 tests so they'll work with the SunSpider test harness.
  67         Note that these files are external source and therefore don't conform
  68         to WebKit coding standards.
  69         https://bugs.webkit.org/show_bug.cgi?id=44638
  70
  71         * tests/v8-v5: Added.
  72         * tests/v8-v5/LIST: Added.
  73         * tests/v8-v5/v8-crypto.js: Added.
  74         * tests/v8-v5/v8-deltablue.js: Added.
  75         * tests/v8-v5/v8-earley-boyer.js: Added.
  76         * tests/v8-v5/v8-raytrace.js: Added.
  77         * tests/v8-v5/v8-regexp.js: Added.
  78         * tests/v8-v5/v8-richards.js: Added.
  79         * tests/v8-v5/v8-splay.js: Added.
  80         * tests/v8-v6: Added.
  81         * tests/v8-v6/LIST: Added.
  82         * tests/v8-v6/v8-crypto.js: Added.
  83         * tests/v8-v6/v8-deltablue.js: Added.
  84         * tests/v8-v6/v8-earley-boyer.js: Added.
  85         * tests/v8-v6/v8-raytrace.js: Added.
  86         * tests/v8-v6/v8-regexp.js: Added.
  87         * tests/v8-v6/v8-richards.js: Added.
  88         * tests/v8-v6/v8-splay.js: Added.
  89
  90 2010-07-15  Stephanie Lewis  <slewis@apple.com>
  91
  92         Reviewed by Geoff Garen.
  93
  94         http://bugs.webkit.org/show_bug.cgi?id=42406
  95         Output location of results file.
  96
  97         * sunspider:
  98
  99 2010-05-17  Maciej Stachowiak  <mjs@apple.com>
 100
 101         Reviewed by Mark Rowe.
 102
 103         Browser-hosted version of SunSpider runs tests twice as many times as advertised
 104         https://bugs.webkit.org/show_bug.cgi?id=39201
 105
 106         * make-hosted: Make sure to reset test list after every test suite.
 107
 108 2010-05-07  Adam Roben  <aroben@apple.com>
 109
 110         Fix typo in sunspider-compare-results.js that caused incorrect
 111         two-sample t statistics to be computed
 112
 113         Fixes <http://webkit.org/b/38768> SunSpider computes incorrect
 114         two-sample t statistics
 115
 116         Reviewed by Darin Adler.
 117
 118         * resources/sunspider-compare-results.js:
 119         (sunspiderCompareResults.resultLine): Fixed the calculation of the t
 120         variable to match the definition of a two-sample t statistic. Looks
 121         like this was just a simple typo.
 122
 123 2010-04-05  Darin Adler  <darin@apple.com>
 124
 125         Updated the ignore property for changes to the contents here.
 126
 127         * .: Modified property svn:ignore.
 128
 129 2010-02-20  Maciej Stachowiak  <mjs@apple.com>
 130
 131         Reviewed by Adam Barth.
 132
 133         Add sunspider-0.9.1 to the site in preparation for announcing it.
 134         https://bugs.webkit.org/show_bug.cgi?id=35206
 135
 136         * hosted/sunspider.css: Remove tabs, which I noticed where present while
 137         copying to the WebKitSite directory.
 138
 139 2010-01-11  Oliver Hunt  <oliver@apple.com>
 140
 141         Reviewed by Darin Adler.
 142
 143         Sunspider parse-only tests don't run
 144         https://bugs.webkit.org/show_bug.cgi?id=33489
 145
 146         Fix parse-only test list
 147
 148         * tests/parse-only/LIST:
 149
 150 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
 151
 152         Reviewed by Adele Peterson.
 153
 154         Make SunSpider version more prominent in the title
 155         https://bugs.webkit.org/show_bug.cgi?id=32574
 156
 157         * hosted/sunspider.html:
 158         * hosted/versions.html:
 159         * resources/driver-TEMPLATE.html:
 160         * resources/results-TEMPLATE.html:
 161
 162 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
 163
 164         Reviewed by Darin Adler.
 165
 166         Make SunSpider refuse to compare results across content versions
 167         https://bugs.webkit.org/show_bug.cgi?id=32573
 168
 169         The results URL now includes a version indicator, and compare mode
 170         will refuse to compare between versions.
 171
 172         * resources/driver-TEMPLATE.html:
 173         * resources/results-TEMPLATE.html:
 174
 175 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
 176
 177         Reviewed by Darin Adler.
 178
 179         Some Browser-hosted SunSpider files are not valid HTML5
 180         https://bugs.webkit.org/show_bug.cgi?id=32536
 181
 182         Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
 183         close tags and adding a meta charset declaration).
 184
 185         * hosted/sunspider.html:
 186         * hosted/versions.html:
 187         * resources/TEMPLATE.html:
 188         * resources/driver-TEMPLATE.html:
 189         * resources/results-TEMPLATE.html:
 190
 191 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
 192
 193         Reviewed by Eric Seidel.
 194
 195         Make sunspider-0.9.1 the default content set (both command-line and hosted)
 196         https://bugs.webkit.org/show_bug.cgi?id=32537
 197
 198         * hosted/sunspider.html: Change default suite link.
 199         * sunspider: Change default suite.
 200         * sunspider-compare-results: Likewise.
 201
 202 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
 203
 204         Reviewed by Sam Weinig.
 205
 206         in-browser SunSpider suffers excessive penalty under power management
 207         https://bugs.webkit.org/show_bug.cgi?id=32505
 208
 209         I have made a few changes to address this:
 210
 211         1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
 212         document.write to reduce triggering of progress UI.
 213         2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
 214         stepped all the way down the whole time.
 215         3) Run the test cycle an extra time for warmup before the runs that count.
 216
 217         * hosted/sunspider-record-result.js: Removed.
 218         * make-hosted:
 219         * resources/TEMPLATE.html:
 220         * resources/driver-TEMPLATE.html:
 221
 222 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
 223
 224         Reviewed by Gavin Barraclough.
 225
 226         SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
 227         https://bugs.webkit.org/show_bug.cgi?id=16806
 228
 229         Based on a patch by Eric Seidel.
 230
 231         Fix the base64 computation to actually compute correct results. The impact on runtime of
 232         the test is pretty small, but noticeable for some browsers. But at least it's not
 233         doing a wrong and meaningless computation any more.
 234
 235         * tests/sunspider-0.9.1/string-base64.js:
 236         ():
 237         (base64ToString):
 238
 239 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
 240
 241         Fixing commit error...
 242
 243         I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
 244
 245         * resources/driver-TEMPLATE.html:
 246
 247 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
 248
 249         Reviewed by Sam Weinig.
 250
 251         Use JSON.parse instead of eval for Web-hosted SunSpider results processing
 252         https://bugs.webkit.org/show_bug.cgi?id=32490
 253
 254         * hosted/json2.js: Added. Incorporated from json.org
 255         * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
 256         * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
 257         results.
 258
 259 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
 260
 261         Reviewed by Oliver Hunt.
 262
 263         Enable Web-hosted version of SunSpider to handle multiple versions
 264         https://bugs.webkit.org/show_bug.cgi?id=32478
 265
 266         * make-hosted: Now generate the hosted version to be able to run both the
 267         0.9 and the 0.9.1 test suites.
 268         * hosted: Modified property svn:ignore.
 269         * hosted/sunspider-driver.html: Removed.
 270         * hosted/sunspider-results.html: Removed.
 271         * hosted/sunspider.html:
 272         * hosted/versions.html: Added.
 273         * resources/TEMPLATE.html:
 274         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
 275         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
 276
 277 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
 278
 279         Reviewed by Oliver Hunt.
 280
 281         Give command-line sunspider the ability to handle multiple suites and versions
 282         https://bugs.webkit.org/show_bug.cgi?id=32477
 283
 284         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
 285         enumerating the tests. Current suites are:
 286
 287         sunspider-0.9 - riginal SunSpider content
 288         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
 289         ubench - "ubench" microbenchmark suite
 290         v8-v4 - version 4 of the v8 benchmark
 291         parse-only - some parser tests
 292
 293         Sunspider can now be run with the --suite= parameter to select a
 294         suite. The default is --suite=sunspider-0.9.
 295
 296         * .: Modified property svn:ignore to ignore new results directories.
 297         * make-hosted: Generate based on sunspider-0.9 suite.
 298         * resources/TEMPLATE.html: Fix an HTML compliance problem.
 299         * resources/sunspider-standalone-driver.js: Become aware of suite names.
 300         * sunspider: Modified to handle multiple suites as described above.
 301         * sunspider-compare-results: ditto
 302         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
 303         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
 304         * tests/LIST-UBENCH: Removed.
 305         * tests/LIST-V8: Removed.
 306         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
 307         * tests/3d-morph.js: ditto
 308         * tests/3d-raytrace.js: ditto
 309         * tests/access-binary-trees.js: ditto
 310         * tests/access-fannkuch.js: ditto
 311         * tests/access-nbody.js: ditto
 312         * tests/access-nsieve.js: ditto
 313         * tests/bitops-3bit-bits-in-byte.js: ditto
 314         * tests/bitops-bits-in-byte.js: ditto
 315         * tests/bitops-bitwise-and.js: ditto
 316         * tests/bitops-nsieve-bits.js: ditto
 317         * tests/controlflow-recursive.js: ditto
 318         * tests/crypto-aes.js: ditto
 319         * tests/crypto-md5.js: ditto
 320         * tests/crypto-sha1.js: ditto
 321         * tests/date-format-tofte.js: ditto
 322         * tests/date-format-xparb.js: ditto
 323         * tests/math-cordic.js: ditto
 324         * tests/math-partial-sums.js: ditto
 325         * tests/math-spectral-norm.js: ditto
 326         * tests/regexp-dna.js: ditto
 327         * tests/string-base64.js: ditto
 328         * tests/string-fasta.js: ditto
 329         * tests/string-tagcloud.js: ditto
 330         * tests/string-unpack-code.js: ditto
 331         * tests/string-validate-input.js: ditto
 332         * tests/sunspider-0.9/: Added
 333         * tests/sunspider-0.9/*.js: Moved from one level up
 334         * tests/sunspider-0.9.1: Added.
 335         * tests/sunspider-0.9.1/*.js: Moved from one level up.
 336         * tests/v8-crypto.js: Moved to v8-v4/ directory.
 337         * tests/v8-deltablue.js: ditto
 338         * tests/v8-earley-boyer.js: ditto
 339         * tests/v8-raytrace.js: ditto
 340         * tests/v8-regexp.js: ditto
 341         * tests/v8-richards.js: ditto
 342         * tests/v8-splay.js: ditto
 343         * tests/v8-v4: Added.
 344
 345 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
 346
 347         Reviewed by Darin Adler.
 348
 349         Platform specific null device replaced with a platform independent.
 350         https://bugs.webkit.org/show_bug.cgi?id=29544
 351
 352         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
 353
 354 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
 355
 356         Reviewed by Maciej Stachowiak.
 357
 358         [Qt] Make sunspider script work on Windows platform.
 359         https://bugs.webkit.org/show_bug.cgi?id=29544
 360
 361         * sunspider:
 362         Using platform specific null device instead of /dev/null.
 363
 364 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
 365
 366         Reviewed by Oliver Hunt.
 367
 368         https://bugs.webkit.org/show_bug.cgi?id=26540
 369         Currently the SunSpider test driver lacks an option to run a test suite that
 370         will test JavaScriptCore parsing performance only.  This patch adds just such
 371         a test suite and option to SunSpider as well as the jsc test shell.  I've included
 372         three large javascript source files found in the wild: jquery, mootools and prototype.
 373         Combined with the concatenation of all three, these form a new testsuite to measure
 374         and test pure JavaScriptCore parsing performance.
 375
 376         * resources/sunspider-standalone-driver.js:
 377           Load the js with parse-only flag if it is part of the parse-only suite.
 378         * sunspider:
 379           Add the parse-only suite to the list of options.
 380         * sunspider-compare-results:
 381           Add the parse-only suite to the list of options.
 382         * tests/LIST-PARSE-ONLY: Added.
 383         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
 384           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
 385         * tests/parse-only/jquery-1.3.2.js: Added.
 386           Latest version from jquery.com.
 387         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
 388           Latest version from mootools.net.
 389         * tests/parse-only/prototype-1.6.0.3.js: Added.
 390           Latest version from prototypejs.org.
 391
 392 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
 393
 394         Update V8 benchmark to version 4.
 395
 396         Reviewed by Cameron Zwarich.
 397
 398         * tests/LIST-V8:
 399         * tests/v8-crypto.js:
 400         * tests/v8-deltablue.js:
 401         * tests/v8-earley-boyer.js:
 402         * tests/v8-raytrace.js:
 403         * tests/v8-regexp.js: Added.
 404         * tests/v8-richards.js:
 405         * tests/v8-splay.js: Added.
 406
 407 2009-05-26  Olivier DOLE  <odole@pleyo.com>
 408
 409         Reviewed by Maciej.
 410
 411         Store sunspider test suite results in a specific directory for each
 412         different type of benchmark (sunspider, V8 or ubench).
 413
 414         * sunspider:
 415         * sunspider-compare-results:
 416
 417 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
 418
 419         Build fix.
 420
 421         * SunSpider.make:
 422
 423 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
 424
 425         Include SunSpider tests in production builds.
 426
 427         Reviewed by Adam Roben.
 428
 429         * SunSpider.make: Added.
 430         * xcopy.excludes: Added.
 431
 432 2009-03-04  Oliver Hunt  <oliver@apple.com>
 433
 434         Added a few more tests to ubench
 435
 436         RS=Gavin Barraclough
 437
 438         Adding a couple of test to cover too few and too many arguments.  This
 439         is needed to prevent regressions in all call paths from my current refactoring.
 440
 441         * tests/LIST-UBENCH:
 442         * tests/ubench/function-correct-args.js: Added.
 443         (f):
 444         * tests/ubench/function-excess-args.js: Added.
 445         (f):
 446
 447 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
 448
 449         Fix a bug where when V8's or UBENCH's test suite list file is missing
 450         the error message is suggesting that SunSpider's list file is missing.
 451
 452         Reviewed by Sam Weinig.
 453
 454         * sunspider:
 455         (loadTestsList): Fix the error message to use the right list name.
 456
 457 2008-10-07  David Hyatt  <hyatt@apple.com>
 458
 459         https://bugs.webkit.org/show_bug.cgi?id=18989
 460
 461         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
 462         we want all browsers to be doing the same thing on the test.
 463
 464         Reviewed by Maciej
 465
 466         * tests/regexp-dna.js:
 467
 468 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 469
 470         Rubber-stamped by Maciej Stachowiak.
 471
 472         The bleeding edge branch of V8 now supports passing fileanmes after
 473         '-f', so the SunSpider script should be changed to do this again.
 474         This allows all 3 major JavaScript engines to be used with the
 475         unmodified SunSpider script.
 476
 477         * sunspider:
 478
 479 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 480
 481         Reviewed by Maciej Stachowiak.
 482
 483         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
 484         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
 485         correct the punctuation in its documentation.
 486
 487         * sunspider:
 488
 489 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 490
 491         Reviewed by Oliver Hunt.
 492
 493         Bug 20616: Incorporate V8 benchmarks in testing
 494         <https://bugs.webkit.org/show_bug.cgi?id=20616>
 495
 496         This is a first attempt at running the V8 benchmarks within SunSpider,
 497         with the --v8 option. The number of iterations is fixed for each
 498         benchmark, and was chosen to make each of the benchmarks run between
 499         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
 500
 501         * sunspider:
 502         * tests/LIST-V8: Added.
 503         * tests/v8-crypto.js: Added.
 504         * tests/v8-deltablue.js: Added.
 505         * tests/v8-earley-boyer.js: Added.
 506         * tests/v8-raytrace.js: Added.
 507         * tests/v8-richards.js: Added.
 508
 509 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 510
 511         Not reviewed.
 512
 513         Remove a local change that was accidentally committed in r36034.
 514
 515         * resources/sunspider-standalone-driver.js:
 516
 517 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 518
 519         Rubber-stamped by Oliver Hunt.
 520
 521         Make SunSpider work with JavaScript shells that don't take filenames
 522         after a '-f' argument.
 523
 524         * sunspider:
 525
 526 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 527
 528         Reviewed by Timothy.
 529
 530         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
 531
 532         * sunspider:
 533         * tests/LIST-SQUIRRELFISH: Removed.
 534
 535 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
 536
 537         Rubber-stamped by Mark Rowe.
 538
 539         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
 540
 541         * sunspider:
 542
 543 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
 544
 545         Rubber stamped by Oliver.
 546
 547         - add newly working tests to squirrelfish list
 548
 549         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
 550
 551 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
 552
 553         Rubber stamped by Oliver.
 554
 555         - move ubench tests properly.
 556
 557         * tests/ubench: Added.
 558         * tests/ubench/function-closure.js: Added.
 559         * tests/ubench/function-empty.js: Added.
 560         * tests/ubench/function-missing-args.js: Added.
 561         * tests/ubench/function-sum.js: Added.
 562         * tests/ubench/loop-empty-resolve.js: Added.
 563         * tests/ubench/loop-empty.js: Added.
 564         * tests/ubench/loop-sum.js: Added.
 565
 566 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
 567
 568         Rubber stamped by Oliver.
 569
 570         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
 571
 572 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
 573
 574         Reviewed by Oliver.
 575
 576         - make --squirrelfish mode use the real harness, but a reduced set of tests
 577
 578         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
 579
 580         * sunspider:
 581         * tests/LIST-SQUIRRELFISH:
 582         * tests/LIST-UBENCH: Added.
 583         * tests/squirrelfish: Moved to tests/ubench
 584
 585 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
 586
 587         Reviewed by Maciej Stachowiak.
 588
 589         Removed an unused test.
 590
 591         * tests/squirrelfish/loop-resolve.js: Removed.
 592
 593 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
 594
 595         Reviewed by Oliver.
 596
 597         Add newly runnable tests to --squirrelfish mode.
 598
 599         * tests/LIST-SQUIRRELFISH:
 600
 601 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
 602
 603         Reviewed by Maciej Stachowiak.
 604
 605         A little more love for --squirrelfish mode:
 606
 607         Fixed a misplaced sort. Results properly sort now.
 608
 609         Added a test list just for squirrelfish, and updated it to include
 610         all currently passing SunSpider tests.
 611
 612         Fixed the pruning regexp to match 3d-morph.js.
 613
 614         * sunspider:
 615
 616 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
 617
 618         Reviewed by Oliver Hunt.
 619
 620         --squirrelfish mode: pared down tests for squirrelfish to chew on.
 621
 622         * sunspider:
 623         * tests/LIST:
 624         * tests/squirrelfish: Added.
 625         * tests/squirrelfish/function-closure.js: Added.
 626         * tests/squirrelfish/function-empty.js: Added.
 627         * tests/squirrelfish/function-missing-args.js: Added.
 628         * tests/squirrelfish/function-sum.js: Added.
 629         * tests/squirrelfish/loop-empty-resolve.js: Added.
 630         * tests/squirrelfish/loop-empty.js: Added.
 631         * tests/squirrelfish/loop-resolve.js: Added.
 632         * tests/squirrelfish/loop-sum.js: Added.
 633
 634 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
 635
 636         Reviewed by Geoff.
 637
 638         * sunspider: Pause update daemon on Darwin to make results reliable
 639         again (workaround for <rdar://problem/5811127>).
 640
 641 2008-02-07  Eric Seidel  <eric@webkit.org>
 642
 643         Reviewed by Sam.
 644
 645         * sunspider: add an error message when ./tests/LIST is not found
 646
 647 2007-12-29  Alp Toker  <alp@atoker.com>
 648
 649         Fix typo.
 650
 651         * hosted/sunspider.html:
 652
 653 2007-12-29  Alp Toker  <alp@atoker.com>
 654
 655         Reviewed by Maciej.
 656
 657         Provide the current URL in the SunSpider results page. Useful for
 658         copying benchmark results out of browsers without a location bar or
 659         where the location bar can't handle long URLs.
 660
 661         * hosted/sunspider-results.html:
 662
 663 2007-12-20  Eric Seidel  <eric@webkit.org>
 664
 665         Reviewed by mjs.
 666
 667         * sunspider: increase default number of runs for more accurate numbers
 668
 669 2007-12-18  Mark Rowe  <mrowe@apple.com>
 670
 671         Fix typo.
 672
 673         * hosted/sunspider.html:
 674
 675 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
 676
 677         Reviewed by Eric.
 678
 679         - give an extra digit of precision for small differences.
 680
 681         * resources/sunspider-compare-results.js:
 682
 683 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
 684
 685         Reviewed by Eric.
 686
 687         - work around IE layout bugs
 688
 689         * hosted/sunspider.css:
 690
 691 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
 692
 693         Reviewed by Mark.
 694
 695         - fix compare feature
 696
 697         * hosted/sunspider-results.html:
 698
 699 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
 700
 701         Reviewed by Mark.
 702
 703         - Change the format of the output URL to be shorter, to meet IE constraints
 704
 705         * hosted/sunspider-driver.html:
 706         * hosted/sunspider-results.html:
 707
 708 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
 709
 710         Reviewed by Oliver.
 711
 712         - Add some explanatory test and a dumb logo to SunSpider web hosted version
 713
 714         * hosted/sunspider-driver.html:
 715         * hosted/sunspider-results.html:
 716         * hosted/sunspider.css:
 717         * hosted/sunspider.html:
 718
 719 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
 720
 721         Reviewed by Oliver.
 722
 723         - Make the web-hosted version of SunSpider look a little bit less drab.
 724
 725         * hosted/sunspider-driver.html:
 726         * hosted/sunspider-results.html:
 727         * hosted/sunspider.css: Added.
 728         * hosted/sunspider.html:
 729         * resources/TEMPLATE.html:
 730
 731 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
 732
 733         Reviewed by Sam.
 734
 735         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
 736
 737         A sample of what new-style results would look like:
 738
 739         TEST                   COMPARISON            FROM                 TO             DETAILS
 740         ...
 741         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
 742         ...
 743             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
 744
 745         * resources/sunspider-compare-results.js:
 746
 747 2007-11-14  Eric Seidel  <eric@webkit.org>
 748
 749         Reviewed by Sam.
 750
 751         * sunspider: add --shark-cache for L2 Cache Miss profiling
 752
 753 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
 754
 755         Reviewed by Darin Adler.
 756
 757         http://bugs.webkit.org/show_bug.cgi?id=15925
 758         SunSpider should check for Shark being installed
 759
 760         * sunspider: Explicitly test that shark command line tool is installed.
 761
 762 2007-11-04  Darin Adler  <darin@apple.com>
 763
 764         * hosted/sunspider.html: Tweaked formatting a bit.
 765
 766 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
 767
 768         Reviewed by Darin Adler.
 769
 770         - Add compare mode and a description of the benchmark to the hosted version.
 771
 772         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
 773         * hosted/sunspider.html: Added intro text.
 774         * make-hosted: Copy new file.
 775         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
 776         a function.
 777         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
 778         expects.
 779         * sunspider-compare-results: Use sunspider-standalone-compare.js.
 780
 781 2007-11-02  Darin Adler  <darin@apple.com>
 782
 783         * sunspider: Changed "--base" to "--set-baseline".
 784
 785 2007-11-02  Darin Adler  <darin@apple.com>
 786
 787         Reviewed by Maciej.
 788
 789         * sunspider: Added a "--base" option that records the current test as a baseline.
 790         Moved the code to find the newest .mshark file into a function (since I needed to use
 791         it in the other script and I prefer to copy/paste a function).
 792
 793         * sunspider-compare-results: Added logic to compare the baseline with the last
 794         run when you don't pass any parameters.
 795
 796 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 797
 798         Reviewed by Eric.
 799
 800         - Added list of ECMAScript3 features that SunSpider does not cover currently.
 801
 802         * UNCOVERED: Added.
 803
 804 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 805
 806         Reviewed by Oliver.
 807
 808         - Give better error messages when sunspider-compare-results fails
 809
 810         * sunspider-compare-results:
 811
 812 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 813
 814         Reviewed by Darin Adler.
 815
 816         - fix Date tests to work in command-line SpiderMonkey
 817
 818         * tests/date-format-tofte.js:
 819         * tests/date-format-xparb.js:
 820
 821 2007-10-29  Darin Adler  <darin@apple.com>
 822
 823         Reviewed by Maciej.
 824
 825         * sunspider: Added --shark20 option, to run Shark at its highest sample
 826         resolution instead of its default.
 827
 828 2007-10-28  Adam Roben  <aroben@apple.com>
 829
 830         Fix SunSpider on Windows
 831
 832         Reviewed by Eric.
 833
 834         * sunspider: Don't use colons in filenames.
 835
 836 2007-10-28  Darin Adler  <darin@apple.com>
 837
 838         Reviewed by Adam.
 839
 840         * resources/sunspider-analyze-results.js: Tweak the output format a little.
 841         Change so that when there's only one run we don't write out confidence
 842         intervals at all rather than writing "NaN" over and over again.
 843
 844         * sunspider: Use the time and date as a suffix on the results file, that way
 845         all the results are left behind in the tmp directory. This will make it easy
 846         for us to add features that compare past results in the future. Also rename
 847         the .mshark files using the same scheme.
 848
 849         * sunspider-compare-results: Relax the parsing rules so we can parse the
 850         results file exactly as it's emitted from sunspider.
 851
 852 2007-10-25  Adam Roben  <aroben@apple.com>
 853
 854         Strip carriage returns from results
 855
 856         Carriage returns were screwing up sunspider-compare-results on
 857         Windows.
 858
 859         Rubberstamped by Oliver.
 860
 861         * sunspider:
 862
 863 2007-10-24  Darin Adler  <darin@apple.com>
 864
 865         Reviewed by Eric.
 866
 867         * sunspider: Open the profile in shark after completing a run.
 868         * sunspider-compare-results: Tiny perl tweaks.
 869
 870 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 871
 872         Reviewed by Darin Adler.
 873
 874         - Don't hardcode my path to testkjs
 875
 876         * sunspider-compare-results:
 877
 878 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 879
 880         Reviewed by Darin Adler.
 881
 882         - Add a compare script to compare results, and improve formatting of standard results a bit.
 883
 884         * resources/sunspider-analyze-results.js:
 885         * resources/sunspider-compare-results.js: Added.
 886         * sunspider-compare-results: Added.
 887
 888 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 889
 890         Reviewed by Darin Adler.
 891
 892         * resources/sunspider-analyze-results.js: Do the error range
 893         properly, using the t-distribution instead of 1.96 (which was
 894         based on the normal distribution).
 895
 896         * sunspider: Print results in a way that will be friendlier to the
 897         soon-to-come compare mode.
 898
 899 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
 900
 901         Reviewed by Review.
 902
 903         - add more tests (probably the final set for now)
 904
 905         Already balanced. Added date, regexp, control flow, and a few
 906         more string and object/array access tests.
 907
 908         * tests/LIST:
 909         * tests/access-fannkuch.js: Added.
 910         * tests/access-nbody.js: Added.
 911         * tests/controlflow-recursive.js: Added.
 912         * tests/date-format-tofte.js: Added.
 913         * tests/date-format-xparb.js: Added.
 914         * tests/regexp-dna.js: Added.
 915         * tests/string-validate-input.js: Added.
 916
 917 2007-10-21  Eric Seidel  <eric@webkit.org>
 918
 919         Reviewed by mjs.
 920
 921         * sunspider: make --shark only sample from testkjs
 922
 923 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 924
 925         Rubber stamped by Eric.
 926
 927         - added crypto tests (already balanced)
 928
 929         * tests/LIST:
 930         * tests/crypto-aes.js: Added.
 931         * tests/crypto-md5.js: Added.
 932         * tests/crypto-sha1.js: Added.
 933
 934 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 935
 936         Reviewed by Adam.
 937
 938         - rebalance new tests
 939
 940         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
 941
 942         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 943
 944         This balances them with the existing tests.
 945
 946         * tests/access-binary-trees.js:
 947         * tests/access-nsieve.js:
 948         * tests/bitops-nsieve-bits.js:
 949         * tests/math-partial-sums.js:
 950         * tests/math-spectral-norm.js:
 951         * tests/string-fasta.js:
 952
 953 2007-10-20  Eric Seidel  <eric@webkit.org>
 954
 955         Reviewed by darin.
 956
 957         Add improved argument handling to sunspider, including
 958         --runs=<number>, --tests=<pattern, --shark, and --help
 959         Also re-factor code into subroutines
 960
 961         * sunspider: improved argument handling
 962
 963 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 964
 965         Reviewed by Mark.
 966
 967         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
 968
 969         * TODO:
 970         * tests/LIST:
 971         * tests/access-binary-trees.js: Added.
 972         * tests/access-nsieve.js: Added.
 973         * tests/bitops-nsieve-bits.js: Added.
 974         * tests/math-partial-sums.js: Added.
 975         * tests/math-spectral-norm.js: Added.
 976         * tests/string-fasta.js: Added.
 977
 978 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 979
 980         Reviewed by Darin Adler.
 981
 982         * TODO: Updated to note areas that are now well-covered.
 983
 984         - Change the tests and the driver to avoid leaving large object
 985         graphs hanging around, since that throws off the subsequent tests.
 986
 987         * resources/sunspider-standalone-driver.js:
 988         * tests/3d-cube.js:
 989         * tests/3d-morph.js:
 990         * tests/3d-raytrace.js:
 991         * tests/bitops-bitwise-and.js:
 992         * tests/string-base64.js:
 993         * tests/string-tagcloud.js:
 994
 995 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 996
 997         Reviewed by Nikolas.
 998
 999         - Rebalance test complexity.
1000
1001         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
1002
1003         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1004
1005         However, for some tests, IE7 was a huge outlier compared to other
1006         major browsers. For those tests, I used Firefox 2.0.0.8 on the
1007         same OS and hardware as a normalization basis.
1008
1009         * tests/3d-morph.js:
1010         * tests/3d-raytrace.js:
1011         * tests/bitops-3bit-bits-in-byte.js:
1012         * tests/bitops-bits-in-byte.js:
1013         * tests/bitops-bitwise-and.js:
1014         * tests/math-cordic.js:
1015         * tests/string-base64.js:
1016         * tests/string-tagcloud.js:
1017         * tests/string-unpack-code.js:
1018
1019 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1020
1021         Reviewed by Oliver.
1022
1023         - Fix the browser-hosted driver to be compatible with Safari 2
1024
1025         * hosted/sunspider-driver.html:
1026
1027 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1028
1029         Rubber stamped by Hyatt.
1030
1031         - Add BSD license notice to cordic test, the original author said to do so.
1032
1033         * tests/math-cordic.js:
1034
1035 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1036
1037         Reviewed by Eric.
1038
1039         - Support Windows and Windows browsers.
1040
1041         * hosted/sunspider-results.html: Display results correctly in IE.
1042         * sunspider: tweaks to work on Windows, and with spaces in the path.
1043
1044 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1045
1046         Rubber stamped by Oliver.
1047
1048         * tests/string-tagcloud.js: Remove stray print()
1049
1050 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
1051
1052         Reviewed by Oliver.
1053
1054         Add some string test cases.
1055
1056         * tests/string-base64.js: Added. Does base64 encoding and decoding.
1057         * tests/string-tagcloud.js: Added. Parses a JSON string and
1058         generates tagcloud HTML markup based on the data.
1059         * tests/string-unpack-code.js: Added. Unpack compressed versions of
1060         four of the most popular JavaScript libraries.
1061         * tests/LIST: Added the new tests.
1062
1063 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
1064
1065         Reviewed by Anders.
1066
1067         * TODO: Updated with info on how to do the stats properly.
1068
1069         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
1070         avoid overly slowing down the benchmark.
1071
1072         Added some 3D-related test cases.
1073
1074         * tests/3d-cube.js: Added.
1075         * tests/3d-morph.js: Added.
1076         * tests/3d-raytrace.js: Added.
1077         * tests/LIST:
1078
1079 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
1080
1081         Reviewed by Darin Adler.
1082
1083         - New JavaScript benchmark
1084         http://bugs.webkit.org/show_bug.cgi?id=15515
1085
1086         A start on a new JavaScript benchmark, with standalone and
1087         browser-hosted drivers.
1088
1089         The test content is still incomplete and the drivers could use
1090         some more features but there is enough here to make a good start.
1091
1092         The drivers compute a 95% confidence interval on the mean for the
1093         whole test, each category, and each individual test to make it
1094         easier to tell whether differences are statistically
1095         significant. The confidence interval can be narrowed by running
1096         the test more times.
1097
1098         * TODO: Added. Remaining things that need to be done.
1099         * sunspider: Added. Perl script that acts as the standalone test driver.
1100         * resources/sunspider-standalone-driver.js: Added. JavaScript part
1101         of standalone test driver.
1102         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
1103         analysis code.
1104         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
1105         * make-hosted: Added. Script to generate browser-hosted tests.
1106         * hosted/sunspider.html: Added. Start page for browser-hosted test.
1107         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
1108         * hosted/sunspider-results.html: Added. Results page for browser-hosted
1109         tests.
1110         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
1111         tests.
1112         * tests/LIST: Added. List of tests to use.
1113         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
1114         * tests/bitops-bits-in-byte.js: Added. Ditto.
1115         * tests/bitops-bitwise-and.js: Added. Ditto.
1116         * tests/math-cordic.js: Added. Ditto.
1117