PerformanceTests/SunSpider/ChangeLog

   1 2011-11-15  Andy Wingo  <wingo@igalia.com>
   2
   3         Extend sunspider driver to be able to run kraken
   4         https://bugs.webkit.org/show_bug.cgi?id=71799
   5
   6         Reviewed by Filip Pizlo.
   7
   8         * resources/sunspider-standalone-driver.js: Try to load a -data
   9         file.  If that succeeds, we have a kraken-like test, so we time
  10         the test using `load'.  Otherwise fall back to using `run'.
  11
  12 2011-08-11  Mark Rowe  <mrowe@apple.com>
  13
  14         Fix sunspider-compare-results to default to the same suite version as the sunspider script.
  15
  16         Rubber-stamped by Maciej Stachowiak.
  17
  18         * sunspider-compare-results:
  19
  20 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
  21
  22         Update SunSpider versioning to 1.0
  23         https://bugs.webkit.org/show_bug.cgi?id=63870
  24
  25         Reviewed by Dan Bernstein.
  26
  27         * hosted/sunspider.html:
  28         * hosted/versions.html:
  29         * resources/driver-TEMPLATE.html:
  30         * resources/results-TEMPLATE.html:
  31         * sunspider:
  32
  33 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
  34
  35         Sunspider 0.9.1 harness doesn't actually close() its test documents
  36         https://bugs.webkit.org/show_bug.cgi?id=47045
  37
  38         Reviewed by Daniel Bates.
  39
  40         * hosted/versions.html:
  41         * resources/driver-TEMPLATE.html:
  42
  43 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
  44
  45         Not Reviewed.
  46
  47         Fix obviou typo in previous commit.
  48
  49         * tests/sunspider-1.0/bitops-bitwise-and.js:
  50
  51 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
  52
  53         Reviewed by Anders Carlsson.
  54
  55         SunSpider: all four bitops benchmarks can be replaced with NOP
  56         https://bugs.webkit.org/show_bug.cgi?id=38446
  57
  58         * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
  59         (TimeFunc): Save the result.
  60         * tests/sunspider-1.0/bitops-bits-in-byte.js:
  61         (TimeFunc): Save the result.
  62         * tests/sunspider-1.0/bitops-bitwise-and.js: More explicitly
  63         save the results.
  64         * tests/sunspider-1.0/bitops-nsieve-bits.js:
  65         (sieve): Save the result.
  66
  67 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
  68
  69         SunSpider: string-validate-input.js uses an undeclared variable named 'name', which is a DOM API
  70         https://bugs.webkit.org/show_bug.cgi?id=60937
  71
  72         Reviewed by Dan Bernstein.
  73
  74         * tests/sunspider-1.0/string-validate-input.js:
  75         (doTest): Rename the variable to username
  76
  77 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
  78
  79         Reviewed by Darin Adler.
  80
  81         SunSpider: The main function of math-cordic is dead code
  82         https://bugs.webkit.org/show_bug.cgi?id=63863
  83
  84         Try to prevent math-cordic from being eliminated as dead code or
  85         compiled down to a constant.
  86
  87         * tests/sunspider-1.0/math-cordic.js:
  88         (cordicsincos):
  89         (cordic):
  90
  91 2011-06-30  Maciej Stachowiak  <mjs@apple.com>
  92
  93         Reviewed by Adam Barth.
  94
  95         Create sunspider-1.0 directory in preparation for fixing a bunch of bugs
  96         https://bugs.webkit.org/show_bug.cgi?id=63782
  97
  98         * make-hosted: Learn about the new directory.
  99         * tests/sunspider-1.0: Copied from PerformanceTests/SunSpider/tests/sunspider-0.9.1.
 100
 101 2010-11-29  Geoffrey Garen  <ggaren@apple.com>
 102
 103         Reviewed by Gavin Barraclough.
 104
 105         Improved accuracy of command-line SunSpider.
 106
 107         * resources/sunspider-standalone-driver.js: Use the returned number of
 108         elapsed milliseconds from "run" and "checkSyntax" instead of measuring
 109         ourselves, for slightly more accurate numbers.
 110
 111 2010-09-16  Gavin Barraclough  <barraclough@apple.com>
 112
 113         Reviewed by Mark Rowe.
 114         https://bugs.webkit.org/show_bug.cgi?id=45924
 115
 116         Modify the SunSpider harness to allow a path to be passed to the --suite
 117         argument, supress the normal prepend-"tests/" behaviour for suite names
 118         containing a '/'.
 119
 120         Also wrap the code in an anonymous function to move variables out of global
 121         scope (currently the harness cannot handle running tests the overwrite the
 122         variable 'j').
 123
 124         * resources/sunspider-standalone-driver.js:
 125         * sunspider:
 126
 127 2010-08-27  Michael Saboff  <msaboff@apple.com>
 128
 129         Reviewed by Stephanie Lewis.
 130
 131         Added version 5 and version 6 of V8 tests with modifications similar
 132         to the version 4 tests so they'll work with the SunSpider test harness.
 133         Note that these files are external source and therefore don't conform
 134         to WebKit coding standards.
 135         https://bugs.webkit.org/show_bug.cgi?id=44638
 136
 137         * tests/v8-v5: Added.
 138         * tests/v8-v5/LIST: Added.
 139         * tests/v8-v5/v8-crypto.js: Added.
 140         * tests/v8-v5/v8-deltablue.js: Added.
 141         * tests/v8-v5/v8-earley-boyer.js: Added.
 142         * tests/v8-v5/v8-raytrace.js: Added.
 143         * tests/v8-v5/v8-regexp.js: Added.
 144         * tests/v8-v5/v8-richards.js: Added.
 145         * tests/v8-v5/v8-splay.js: Added.
 146         * tests/v8-v6: Added.
 147         * tests/v8-v6/LIST: Added.
 148         * tests/v8-v6/v8-crypto.js: Added.
 149         * tests/v8-v6/v8-deltablue.js: Added.
 150         * tests/v8-v6/v8-earley-boyer.js: Added.
 151         * tests/v8-v6/v8-raytrace.js: Added.
 152         * tests/v8-v6/v8-regexp.js: Added.
 153         * tests/v8-v6/v8-richards.js: Added.
 154         * tests/v8-v6/v8-splay.js: Added.
 155
 156 2010-07-15  Stephanie Lewis  <slewis@apple.com>
 157
 158         Reviewed by Geoff Garen.
 159
 160         http://bugs.webkit.org/show_bug.cgi?id=42406
 161         Output location of results file.
 162
 163         * sunspider:
 164
 165 2010-05-17  Maciej Stachowiak  <mjs@apple.com>
 166
 167         Reviewed by Mark Rowe.
 168
 169         Browser-hosted version of SunSpider runs tests twice as many times as advertised
 170         https://bugs.webkit.org/show_bug.cgi?id=39201
 171
 172         * make-hosted: Make sure to reset test list after every test suite.
 173
 174 2010-05-07  Adam Roben  <aroben@apple.com>
 175
 176         Fix typo in sunspider-compare-results.js that caused incorrect
 177         two-sample t statistics to be computed
 178
 179         Fixes <http://webkit.org/b/38768> SunSpider computes incorrect
 180         two-sample t statistics
 181
 182         Reviewed by Darin Adler.
 183
 184         * resources/sunspider-compare-results.js:
 185         (sunspiderCompareResults.resultLine): Fixed the calculation of the t
 186         variable to match the definition of a two-sample t statistic. Looks
 187         like this was just a simple typo.
 188
 189 2010-04-05  Darin Adler  <darin@apple.com>
 190
 191         Updated the ignore property for changes to the contents here.
 192
 193         * .: Modified property svn:ignore.
 194
 195 2010-02-20  Maciej Stachowiak  <mjs@apple.com>
 196
 197         Reviewed by Adam Barth.
 198
 199         Add sunspider-0.9.1 to the site in preparation for announcing it.
 200         https://bugs.webkit.org/show_bug.cgi?id=35206
 201
 202         * hosted/sunspider.css: Remove tabs, which I noticed where present while
 203         copying to the WebKitSite directory.
 204
 205 2010-01-11  Oliver Hunt  <oliver@apple.com>
 206
 207         Reviewed by Darin Adler.
 208
 209         Sunspider parse-only tests don't run
 210         https://bugs.webkit.org/show_bug.cgi?id=33489
 211
 212         Fix parse-only test list
 213
 214         * tests/parse-only/LIST:
 215
 216 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
 217
 218         Reviewed by Adele Peterson.
 219
 220         Make SunSpider version more prominent in the title
 221         https://bugs.webkit.org/show_bug.cgi?id=32574
 222
 223         * hosted/sunspider.html:
 224         * hosted/versions.html:
 225         * resources/driver-TEMPLATE.html:
 226         * resources/results-TEMPLATE.html:
 227
 228 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
 229
 230         Reviewed by Darin Adler.
 231
 232         Make SunSpider refuse to compare results across content versions
 233         https://bugs.webkit.org/show_bug.cgi?id=32573
 234
 235         The results URL now includes a version indicator, and compare mode
 236         will refuse to compare between versions.
 237
 238         * resources/driver-TEMPLATE.html:
 239         * resources/results-TEMPLATE.html:
 240
 241 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
 242
 243         Reviewed by Darin Adler.
 244
 245         Some Browser-hosted SunSpider files are not valid HTML5
 246         https://bugs.webkit.org/show_bug.cgi?id=32536
 247
 248         Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
 249         close tags and adding a meta charset declaration).
 250
 251         * hosted/sunspider.html:
 252         * hosted/versions.html:
 253         * resources/TEMPLATE.html:
 254         * resources/driver-TEMPLATE.html:
 255         * resources/results-TEMPLATE.html:
 256
 257 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
 258
 259         Reviewed by Eric Seidel.
 260
 261         Make sunspider-0.9.1 the default content set (both command-line and hosted)
 262         https://bugs.webkit.org/show_bug.cgi?id=32537
 263
 264         * hosted/sunspider.html: Change default suite link.
 265         * sunspider: Change default suite.
 266         * sunspider-compare-results: Likewise.
 267
 268 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
 269
 270         Reviewed by Sam Weinig.
 271
 272         in-browser SunSpider suffers excessive penalty under power management
 273         https://bugs.webkit.org/show_bug.cgi?id=32505
 274
 275         I have made a few changes to address this:
 276
 277         1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
 278         document.write to reduce triggering of progress UI.
 279         2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
 280         stepped all the way down the whole time.
 281         3) Run the test cycle an extra time for warmup before the runs that count.
 282
 283         * hosted/sunspider-record-result.js: Removed.
 284         * make-hosted:
 285         * resources/TEMPLATE.html:
 286         * resources/driver-TEMPLATE.html:
 287
 288 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
 289
 290         Reviewed by Gavin Barraclough.
 291
 292         SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
 293         https://bugs.webkit.org/show_bug.cgi?id=16806
 294
 295         Based on a patch by Eric Seidel.
 296
 297         Fix the base64 computation to actually compute correct results. The impact on runtime of
 298         the test is pretty small, but noticeable for some browsers. But at least it's not
 299         doing a wrong and meaningless computation any more.
 300
 301         * tests/sunspider-0.9.1/string-base64.js:
 302         ():
 303         (base64ToString):
 304
 305 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
 306
 307         Fixing commit error...
 308
 309         I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
 310
 311         * resources/driver-TEMPLATE.html:
 312
 313 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
 314
 315         Reviewed by Sam Weinig.
 316
 317         Use JSON.parse instead of eval for Web-hosted SunSpider results processing
 318         https://bugs.webkit.org/show_bug.cgi?id=32490
 319
 320         * hosted/json2.js: Added. Incorporated from json.org
 321         * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
 322         * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
 323         results.
 324
 325 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
 326
 327         Reviewed by Oliver Hunt.
 328
 329         Enable Web-hosted version of SunSpider to handle multiple versions
 330         https://bugs.webkit.org/show_bug.cgi?id=32478
 331
 332         * make-hosted: Now generate the hosted version to be able to run both the
 333         0.9 and the 0.9.1 test suites.
 334         * hosted: Modified property svn:ignore.
 335         * hosted/sunspider-driver.html: Removed.
 336         * hosted/sunspider-results.html: Removed.
 337         * hosted/sunspider.html:
 338         * hosted/versions.html: Added.
 339         * resources/TEMPLATE.html:
 340         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
 341         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
 342
 343 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
 344
 345         Reviewed by Oliver Hunt.
 346
 347         Give command-line sunspider the ability to handle multiple suites and versions
 348         https://bugs.webkit.org/show_bug.cgi?id=32477
 349
 350         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
 351         enumerating the tests. Current suites are:
 352
 353         sunspider-0.9 - riginal SunSpider content
 354         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
 355         ubench - "ubench" microbenchmark suite
 356         v8-v4 - version 4 of the v8 benchmark
 357         parse-only - some parser tests
 358
 359         Sunspider can now be run with the --suite= parameter to select a
 360         suite. The default is --suite=sunspider-0.9.
 361
 362         * .: Modified property svn:ignore to ignore new results directories.
 363         * make-hosted: Generate based on sunspider-0.9 suite.
 364         * resources/TEMPLATE.html: Fix an HTML compliance problem.
 365         * resources/sunspider-standalone-driver.js: Become aware of suite names.
 366         * sunspider: Modified to handle multiple suites as described above.
 367         * sunspider-compare-results: ditto
 368         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
 369         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
 370         * tests/LIST-UBENCH: Removed.
 371         * tests/LIST-V8: Removed.
 372         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
 373         * tests/3d-morph.js: ditto
 374         * tests/3d-raytrace.js: ditto
 375         * tests/access-binary-trees.js: ditto
 376         * tests/access-fannkuch.js: ditto
 377         * tests/access-nbody.js: ditto
 378         * tests/access-nsieve.js: ditto
 379         * tests/bitops-3bit-bits-in-byte.js: ditto
 380         * tests/bitops-bits-in-byte.js: ditto
 381         * tests/bitops-bitwise-and.js: ditto
 382         * tests/bitops-nsieve-bits.js: ditto
 383         * tests/controlflow-recursive.js: ditto
 384         * tests/crypto-aes.js: ditto
 385         * tests/crypto-md5.js: ditto
 386         * tests/crypto-sha1.js: ditto
 387         * tests/date-format-tofte.js: ditto
 388         * tests/date-format-xparb.js: ditto
 389         * tests/math-cordic.js: ditto
 390         * tests/math-partial-sums.js: ditto
 391         * tests/math-spectral-norm.js: ditto
 392         * tests/regexp-dna.js: ditto
 393         * tests/string-base64.js: ditto
 394         * tests/string-fasta.js: ditto
 395         * tests/string-tagcloud.js: ditto
 396         * tests/string-unpack-code.js: ditto
 397         * tests/string-validate-input.js: ditto
 398         * tests/sunspider-0.9/: Added
 399         * tests/sunspider-0.9/*.js: Moved from one level up
 400         * tests/sunspider-0.9.1: Added.
 401         * tests/sunspider-0.9.1/*.js: Moved from one level up.
 402         * tests/v8-crypto.js: Moved to v8-v4/ directory.
 403         * tests/v8-deltablue.js: ditto
 404         * tests/v8-earley-boyer.js: ditto
 405         * tests/v8-raytrace.js: ditto
 406         * tests/v8-regexp.js: ditto
 407         * tests/v8-richards.js: ditto
 408         * tests/v8-splay.js: ditto
 409         * tests/v8-v4: Added.
 410
 411 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
 412
 413         Reviewed by Darin Adler.
 414
 415         Platform specific null device replaced with a platform independent.
 416         https://bugs.webkit.org/show_bug.cgi?id=29544
 417
 418         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
 419
 420 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
 421
 422         Reviewed by Maciej Stachowiak.
 423
 424         [Qt] Make sunspider script work on Windows platform.
 425         https://bugs.webkit.org/show_bug.cgi?id=29544
 426
 427         * sunspider:
 428         Using platform specific null device instead of /dev/null.
 429
 430 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
 431
 432         Reviewed by Oliver Hunt.
 433
 434         https://bugs.webkit.org/show_bug.cgi?id=26540
 435         Currently the SunSpider test driver lacks an option to run a test suite that
 436         will test JavaScriptCore parsing performance only.  This patch adds just such
 437         a test suite and option to SunSpider as well as the jsc test shell.  I've included
 438         three large javascript source files found in the wild: jquery, mootools and prototype.
 439         Combined with the concatenation of all three, these form a new testsuite to measure
 440         and test pure JavaScriptCore parsing performance.
 441
 442         * resources/sunspider-standalone-driver.js:
 443           Load the js with parse-only flag if it is part of the parse-only suite.
 444         * sunspider:
 445           Add the parse-only suite to the list of options.
 446         * sunspider-compare-results:
 447           Add the parse-only suite to the list of options.
 448         * tests/LIST-PARSE-ONLY: Added.
 449         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
 450           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
 451         * tests/parse-only/jquery-1.3.2.js: Added.
 452           Latest version from jquery.com.
 453         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
 454           Latest version from mootools.net.
 455         * tests/parse-only/prototype-1.6.0.3.js: Added.
 456           Latest version from prototypejs.org.
 457
 458 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
 459
 460         Update V8 benchmark to version 4.
 461
 462         Reviewed by Cameron Zwarich.
 463
 464         * tests/LIST-V8:
 465         * tests/v8-crypto.js:
 466         * tests/v8-deltablue.js:
 467         * tests/v8-earley-boyer.js:
 468         * tests/v8-raytrace.js:
 469         * tests/v8-regexp.js: Added.
 470         * tests/v8-richards.js:
 471         * tests/v8-splay.js: Added.
 472
 473 2009-05-26  Olivier DOLE  <odole@pleyo.com>
 474
 475         Reviewed by Maciej.
 476
 477         Store sunspider test suite results in a specific directory for each
 478         different type of benchmark (sunspider, V8 or ubench).
 479
 480         * sunspider:
 481         * sunspider-compare-results:
 482
 483 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
 484
 485         Build fix.
 486
 487         * SunSpider.make:
 488
 489 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
 490
 491         Include SunSpider tests in production builds.
 492
 493         Reviewed by Adam Roben.
 494
 495         * SunSpider.make: Added.
 496         * xcopy.excludes: Added.
 497
 498 2009-03-04  Oliver Hunt  <oliver@apple.com>
 499
 500         Added a few more tests to ubench
 501
 502         RS=Gavin Barraclough
 503
 504         Adding a couple of test to cover too few and too many arguments.  This
 505         is needed to prevent regressions in all call paths from my current refactoring.
 506
 507         * tests/LIST-UBENCH:
 508         * tests/ubench/function-correct-args.js: Added.
 509         (f):
 510         * tests/ubench/function-excess-args.js: Added.
 511         (f):
 512
 513 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
 514
 515         Fix a bug where when V8's or UBENCH's test suite list file is missing
 516         the error message is suggesting that SunSpider's list file is missing.
 517
 518         Reviewed by Sam Weinig.
 519
 520         * sunspider:
 521         (loadTestsList): Fix the error message to use the right list name.
 522
 523 2008-10-07  David Hyatt  <hyatt@apple.com>
 524
 525         https://bugs.webkit.org/show_bug.cgi?id=18989
 526
 527         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
 528         we want all browsers to be doing the same thing on the test.
 529
 530         Reviewed by Maciej
 531
 532         * tests/regexp-dna.js:
 533
 534 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 535
 536         Rubber-stamped by Maciej Stachowiak.
 537
 538         The bleeding edge branch of V8 now supports passing fileanmes after
 539         '-f', so the SunSpider script should be changed to do this again.
 540         This allows all 3 major JavaScript engines to be used with the
 541         unmodified SunSpider script.
 542
 543         * sunspider:
 544
 545 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 546
 547         Reviewed by Maciej Stachowiak.
 548
 549         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
 550         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
 551         correct the punctuation in its documentation.
 552
 553         * sunspider:
 554
 555 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 556
 557         Reviewed by Oliver Hunt.
 558
 559         Bug 20616: Incorporate V8 benchmarks in testing
 560         <https://bugs.webkit.org/show_bug.cgi?id=20616>
 561
 562         This is a first attempt at running the V8 benchmarks within SunSpider,
 563         with the --v8 option. The number of iterations is fixed for each
 564         benchmark, and was chosen to make each of the benchmarks run between
 565         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
 566
 567         * sunspider:
 568         * tests/LIST-V8: Added.
 569         * tests/v8-crypto.js: Added.
 570         * tests/v8-deltablue.js: Added.
 571         * tests/v8-earley-boyer.js: Added.
 572         * tests/v8-raytrace.js: Added.
 573         * tests/v8-richards.js: Added.
 574
 575 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 576
 577         Not reviewed.
 578
 579         Remove a local change that was accidentally committed in r36034.
 580
 581         * resources/sunspider-standalone-driver.js:
 582
 583 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 584
 585         Rubber-stamped by Oliver Hunt.
 586
 587         Make SunSpider work with JavaScript shells that don't take filenames
 588         after a '-f' argument.
 589
 590         * sunspider:
 591
 592 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 593
 594         Reviewed by Timothy.
 595
 596         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
 597
 598         * sunspider:
 599         * tests/LIST-SQUIRRELFISH: Removed.
 600
 601 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
 602
 603         Rubber-stamped by Mark Rowe.
 604
 605         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
 606
 607         * sunspider:
 608
 609 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
 610
 611         Rubber stamped by Oliver.
 612
 613         - add newly working tests to squirrelfish list
 614
 615         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
 616
 617 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
 618
 619         Rubber stamped by Oliver.
 620
 621         - move ubench tests properly.
 622
 623         * tests/ubench: Added.
 624         * tests/ubench/function-closure.js: Added.
 625         * tests/ubench/function-empty.js: Added.
 626         * tests/ubench/function-missing-args.js: Added.
 627         * tests/ubench/function-sum.js: Added.
 628         * tests/ubench/loop-empty-resolve.js: Added.
 629         * tests/ubench/loop-empty.js: Added.
 630         * tests/ubench/loop-sum.js: Added.
 631
 632 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
 633
 634         Rubber stamped by Oliver.
 635
 636         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
 637
 638 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
 639
 640         Reviewed by Oliver.
 641
 642         - make --squirrelfish mode use the real harness, but a reduced set of tests
 643
 644         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
 645
 646         * sunspider:
 647         * tests/LIST-SQUIRRELFISH:
 648         * tests/LIST-UBENCH: Added.
 649         * tests/squirrelfish: Moved to tests/ubench
 650
 651 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
 652
 653         Reviewed by Maciej Stachowiak.
 654
 655         Removed an unused test.
 656
 657         * tests/squirrelfish/loop-resolve.js: Removed.
 658
 659 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
 660
 661         Reviewed by Oliver.
 662
 663         Add newly runnable tests to --squirrelfish mode.
 664
 665         * tests/LIST-SQUIRRELFISH:
 666
 667 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
 668
 669         Reviewed by Maciej Stachowiak.
 670
 671         A little more love for --squirrelfish mode:
 672
 673         Fixed a misplaced sort. Results properly sort now.
 674
 675         Added a test list just for squirrelfish, and updated it to include
 676         all currently passing SunSpider tests.
 677
 678         Fixed the pruning regexp to match 3d-morph.js.
 679
 680         * sunspider:
 681
 682 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
 683
 684         Reviewed by Oliver Hunt.
 685
 686         --squirrelfish mode: pared down tests for squirrelfish to chew on.
 687
 688         * sunspider:
 689         * tests/LIST:
 690         * tests/squirrelfish: Added.
 691         * tests/squirrelfish/function-closure.js: Added.
 692         * tests/squirrelfish/function-empty.js: Added.
 693         * tests/squirrelfish/function-missing-args.js: Added.
 694         * tests/squirrelfish/function-sum.js: Added.
 695         * tests/squirrelfish/loop-empty-resolve.js: Added.
 696         * tests/squirrelfish/loop-empty.js: Added.
 697         * tests/squirrelfish/loop-resolve.js: Added.
 698         * tests/squirrelfish/loop-sum.js: Added.
 699
 700 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
 701
 702         Reviewed by Geoff.
 703
 704         * sunspider: Pause update daemon on Darwin to make results reliable
 705         again (workaround for <rdar://problem/5811127>).
 706
 707 2008-02-07  Eric Seidel  <eric@webkit.org>
 708
 709         Reviewed by Sam.
 710
 711         * sunspider: add an error message when ./tests/LIST is not found
 712
 713 2007-12-29  Alp Toker  <alp@atoker.com>
 714
 715         Fix typo.
 716
 717         * hosted/sunspider.html:
 718
 719 2007-12-29  Alp Toker  <alp@atoker.com>
 720
 721         Reviewed by Maciej.
 722
 723         Provide the current URL in the SunSpider results page. Useful for
 724         copying benchmark results out of browsers without a location bar or
 725         where the location bar can't handle long URLs.
 726
 727         * hosted/sunspider-results.html:
 728
 729 2007-12-20  Eric Seidel  <eric@webkit.org>
 730
 731         Reviewed by mjs.
 732
 733         * sunspider: increase default number of runs for more accurate numbers
 734
 735 2007-12-18  Mark Rowe  <mrowe@apple.com>
 736
 737         Fix typo.
 738
 739         * hosted/sunspider.html:
 740
 741 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
 742
 743         Reviewed by Eric.
 744
 745         - give an extra digit of precision for small differences.
 746
 747         * resources/sunspider-compare-results.js:
 748
 749 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
 750
 751         Reviewed by Eric.
 752
 753         - work around IE layout bugs
 754
 755         * hosted/sunspider.css:
 756
 757 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
 758
 759         Reviewed by Mark.
 760
 761         - fix compare feature
 762
 763         * hosted/sunspider-results.html:
 764
 765 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
 766
 767         Reviewed by Mark.
 768
 769         - Change the format of the output URL to be shorter, to meet IE constraints
 770
 771         * hosted/sunspider-driver.html:
 772         * hosted/sunspider-results.html:
 773
 774 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
 775
 776         Reviewed by Oliver.
 777
 778         - Add some explanatory test and a dumb logo to SunSpider web hosted version
 779
 780         * hosted/sunspider-driver.html:
 781         * hosted/sunspider-results.html:
 782         * hosted/sunspider.css:
 783         * hosted/sunspider.html:
 784
 785 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
 786
 787         Reviewed by Oliver.
 788
 789         - Make the web-hosted version of SunSpider look a little bit less drab.
 790
 791         * hosted/sunspider-driver.html:
 792         * hosted/sunspider-results.html:
 793         * hosted/sunspider.css: Added.
 794         * hosted/sunspider.html:
 795         * resources/TEMPLATE.html:
 796
 797 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
 798
 799         Reviewed by Sam.
 800
 801         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
 802
 803         A sample of what new-style results would look like:
 804
 805         TEST                   COMPARISON            FROM                 TO             DETAILS
 806         ...
 807         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
 808         ...
 809             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
 810
 811         * resources/sunspider-compare-results.js:
 812
 813 2007-11-14  Eric Seidel  <eric@webkit.org>
 814
 815         Reviewed by Sam.
 816
 817         * sunspider: add --shark-cache for L2 Cache Miss profiling
 818
 819 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
 820
 821         Reviewed by Darin Adler.
 822
 823         http://bugs.webkit.org/show_bug.cgi?id=15925
 824         SunSpider should check for Shark being installed
 825
 826         * sunspider: Explicitly test that shark command line tool is installed.
 827
 828 2007-11-04  Darin Adler  <darin@apple.com>
 829
 830         * hosted/sunspider.html: Tweaked formatting a bit.
 831
 832 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
 833
 834         Reviewed by Darin Adler.
 835
 836         - Add compare mode and a description of the benchmark to the hosted version.
 837
 838         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
 839         * hosted/sunspider.html: Added intro text.
 840         * make-hosted: Copy new file.
 841         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
 842         a function.
 843         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
 844         expects.
 845         * sunspider-compare-results: Use sunspider-standalone-compare.js.
 846
 847 2007-11-02  Darin Adler  <darin@apple.com>
 848
 849         * sunspider: Changed "--base" to "--set-baseline".
 850
 851 2007-11-02  Darin Adler  <darin@apple.com>
 852
 853         Reviewed by Maciej.
 854
 855         * sunspider: Added a "--base" option that records the current test as a baseline.
 856         Moved the code to find the newest .mshark file into a function (since I needed to use
 857         it in the other script and I prefer to copy/paste a function).
 858
 859         * sunspider-compare-results: Added logic to compare the baseline with the last
 860         run when you don't pass any parameters.
 861
 862 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 863
 864         Reviewed by Eric.
 865
 866         - Added list of ECMAScript3 features that SunSpider does not cover currently.
 867
 868         * UNCOVERED: Added.
 869
 870 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 871
 872         Reviewed by Oliver.
 873
 874         - Give better error messages when sunspider-compare-results fails
 875
 876         * sunspider-compare-results:
 877
 878 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 879
 880         Reviewed by Darin Adler.
 881
 882         - fix Date tests to work in command-line SpiderMonkey
 883
 884         * tests/date-format-tofte.js:
 885         * tests/date-format-xparb.js:
 886
 887 2007-10-29  Darin Adler  <darin@apple.com>
 888
 889         Reviewed by Maciej.
 890
 891         * sunspider: Added --shark20 option, to run Shark at its highest sample
 892         resolution instead of its default.
 893
 894 2007-10-28  Adam Roben  <aroben@apple.com>
 895
 896         Fix SunSpider on Windows
 897
 898         Reviewed by Eric.
 899
 900         * sunspider: Don't use colons in filenames.
 901
 902 2007-10-28  Darin Adler  <darin@apple.com>
 903
 904         Reviewed by Adam.
 905
 906         * resources/sunspider-analyze-results.js: Tweak the output format a little.
 907         Change so that when there's only one run we don't write out confidence
 908         intervals at all rather than writing "NaN" over and over again.
 909
 910         * sunspider: Use the time and date as a suffix on the results file, that way
 911         all the results are left behind in the tmp directory. This will make it easy
 912         for us to add features that compare past results in the future. Also rename
 913         the .mshark files using the same scheme.
 914
 915         * sunspider-compare-results: Relax the parsing rules so we can parse the
 916         results file exactly as it's emitted from sunspider.
 917
 918 2007-10-25  Adam Roben  <aroben@apple.com>
 919
 920         Strip carriage returns from results
 921
 922         Carriage returns were screwing up sunspider-compare-results on
 923         Windows.
 924
 925         Rubberstamped by Oliver.
 926
 927         * sunspider:
 928
 929 2007-10-24  Darin Adler  <darin@apple.com>
 930
 931         Reviewed by Eric.
 932
 933         * sunspider: Open the profile in shark after completing a run.
 934         * sunspider-compare-results: Tiny perl tweaks.
 935
 936 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 937
 938         Reviewed by Darin Adler.
 939
 940         - Don't hardcode my path to testkjs
 941
 942         * sunspider-compare-results:
 943
 944 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 945
 946         Reviewed by Darin Adler.
 947
 948         - Add a compare script to compare results, and improve formatting of standard results a bit.
 949
 950         * resources/sunspider-analyze-results.js:
 951         * resources/sunspider-compare-results.js: Added.
 952         * sunspider-compare-results: Added.
 953
 954 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 955
 956         Reviewed by Darin Adler.
 957
 958         * resources/sunspider-analyze-results.js: Do the error range
 959         properly, using the t-distribution instead of 1.96 (which was
 960         based on the normal distribution).
 961
 962         * sunspider: Print results in a way that will be friendlier to the
 963         soon-to-come compare mode.
 964
 965 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
 966
 967         Reviewed by Review.
 968
 969         - add more tests (probably the final set for now)
 970
 971         Already balanced. Added date, regexp, control flow, and a few
 972         more string and object/array access tests.
 973
 974         * tests/LIST:
 975         * tests/access-fannkuch.js: Added.
 976         * tests/access-nbody.js: Added.
 977         * tests/controlflow-recursive.js: Added.
 978         * tests/date-format-tofte.js: Added.
 979         * tests/date-format-xparb.js: Added.
 980         * tests/regexp-dna.js: Added.
 981         * tests/string-validate-input.js: Added.
 982
 983 2007-10-21  Eric Seidel  <eric@webkit.org>
 984
 985         Reviewed by mjs.
 986
 987         * sunspider: make --shark only sample from testkjs
 988
 989 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 990
 991         Rubber stamped by Eric.
 992
 993         - added crypto tests (already balanced)
 994
 995         * tests/LIST:
 996         * tests/crypto-aes.js: Added.
 997         * tests/crypto-md5.js: Added.
 998         * tests/crypto-sha1.js: Added.
 999
1000 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
1001
1002         Reviewed by Adam.
1003
1004         - rebalance new tests
1005
1006         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
1007
1008         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1009
1010         This balances them with the existing tests.
1011
1012         * tests/access-binary-trees.js:
1013         * tests/access-nsieve.js:
1014         * tests/bitops-nsieve-bits.js:
1015         * tests/math-partial-sums.js:
1016         * tests/math-spectral-norm.js:
1017         * tests/string-fasta.js:
1018
1019 2007-10-20  Eric Seidel  <eric@webkit.org>
1020
1021         Reviewed by darin.
1022
1023         Add improved argument handling to sunspider, including
1024         --runs=<number>, --tests=<pattern, --shark, and --help
1025         Also re-factor code into subroutines
1026
1027         * sunspider: improved argument handling
1028
1029 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1030
1031         Reviewed by Mark.
1032
1033         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
1034
1035         * TODO:
1036         * tests/LIST:
1037         * tests/access-binary-trees.js: Added.
1038         * tests/access-nsieve.js: Added.
1039         * tests/bitops-nsieve-bits.js: Added.
1040         * tests/math-partial-sums.js: Added.
1041         * tests/math-spectral-norm.js: Added.
1042         * tests/string-fasta.js: Added.
1043
1044 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1045
1046         Reviewed by Darin Adler.
1047
1048         * TODO: Updated to note areas that are now well-covered.
1049
1050         - Change the tests and the driver to avoid leaving large object
1051         graphs hanging around, since that throws off the subsequent tests.
1052
1053         * resources/sunspider-standalone-driver.js:
1054         * tests/3d-cube.js:
1055         * tests/3d-morph.js:
1056         * tests/3d-raytrace.js:
1057         * tests/bitops-bitwise-and.js:
1058         * tests/string-base64.js:
1059         * tests/string-tagcloud.js:
1060
1061 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1062
1063         Reviewed by Nikolas.
1064
1065         - Rebalance test complexity.
1066
1067         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
1068
1069         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1070
1071         However, for some tests, IE7 was a huge outlier compared to other
1072         major browsers. For those tests, I used Firefox 2.0.0.8 on the
1073         same OS and hardware as a normalization basis.
1074
1075         * tests/3d-morph.js:
1076         * tests/3d-raytrace.js:
1077         * tests/bitops-3bit-bits-in-byte.js:
1078         * tests/bitops-bits-in-byte.js:
1079         * tests/bitops-bitwise-and.js:
1080         * tests/math-cordic.js:
1081         * tests/string-base64.js:
1082         * tests/string-tagcloud.js:
1083         * tests/string-unpack-code.js:
1084
1085 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1086
1087         Reviewed by Oliver.
1088
1089         - Fix the browser-hosted driver to be compatible with Safari 2
1090
1091         * hosted/sunspider-driver.html:
1092
1093 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1094
1095         Rubber stamped by Hyatt.
1096
1097         - Add BSD license notice to cordic test, the original author said to do so.
1098
1099         * tests/math-cordic.js:
1100
1101 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1102
1103         Reviewed by Eric.
1104
1105         - Support Windows and Windows browsers.
1106
1107         * hosted/sunspider-results.html: Display results correctly in IE.
1108         * sunspider: tweaks to work on Windows, and with spaces in the path.
1109
1110 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1111
1112         Rubber stamped by Oliver.
1113
1114         * tests/string-tagcloud.js: Remove stray print()
1115
1116 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
1117
1118         Reviewed by Oliver.
1119
1120         Add some string test cases.
1121
1122         * tests/string-base64.js: Added. Does base64 encoding and decoding.
1123         * tests/string-tagcloud.js: Added. Parses a JSON string and
1124         generates tagcloud HTML markup based on the data.
1125         * tests/string-unpack-code.js: Added. Unpack compressed versions of
1126         four of the most popular JavaScript libraries.
1127         * tests/LIST: Added the new tests.
1128
1129 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
1130
1131         Reviewed by Anders.
1132
1133         * TODO: Updated with info on how to do the stats properly.
1134
1135         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
1136         avoid overly slowing down the benchmark.
1137
1138         Added some 3D-related test cases.
1139
1140         * tests/3d-cube.js: Added.
1141         * tests/3d-morph.js: Added.
1142         * tests/3d-raytrace.js: Added.
1143         * tests/LIST:
1144
1145 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
1146
1147         Reviewed by Darin Adler.
1148
1149         - New JavaScript benchmark
1150         http://bugs.webkit.org/show_bug.cgi?id=15515
1151
1152         A start on a new JavaScript benchmark, with standalone and
1153         browser-hosted drivers.
1154
1155         The test content is still incomplete and the drivers could use
1156         some more features but there is enough here to make a good start.
1157
1158         The drivers compute a 95% confidence interval on the mean for the
1159         whole test, each category, and each individual test to make it
1160         easier to tell whether differences are statistically
1161         significant. The confidence interval can be narrowed by running
1162         the test more times.
1163
1164         * TODO: Added. Remaining things that need to be done.
1165         * sunspider: Added. Perl script that acts as the standalone test driver.
1166         * resources/sunspider-standalone-driver.js: Added. JavaScript part
1167         of standalone test driver.
1168         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
1169         analysis code.
1170         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
1171         * make-hosted: Added. Script to generate browser-hosted tests.
1172         * hosted/sunspider.html: Added. Start page for browser-hosted test.
1173         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
1174         * hosted/sunspider-results.html: Added. Results page for browser-hosted
1175         tests.
1176         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
1177         tests.
1178         * tests/LIST: Added. List of tests to use.
1179         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
1180         * tests/bitops-bits-in-byte.js: Added. Ditto.
1181         * tests/bitops-bitwise-and.js: Added. Ditto.
1182         * tests/math-cordic.js: Added. Ditto.
1183