SunSpider/ChangeLog

   1 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
   2
   3         Not reviewed.
   4
   5         Remove a local change that was accidentally committed in r36034.
   6
   7         * resources/sunspider-standalone-driver.js:
   8
   9 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
  10
  11         Rubber-stamped by Oliver Hunt.
  12
  13         Make SunSpider work with JavaScript shells that don't take filenames
  14         after a '-f' argument.
  15
  16         * sunspider:
  17
  18 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
  19
  20         Reviewed by Timothy.
  21
  22         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
  23
  24         * sunspider:
  25         * tests/LIST-SQUIRRELFISH: Removed.
  26
  27 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
  28
  29         Rubber-stamped by Mark Rowe.
  30
  31         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
  32
  33         * sunspider:
  34
  35 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
  36
  37         Rubber stamped by Oliver.
  38
  39         - add newly working tests to squirrelfish list
  40
  41         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
  42
  43 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
  44
  45         Rubber stamped by Oliver.
  46
  47         - move ubench tests properly.
  48
  49         * tests/ubench: Added.
  50         * tests/ubench/function-closure.js: Added.
  51         * tests/ubench/function-empty.js: Added.
  52         * tests/ubench/function-missing-args.js: Added.
  53         * tests/ubench/function-sum.js: Added.
  54         * tests/ubench/loop-empty-resolve.js: Added.
  55         * tests/ubench/loop-empty.js: Added.
  56         * tests/ubench/loop-sum.js: Added.
  57
  58 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
  59
  60         Rubber stamped by Oliver.
  61
  62         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
  63
  64 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
  65
  66         Reviewed by Oliver.
  67
  68         - make --squirrelfish mode use the real harness, but a reduced set of tests
  69
  70         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
  71
  72         * sunspider:
  73         * tests/LIST-SQUIRRELFISH:
  74         * tests/LIST-UBENCH: Added.
  75         * tests/squirrelfish: Moved to tests/ubench
  76
  77 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
  78
  79         Reviewed by Maciej Stachowiak.
  80
  81         Removed an unused test.
  82
  83         * tests/squirrelfish/loop-resolve.js: Removed.
  84
  85 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
  86
  87         Reviewed by Oliver.
  88
  89         Add newly runnable tests to --squirrelfish mode.
  90
  91         * tests/LIST-SQUIRRELFISH:
  92
  93 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
  94
  95         Reviewed by Maciej Stachowiak.
  96
  97         A little more love for --squirrelfish mode:
  98
  99         Fixed a misplaced sort. Results properly sort now.
 100
 101         Added a test list just for squirrelfish, and updated it to include
 102         all currently passing SunSpider tests.
 103
 104         Fixed the pruning regexp to match 3d-morph.js.
 105
 106         * sunspider:
 107
 108 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
 109
 110         Reviewed by Oliver Hunt.
 111
 112         --squirrelfish mode: pared down tests for squirrelfish to chew on.
 113
 114         * sunspider:
 115         * tests/LIST:
 116         * tests/squirrelfish: Added.
 117         * tests/squirrelfish/function-closure.js: Added.
 118         * tests/squirrelfish/function-empty.js: Added.
 119         * tests/squirrelfish/function-missing-args.js: Added.
 120         * tests/squirrelfish/function-sum.js: Added.
 121         * tests/squirrelfish/loop-empty-resolve.js: Added.
 122         * tests/squirrelfish/loop-empty.js: Added.
 123         * tests/squirrelfish/loop-resolve.js: Added.
 124         * tests/squirrelfish/loop-sum.js: Added.
 125
 126 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
 127
 128         Reviewed by Geoff.
 129
 130         * sunspider: Pause update daemon on Darwin to make results reliable
 131         again (workaround for <rdar://problem/5811127>).
 132
 133 2008-02-07  Eric Seidel  <eric@webkit.org>
 134
 135         Reviewed by Sam.
 136
 137         * sunspider: add an error message when ./tests/LIST is not found
 138
 139 2007-12-29  Alp Toker  <alp@atoker.com>
 140
 141         Fix typo.
 142
 143         * hosted/sunspider.html:
 144
 145 2007-12-29  Alp Toker  <alp@atoker.com>
 146
 147         Reviewed by Maciej.
 148
 149         Provide the current URL in the SunSpider results page. Useful for
 150         copying benchmark results out of browsers without a location bar or
 151         where the location bar can't handle long URLs.
 152
 153         * hosted/sunspider-results.html:
 154
 155 2007-12-20  Eric Seidel  <eric@webkit.org>
 156
 157         Reviewed by mjs.
 158
 159         * sunspider: increase default number of runs for more accurate numbers
 160
 161 2007-12-18  Mark Rowe  <mrowe@apple.com>
 162
 163         Fix typo.
 164
 165         * hosted/sunspider.html:
 166
 167 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
 168
 169         Reviewed by Eric.
 170
 171         - give an extra digit of precision for small differences.
 172
 173         * resources/sunspider-compare-results.js:
 174
 175 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
 176
 177         Reviewed by Eric.
 178
 179         - work around IE layout bugs
 180
 181         * hosted/sunspider.css:
 182
 183 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
 184
 185         Reviewed by Mark.
 186
 187         - fix compare feature
 188
 189         * hosted/sunspider-results.html:
 190
 191 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
 192
 193         Reviewed by Mark.
 194
 195         - Change the format of the output URL to be shorter, to meet IE constraints
 196
 197         * hosted/sunspider-driver.html:
 198         * hosted/sunspider-results.html:
 199
 200 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
 201
 202         Reviewed by Oliver.
 203
 204         - Add some explanatory test and a dumb logo to SunSpider web hosted version
 205
 206         * hosted/sunspider-driver.html:
 207         * hosted/sunspider-results.html:
 208         * hosted/sunspider.css:
 209         * hosted/sunspider.html:
 210
 211 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
 212
 213         Reviewed by Oliver.
 214
 215         - Make the web-hosted version of SunSpider look a little bit less drab.
 216
 217         * hosted/sunspider-driver.html:
 218         * hosted/sunspider-results.html:
 219         * hosted/sunspider.css: Added.
 220         * hosted/sunspider.html:
 221         * resources/TEMPLATE.html:
 222
 223 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
 224
 225         Reviewed by Sam.
 226
 227         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
 228
 229         A sample of what new-style results would look like:
 230
 231         TEST                   COMPARISON            FROM                 TO             DETAILS
 232         ...
 233         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
 234         ...
 235             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
 236
 237         * resources/sunspider-compare-results.js:
 238
 239 2007-11-14  Eric Seidel  <eric@webkit.org>
 240
 241         Reviewed by Sam.
 242
 243         * sunspider: add --shark-cache for L2 Cache Miss profiling
 244
 245 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
 246
 247         Reviewed by Darin.
 248
 249         http://bugs.webkit.org/show_bug.cgi?id=15925
 250         SunSpider should check for Shark being installed
 251
 252         * sunspider: Explicitly test that shark command line tool is installed.
 253
 254 2007-11-04  Darin Adler  <darin@apple.com>
 255
 256         * hosted/sunspider.html: Tweaked formatting a bit.
 257
 258 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
 259
 260         Reviewed by Darin.
 261
 262         - Add compare mode and a description of the benchmark to the hosted version.
 263
 264         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
 265         * hosted/sunspider.html: Added intro text.
 266         * make-hosted: Copy new file.
 267         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
 268         a function.
 269         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
 270         expects.
 271         * sunspider-compare-results: Use sunspider-standalone-compare.js.
 272
 273 2007-11-02  Darin Adler  <darin@apple.com>
 274
 275         * sunspider: Changed "--base" to "--set-baseline".
 276
 277 2007-11-02  Darin Adler  <darin@apple.com>
 278
 279         Reviewed by Maciej.
 280
 281         * sunspider: Added a "--base" option that records the current test as a baseline.
 282         Moved the code to find the newest .mshark file into a function (since I needed to use
 283         it in the other script and I prefer to copy/paste a function).
 284
 285         * sunspider-compare-results: Added logic to compare the baseline with the last
 286         run when you don't pass any parameters.
 287
 288 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 289
 290         Reviewed by Eric.
 291
 292         - Added list of ECMAScript3 features that SunSpider does not cover currently.
 293
 294         * UNCOVERED: Added.
 295
 296 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 297
 298         Reviewed by Oliver.
 299
 300         - Give better error messages when sunspider-compare-results fails
 301
 302         * sunspider-compare-results:
 303
 304 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 305
 306         Reviewed by Darin.
 307
 308         - fix Date tests to work in command-line SpiderMonkey
 309
 310         * tests/date-format-tofte.js:
 311         * tests/date-format-xparb.js:
 312
 313 2007-10-29  Darin Adler  <darin@apple.com>
 314
 315         Reviewed by Maciej.
 316
 317         * sunspider: Added --shark20 option, to run Shark at its highest sample
 318         resolution instead of its default.
 319
 320 2007-10-28  Adam Roben  <aroben@apple.com>
 321
 322         Fix SunSpider on Windows
 323
 324         Reviewed by Eric.
 325
 326         * sunspider: Don't use colons in filenames.
 327
 328 2007-10-28  Darin Adler  <darin@apple.com>
 329
 330         Reviewed by Adam.
 331
 332         * resources/sunspider-analyze-results.js: Tweak the output format a little.
 333         Change so that when there's only one run we don't write out confidence
 334         intervals at all rather than writing "NaN" over and over again.
 335
 336         * sunspider: Use the time and date as a suffix on the results file, that way
 337         all the results are left behind in the tmp directory. This will make it easy
 338         for us to add features that compare past results in the future. Also rename
 339         the .mshark files using the same scheme.
 340
 341         * sunspider-compare-results: Relax the parsing rules so we can parse the
 342         results file exactly as it's emitted from sunspider.
 343
 344 2007-10-25  Adam Roben  <aroben@apple.com>
 345
 346         Strip carriage returns from results
 347
 348         Carriage returns were screwing up sunspider-compare-results on
 349         Windows.
 350
 351         Rubberstamped by Oliver.
 352
 353         * sunspider:
 354
 355 2007-10-24  Darin Adler  <darin@apple.com>
 356
 357         Reviewed by Eric.
 358
 359         * sunspider: Open the profile in shark after completing a run.
 360         * sunspider-compare-results: Tiny perl tweaks.
 361
 362 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 363
 364         Reviewed by Darin.
 365
 366         - Don't hardcode my path to testkjs
 367
 368         * sunspider-compare-results:
 369
 370 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 371
 372         Reviewed by Darin.
 373
 374         - Add a compare script to compare results, and improve formatting of standard results a bit.
 375
 376         * resources/sunspider-analyze-results.js:
 377         * resources/sunspider-compare-results.js: Added.
 378         * sunspider-compare-results: Added.
 379
 380 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 381
 382         Reviewed by Darin.
 383
 384         * resources/sunspider-analyze-results.js: Do the error range
 385         properly, using the t-distribution instead of 1.96 (which was
 386         based on the normal distribution).
 387
 388         * sunspider: Print results in a way that will be friendlier to the
 389         soon-to-come compare mode.
 390
 391 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
 392
 393         Reviewed by Review.
 394
 395         - add more tests (probably the final set for now)
 396
 397         Already balanced. Added date, regexp, control flow, and a few
 398         more string and object/array access tests.
 399
 400         * tests/LIST:
 401         * tests/access-fannkuch.js: Added.
 402         * tests/access-nbody.js: Added.
 403         * tests/controlflow-recursive.js: Added.
 404         * tests/date-format-tofte.js: Added.
 405         * tests/date-format-xparb.js: Added.
 406         * tests/regexp-dna.js: Added.
 407         * tests/string-validate-input.js: Added.
 408
 409 2007-10-21  Eric Seidel  <eric@webkit.org>
 410
 411         Reviewed by mjs.
 412
 413         * sunspider: make --shark only sample from testkjs
 414
 415 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 416
 417         Rubber stamped by Eric.
 418
 419         - added crypto tests (already balanced)
 420
 421         * tests/LIST:
 422         * tests/crypto-aes.js: Added.
 423         * tests/crypto-md5.js: Added.
 424         * tests/crypto-sha1.js: Added.
 425
 426 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 427
 428         Reviewed by Adam.
 429
 430         - rebalance new tests
 431
 432         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
 433
 434         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 435
 436         This balances them with the existing tests.
 437
 438         * tests/access-binary-trees.js:
 439         * tests/access-nsieve.js:
 440         * tests/bitops-nsieve-bits.js:
 441         * tests/math-partial-sums.js:
 442         * tests/math-spectral-norm.js:
 443         * tests/string-fasta.js:
 444
 445 2007-10-20  Eric Seidel  <eric@webkit.org>
 446
 447         Reviewed by darin.
 448
 449         Add improved argument handling to sunspider, including
 450         --runs=<number>, --tests=<pattern, --shark, and --help
 451         Also re-factor code into subroutines
 452
 453         * sunspider: improved argument handling
 454
 455 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 456
 457         Reviewed by Mark.
 458
 459         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
 460
 461         * TODO:
 462         * tests/LIST:
 463         * tests/access-binary-trees.js: Added.
 464         * tests/access-nsieve.js: Added.
 465         * tests/bitops-nsieve-bits.js: Added.
 466         * tests/math-partial-sums.js: Added.
 467         * tests/math-spectral-norm.js: Added.
 468         * tests/string-fasta.js: Added.
 469
 470 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 471
 472         Reviewed by Darin.
 473
 474         * TODO: Updated to note areas that are now well-covered.
 475
 476         - Change the tests and the driver to avoid leaving large object
 477         graphs hanging around, since that throws off the subsequent tests.
 478
 479         * resources/sunspider-standalone-driver.js:
 480         * tests/3d-cube.js:
 481         * tests/3d-morph.js:
 482         * tests/3d-raytrace.js:
 483         * tests/bitops-bitwise-and.js:
 484         * tests/string-base64.js:
 485         * tests/string-tagcloud.js:
 486
 487 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 488
 489         Reviewed by Nikolas.
 490
 491         - Rebalance test complexity.
 492
 493         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
 494
 495         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 496
 497         However, for some tests, IE7 was a huge outlier compared to other
 498         major browsers. For those tests, I used Firefox 2.0.0.8 on the
 499         same OS and hardware as a normalization basis.
 500
 501         * tests/3d-morph.js:
 502         * tests/3d-raytrace.js:
 503         * tests/bitops-3bit-bits-in-byte.js:
 504         * tests/bitops-bits-in-byte.js:
 505         * tests/bitops-bitwise-and.js:
 506         * tests/math-cordic.js:
 507         * tests/string-base64.js:
 508         * tests/string-tagcloud.js:
 509         * tests/string-unpack-code.js:
 510
 511 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 512
 513         Reviewed by Oliver.
 514
 515         - Fix the browser-hosted driver to be compatible with Safari 2
 516
 517         * hosted/sunspider-driver.html:
 518
 519 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 520
 521         Rubber stamped by Hyatt.
 522
 523         - Add BSD license notice to cordic test, the original author said to do so.
 524
 525         * tests/math-cordic.js:
 526
 527 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 528
 529         Reviewed by Eric.
 530
 531         - Support Windows and Windows browsers.
 532
 533         * hosted/sunspider-results.html: Display results correctly in IE.
 534         * sunspider: tweaks to work on Windows, and with spaces in the path.
 535
 536 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 537
 538         Rubber stamped by Oliver.
 539
 540         * tests/string-tagcloud.js: Remove stray print()
 541
 542 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
 543
 544         Reviewed by Oliver.
 545
 546         Add some string test cases.
 547
 548         * tests/string-base64.js: Added. Does base64 encoding and decoding.
 549         * tests/string-tagcloud.js: Added. Parses a JSON string and
 550         generates tagcloud HTML markup based on the data.
 551         * tests/string-unpack-code.js: Added. Unpack compressed versions of
 552         four of the most popular JavaScript libraries.
 553         * tests/LIST: Added the new tests.
 554
 555 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
 556
 557         Reviewed by Anders.
 558
 559         * TODO: Updated with info on how to do the stats properly.
 560
 561         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
 562         avoid overly slowing down the benchmark.
 563
 564         Added some 3D-related test cases.
 565
 566         * tests/3d-cube.js: Added.
 567         * tests/3d-morph.js: Added.
 568         * tests/3d-raytrace.js: Added.
 569         * tests/LIST:
 570
 571 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
 572
 573         Reviewed by Darin.
 574
 575         - New JavaScript benchmark
 576         http://bugs.webkit.org/show_bug.cgi?id=15515
 577
 578         A start on a new JavaScript benchmark, with standalone and
 579         browser-hosted drivers.
 580
 581         The test content is still incomplete and the drivers could use
 582         some more features but there is enough here to make a good start.
 583
 584         The drivers compute a 95% confidence interval on the mean for the
 585         whole test, each category, and each individual test to make it
 586         easier to tell whether differences are statistically
 587         significant. The confidence interval can be narrowed by running
 588         the test more times.
 589
 590         * TODO: Added. Remaining things that need to be done.
 591         * sunspider: Added. Perl script that acts as the standalone test driver.
 592         * resources/sunspider-standalone-driver.js: Added. JavaScript part
 593         of standalone test driver.
 594         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
 595         analysis code.
 596         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
 597         * make-hosted: Added. Script to generate browser-hosted tests.
 598         * hosted/sunspider.html: Added. Start page for browser-hosted test.
 599         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
 600         * hosted/sunspider-results.html: Added. Results page for browser-hosted
 601         tests.
 602         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
 603         tests.
 604         * tests/LIST: Added. List of tests to use.
 605         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
 606         * tests/bitops-bits-in-byte.js: Added. Ditto.
 607         * tests/bitops-bitwise-and.js: Added. Ditto.
 608         * tests/math-cordic.js: Added. Ditto.
 609