SunSpider/ChangeLog

   1 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
   2
   3         Rubber-stamped by Oliver Hunt.
   4
   5         Make SunSpider work with JavaScript shells that don't take filenames
   6         after a '-f' argument.
   7
   8         * sunspider:
   9
  10 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
  11
  12         Reviewed by Timothy.
  13
  14         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
  15
  16         * sunspider:
  17         * tests/LIST-SQUIRRELFISH: Removed.
  18
  19 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
  20
  21         Rubber-stamped by Mark Rowe.
  22
  23         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
  24
  25         * sunspider:
  26
  27 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
  28
  29         Rubber stamped by Oliver.
  30
  31         - add newly working tests to squirrelfish list
  32
  33         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
  34
  35 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
  36
  37         Rubber stamped by Oliver.
  38
  39         - move ubench tests properly.
  40
  41         * tests/ubench: Added.
  42         * tests/ubench/function-closure.js: Added.
  43         * tests/ubench/function-empty.js: Added.
  44         * tests/ubench/function-missing-args.js: Added.
  45         * tests/ubench/function-sum.js: Added.
  46         * tests/ubench/loop-empty-resolve.js: Added.
  47         * tests/ubench/loop-empty.js: Added.
  48         * tests/ubench/loop-sum.js: Added.
  49
  50 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
  51
  52         Rubber stamped by Oliver.
  53
  54         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
  55
  56 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
  57
  58         Reviewed by Oliver.
  59
  60         - make --squirrelfish mode use the real harness, but a reduced set of tests
  61
  62         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
  63
  64         * sunspider:
  65         * tests/LIST-SQUIRRELFISH:
  66         * tests/LIST-UBENCH: Added.
  67         * tests/squirrelfish: Moved to tests/ubench
  68
  69 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
  70
  71         Reviewed by Maciej Stachowiak.
  72
  73         Removed an unused test.
  74
  75         * tests/squirrelfish/loop-resolve.js: Removed.
  76
  77 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
  78
  79         Reviewed by Oliver.
  80
  81         Add newly runnable tests to --squirrelfish mode.
  82
  83         * tests/LIST-SQUIRRELFISH:
  84
  85 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
  86
  87         Reviewed by Maciej Stachowiak.
  88
  89         A little more love for --squirrelfish mode:
  90
  91         Fixed a misplaced sort. Results properly sort now.
  92
  93         Added a test list just for squirrelfish, and updated it to include
  94         all currently passing SunSpider tests.
  95
  96         Fixed the pruning regexp to match 3d-morph.js.
  97
  98         * sunspider:
  99
 100 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
 101
 102         Reviewed by Oliver Hunt.
 103
 104         --squirrelfish mode: pared down tests for squirrelfish to chew on.
 105
 106         * sunspider:
 107         * tests/LIST:
 108         * tests/squirrelfish: Added.
 109         * tests/squirrelfish/function-closure.js: Added.
 110         * tests/squirrelfish/function-empty.js: Added.
 111         * tests/squirrelfish/function-missing-args.js: Added.
 112         * tests/squirrelfish/function-sum.js: Added.
 113         * tests/squirrelfish/loop-empty-resolve.js: Added.
 114         * tests/squirrelfish/loop-empty.js: Added.
 115         * tests/squirrelfish/loop-resolve.js: Added.
 116         * tests/squirrelfish/loop-sum.js: Added.
 117
 118 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
 119
 120         Reviewed by Geoff.
 121
 122         * sunspider: Pause update daemon on Darwin to make results reliable
 123         again (workaround for <rdar://problem/5811127>).
 124
 125 2008-02-07  Eric Seidel  <eric@webkit.org>
 126
 127         Reviewed by Sam.
 128
 129         * sunspider: add an error message when ./tests/LIST is not found
 130
 131 2007-12-29  Alp Toker  <alp@atoker.com>
 132
 133         Fix typo.
 134
 135         * hosted/sunspider.html:
 136
 137 2007-12-29  Alp Toker  <alp@atoker.com>
 138
 139         Reviewed by Maciej.
 140
 141         Provide the current URL in the SunSpider results page. Useful for
 142         copying benchmark results out of browsers without a location bar or
 143         where the location bar can't handle long URLs.
 144
 145         * hosted/sunspider-results.html:
 146
 147 2007-12-20  Eric Seidel  <eric@webkit.org>
 148
 149         Reviewed by mjs.
 150
 151         * sunspider: increase default number of runs for more accurate numbers
 152
 153 2007-12-18  Mark Rowe  <mrowe@apple.com>
 154
 155         Fix typo.
 156
 157         * hosted/sunspider.html:
 158
 159 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
 160
 161         Reviewed by Eric.
 162
 163         - give an extra digit of precision for small differences.
 164
 165         * resources/sunspider-compare-results.js:
 166
 167 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
 168
 169         Reviewed by Eric.
 170
 171         - work around IE layout bugs
 172
 173         * hosted/sunspider.css:
 174
 175 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
 176
 177         Reviewed by Mark.
 178
 179         - fix compare feature
 180
 181         * hosted/sunspider-results.html:
 182
 183 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
 184
 185         Reviewed by Mark.
 186
 187         - Change the format of the output URL to be shorter, to meet IE constraints
 188
 189         * hosted/sunspider-driver.html:
 190         * hosted/sunspider-results.html:
 191
 192 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
 193
 194         Reviewed by Oliver.
 195
 196         - Add some explanatory test and a dumb logo to SunSpider web hosted version
 197
 198         * hosted/sunspider-driver.html:
 199         * hosted/sunspider-results.html:
 200         * hosted/sunspider.css:
 201         * hosted/sunspider.html:
 202
 203 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
 204
 205         Reviewed by Oliver.
 206
 207         - Make the web-hosted version of SunSpider look a little bit less drab.
 208
 209         * hosted/sunspider-driver.html:
 210         * hosted/sunspider-results.html:
 211         * hosted/sunspider.css: Added.
 212         * hosted/sunspider.html:
 213         * resources/TEMPLATE.html:
 214
 215 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
 216
 217         Reviewed by Sam.
 218
 219         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
 220
 221         A sample of what new-style results would look like:
 222
 223         TEST                   COMPARISON            FROM                 TO             DETAILS
 224         ...
 225         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
 226         ...
 227             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
 228
 229         * resources/sunspider-compare-results.js:
 230
 231 2007-11-14  Eric Seidel  <eric@webkit.org>
 232
 233         Reviewed by Sam.
 234
 235         * sunspider: add --shark-cache for L2 Cache Miss profiling
 236
 237 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
 238
 239         Reviewed by Darin.
 240
 241         http://bugs.webkit.org/show_bug.cgi?id=15925
 242         SunSpider should check for Shark being installed
 243
 244         * sunspider: Explicitly test that shark command line tool is installed.
 245
 246 2007-11-04  Darin Adler  <darin@apple.com>
 247
 248         * hosted/sunspider.html: Tweaked formatting a bit.
 249
 250 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
 251
 252         Reviewed by Darin.
 253
 254         - Add compare mode and a description of the benchmark to the hosted version.
 255
 256         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
 257         * hosted/sunspider.html: Added intro text.
 258         * make-hosted: Copy new file.
 259         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
 260         a function.
 261         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
 262         expects.
 263         * sunspider-compare-results: Use sunspider-standalone-compare.js.
 264
 265 2007-11-02  Darin Adler  <darin@apple.com>
 266
 267         * sunspider: Changed "--base" to "--set-baseline".
 268
 269 2007-11-02  Darin Adler  <darin@apple.com>
 270
 271         Reviewed by Maciej.
 272
 273         * sunspider: Added a "--base" option that records the current test as a baseline.
 274         Moved the code to find the newest .mshark file into a function (since I needed to use
 275         it in the other script and I prefer to copy/paste a function).
 276
 277         * sunspider-compare-results: Added logic to compare the baseline with the last
 278         run when you don't pass any parameters.
 279
 280 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 281
 282         Reviewed by Eric.
 283
 284         - Added list of ECMAScript3 features that SunSpider does not cover currently.
 285
 286         * UNCOVERED: Added.
 287
 288 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 289
 290         Reviewed by Oliver.
 291
 292         - Give better error messages when sunspider-compare-results fails
 293
 294         * sunspider-compare-results:
 295
 296 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 297
 298         Reviewed by Darin.
 299
 300         - fix Date tests to work in command-line SpiderMonkey
 301
 302         * tests/date-format-tofte.js:
 303         * tests/date-format-xparb.js:
 304
 305 2007-10-29  Darin Adler  <darin@apple.com>
 306
 307         Reviewed by Maciej.
 308
 309         * sunspider: Added --shark20 option, to run Shark at its highest sample
 310         resolution instead of its default.
 311
 312 2007-10-28  Adam Roben  <aroben@apple.com>
 313
 314         Fix SunSpider on Windows
 315
 316         Reviewed by Eric.
 317
 318         * sunspider: Don't use colons in filenames.
 319
 320 2007-10-28  Darin Adler  <darin@apple.com>
 321
 322         Reviewed by Adam.
 323
 324         * resources/sunspider-analyze-results.js: Tweak the output format a little.
 325         Change so that when there's only one run we don't write out confidence
 326         intervals at all rather than writing "NaN" over and over again.
 327
 328         * sunspider: Use the time and date as a suffix on the results file, that way
 329         all the results are left behind in the tmp directory. This will make it easy
 330         for us to add features that compare past results in the future. Also rename
 331         the .mshark files using the same scheme.
 332
 333         * sunspider-compare-results: Relax the parsing rules so we can parse the
 334         results file exactly as it's emitted from sunspider.
 335
 336 2007-10-25  Adam Roben  <aroben@apple.com>
 337
 338         Strip carriage returns from results
 339
 340         Carriage returns were screwing up sunspider-compare-results on
 341         Windows.
 342
 343         Rubberstamped by Oliver.
 344
 345         * sunspider:
 346
 347 2007-10-24  Darin Adler  <darin@apple.com>
 348
 349         Reviewed by Eric.
 350
 351         * sunspider: Open the profile in shark after completing a run.
 352         * sunspider-compare-results: Tiny perl tweaks.
 353
 354 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 355
 356         Reviewed by Darin.
 357
 358         - Don't hardcode my path to testkjs
 359
 360         * sunspider-compare-results:
 361
 362 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 363
 364         Reviewed by Darin.
 365
 366         - Add a compare script to compare results, and improve formatting of standard results a bit.
 367
 368         * resources/sunspider-analyze-results.js:
 369         * resources/sunspider-compare-results.js: Added.
 370         * sunspider-compare-results: Added.
 371
 372 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 373
 374         Reviewed by Darin.
 375
 376         * resources/sunspider-analyze-results.js: Do the error range
 377         properly, using the t-distribution instead of 1.96 (which was
 378         based on the normal distribution).
 379
 380         * sunspider: Print results in a way that will be friendlier to the
 381         soon-to-come compare mode.
 382
 383 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
 384
 385         Reviewed by Review.
 386
 387         - add more tests (probably the final set for now)
 388
 389         Already balanced. Added date, regexp, control flow, and a few
 390         more string and object/array access tests.
 391
 392         * tests/LIST:
 393         * tests/access-fannkuch.js: Added.
 394         * tests/access-nbody.js: Added.
 395         * tests/controlflow-recursive.js: Added.
 396         * tests/date-format-tofte.js: Added.
 397         * tests/date-format-xparb.js: Added.
 398         * tests/regexp-dna.js: Added.
 399         * tests/string-validate-input.js: Added.
 400
 401 2007-10-21  Eric Seidel  <eric@webkit.org>
 402
 403         Reviewed by mjs.
 404
 405         * sunspider: make --shark only sample from testkjs
 406
 407 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 408
 409         Rubber stamped by Eric.
 410
 411         - added crypto tests (already balanced)
 412
 413         * tests/LIST:
 414         * tests/crypto-aes.js: Added.
 415         * tests/crypto-md5.js: Added.
 416         * tests/crypto-sha1.js: Added.
 417
 418 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 419
 420         Reviewed by Adam.
 421
 422         - rebalance new tests
 423
 424         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
 425
 426         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 427
 428         This balances them with the existing tests.
 429
 430         * tests/access-binary-trees.js:
 431         * tests/access-nsieve.js:
 432         * tests/bitops-nsieve-bits.js:
 433         * tests/math-partial-sums.js:
 434         * tests/math-spectral-norm.js:
 435         * tests/string-fasta.js:
 436
 437 2007-10-20  Eric Seidel  <eric@webkit.org>
 438
 439         Reviewed by darin.
 440
 441         Add improved argument handling to sunspider, including
 442         --runs=<number>, --tests=<pattern, --shark, and --help
 443         Also re-factor code into subroutines
 444
 445         * sunspider: improved argument handling
 446
 447 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 448
 449         Reviewed by Mark.
 450
 451         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
 452
 453         * TODO:
 454         * tests/LIST:
 455         * tests/access-binary-trees.js: Added.
 456         * tests/access-nsieve.js: Added.
 457         * tests/bitops-nsieve-bits.js: Added.
 458         * tests/math-partial-sums.js: Added.
 459         * tests/math-spectral-norm.js: Added.
 460         * tests/string-fasta.js: Added.
 461
 462 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 463
 464         Reviewed by Darin.
 465
 466         * TODO: Updated to note areas that are now well-covered.
 467
 468         - Change the tests and the driver to avoid leaving large object
 469         graphs hanging around, since that throws off the subsequent tests.
 470
 471         * resources/sunspider-standalone-driver.js:
 472         * tests/3d-cube.js:
 473         * tests/3d-morph.js:
 474         * tests/3d-raytrace.js:
 475         * tests/bitops-bitwise-and.js:
 476         * tests/string-base64.js:
 477         * tests/string-tagcloud.js:
 478
 479 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 480
 481         Reviewed by Nikolas.
 482
 483         - Rebalance test complexity.
 484
 485         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
 486
 487         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 488
 489         However, for some tests, IE7 was a huge outlier compared to other
 490         major browsers. For those tests, I used Firefox 2.0.0.8 on the
 491         same OS and hardware as a normalization basis.
 492
 493         * tests/3d-morph.js:
 494         * tests/3d-raytrace.js:
 495         * tests/bitops-3bit-bits-in-byte.js:
 496         * tests/bitops-bits-in-byte.js:
 497         * tests/bitops-bitwise-and.js:
 498         * tests/math-cordic.js:
 499         * tests/string-base64.js:
 500         * tests/string-tagcloud.js:
 501         * tests/string-unpack-code.js:
 502
 503 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 504
 505         Reviewed by Oliver.
 506
 507         - Fix the browser-hosted driver to be compatible with Safari 2
 508
 509         * hosted/sunspider-driver.html:
 510
 511 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 512
 513         Rubber stamped by Hyatt.
 514
 515         - Add BSD license notice to cordic test, the original author said to do so.
 516
 517         * tests/math-cordic.js:
 518
 519 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 520
 521         Reviewed by Eric.
 522
 523         - Support Windows and Windows browsers.
 524
 525         * hosted/sunspider-results.html: Display results correctly in IE.
 526         * sunspider: tweaks to work on Windows, and with spaces in the path.
 527
 528 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 529
 530         Rubber stamped by Oliver.
 531
 532         * tests/string-tagcloud.js: Remove stray print()
 533
 534 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
 535
 536         Reviewed by Oliver.
 537
 538         Add some string test cases.
 539
 540         * tests/string-base64.js: Added. Does base64 encoding and decoding.
 541         * tests/string-tagcloud.js: Added. Parses a JSON string and
 542         generates tagcloud HTML markup based on the data.
 543         * tests/string-unpack-code.js: Added. Unpack compressed versions of
 544         four of the most popular JavaScript libraries.
 545         * tests/LIST: Added the new tests.
 546
 547 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
 548
 549         Reviewed by Anders.
 550
 551         * TODO: Updated with info on how to do the stats properly.
 552
 553         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
 554         avoid overly slowing down the benchmark.
 555
 556         Added some 3D-related test cases.
 557
 558         * tests/3d-cube.js: Added.
 559         * tests/3d-morph.js: Added.
 560         * tests/3d-raytrace.js: Added.
 561         * tests/LIST:
 562
 563 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
 564
 565         Reviewed by Darin.
 566
 567         - New JavaScript benchmark
 568         http://bugs.webkit.org/show_bug.cgi?id=15515
 569
 570         A start on a new JavaScript benchmark, with standalone and
 571         browser-hosted drivers.
 572
 573         The test content is still incomplete and the drivers could use
 574         some more features but there is enough here to make a good start.
 575
 576         The drivers compute a 95% confidence interval on the mean for the
 577         whole test, each category, and each individual test to make it
 578         easier to tell whether differences are statistically
 579         significant. The confidence interval can be narrowed by running
 580         the test more times.
 581
 582         * TODO: Added. Remaining things that need to be done.
 583         * sunspider: Added. Perl script that acts as the standalone test driver.
 584         * resources/sunspider-standalone-driver.js: Added. JavaScript part
 585         of standalone test driver.
 586         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
 587         analysis code.
 588         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
 589         * make-hosted: Added. Script to generate browser-hosted tests.
 590         * hosted/sunspider.html: Added. Start page for browser-hosted test.
 591         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
 592         * hosted/sunspider-results.html: Added. Results page for browser-hosted
 593         tests.
 594         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
 595         tests.
 596         * tests/LIST: Added. List of tests to use.
 597         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
 598         * tests/bitops-bits-in-byte.js: Added. Ditto.
 599         * tests/bitops-bitwise-and.js: Added. Ditto.
 600         * tests/math-cordic.js: Added. Ditto.
 601