SunSpider/ChangeLog

   1 2007-12-29  Alp Toker  <alp@atoker.com>
   2
   3         Reviewed by Maciej.
   4
   5         Provide the current URL in the SunSpider results page. Useful for
   6         copying benchmark results out of browsers without a location bar or
   7         where the location bar can't handle long URLs.
   8
   9         * hosted/sunspider-results.html:
  10
  11 2007-12-20  Eric Seidel  <eric@webkit.org>
  12
  13         Reviewed by mjs.
  14
  15         * sunspider: increase default number of runs for more accurate numbers
  16
  17 2007-12-18  Mark Rowe  <mrowe@apple.com>
  18
  19         Fix typo.
  20
  21         * hosted/sunspider.html:
  22
  23 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
  24
  25         Reviewed by Eric.
  26
  27         - give an extra digit of precision for small differences.
  28
  29         * resources/sunspider-compare-results.js:
  30
  31 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
  32
  33         Reviewed by Eric.
  34
  35         - work around IE layout bugs
  36
  37         * hosted/sunspider.css:
  38
  39 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
  40
  41         Reviewed by Mark.
  42
  43         - fix compare feature
  44
  45         * hosted/sunspider-results.html:
  46
  47 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
  48
  49         Reviewed by Mark.
  50
  51         - Change the format of the output URL to be shorter, to meet IE constraints
  52
  53         * hosted/sunspider-driver.html:
  54         * hosted/sunspider-results.html:
  55
  56 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
  57
  58         Reviewed by Oliver.
  59
  60         - Add some explanatory test and a dumb logo to SunSpider web hosted version
  61
  62         * hosted/sunspider-driver.html:
  63         * hosted/sunspider-results.html:
  64         * hosted/sunspider.css:
  65         * hosted/sunspider.html:
  66
  67 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
  68
  69         Reviewed by Oliver.
  70
  71         - Make the web-hosted version of SunSpider look a little bit less drab.
  72
  73         * hosted/sunspider-driver.html:
  74         * hosted/sunspider-results.html:
  75         * hosted/sunspider.css: Added.
  76         * hosted/sunspider.html:
  77         * resources/TEMPLATE.html:
  78
  79 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
  80
  81         Reviewed by Sam.
  82
  83         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
  84
  85         A sample of what new-style results would look like:
  86
  87         TEST                   COMPARISON            FROM                 TO             DETAILS
  88         ...
  89         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
  90         ...
  91             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
  92
  93         * resources/sunspider-compare-results.js:
  94
  95 2007-11-14  Eric Seidel  <eric@webkit.org>
  96
  97         Reviewed by Sam.
  98
  99         * sunspider: add --shark-cache for L2 Cache Miss profiling
 100
 101 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
 102
 103         Reviewed by Darin.
 104
 105         http://bugs.webkit.org/show_bug.cgi?id=15925
 106         SunSpider should check for Shark being installed
 107
 108         * sunspider: Explicitly test that shark command line tool is installed.
 109
 110 2007-11-04  Darin Adler  <darin@apple.com>
 111
 112         * hosted/sunspider.html: Tweaked formatting a bit.
 113
 114 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
 115
 116         Reviewed by Darin.
 117
 118         - Add compare mode and a description of the benchmark to the hosted version.
 119
 120         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
 121         * hosted/sunspider.html: Added intro text.
 122         * make-hosted: Copy new file.
 123         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
 124         a function.
 125         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
 126         expects.
 127         * sunspider-compare-results: Use sunspider-standalone-compare.js.
 128
 129 2007-11-02  Darin Adler  <darin@apple.com>
 130
 131         * sunspider: Changed "--base" to "--set-baseline".
 132
 133 2007-11-02  Darin Adler  <darin@apple.com>
 134
 135         Reviewed by Maciej.
 136
 137         * sunspider: Added a "--base" option that records the current test as a baseline.
 138         Moved the code to find the newest .mshark file into a function (since I needed to use
 139         it in the other script and I prefer to copy/paste a function).
 140
 141         * sunspider-compare-results: Added logic to compare the baseline with the last
 142         run when you don't pass any parameters.
 143
 144 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 145
 146         Reviewed by Eric.
 147
 148         - Added list of ECMAScript3 features that SunSpider does not cover currently.
 149
 150         * UNCOVERED: Added.
 151
 152 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 153
 154         Reviewed by Oliver.
 155
 156         - Give better error messages when sunspider-compare-results fails
 157
 158         * sunspider-compare-results:
 159
 160 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 161
 162         Reviewed by Darin.
 163
 164         - fix Date tests to work in command-line SpiderMonkey
 165
 166         * tests/date-format-tofte.js:
 167         * tests/date-format-xparb.js:
 168
 169 2007-10-29  Darin Adler  <darin@apple.com>
 170
 171         Reviewed by Maciej.
 172
 173         * sunspider: Added --shark20 option, to run Shark at its highest sample
 174         resolution instead of its default.
 175
 176 2007-10-28  Adam Roben  <aroben@apple.com>
 177
 178         Fix SunSpider on Windows
 179
 180         Reviewed by Eric.
 181
 182         * sunspider: Don't use colons in filenames.
 183
 184 2007-10-28  Darin Adler  <darin@apple.com>
 185
 186         Reviewed by Adam.
 187
 188         * resources/sunspider-analyze-results.js: Tweak the output format a little.
 189         Change so that when there's only one run we don't write out confidence
 190         intervals at all rather than writing "NaN" over and over again.
 191
 192         * sunspider: Use the time and date as a suffix on the results file, that way
 193         all the results are left behind in the tmp directory. This will make it easy
 194         for us to add features that compare past results in the future. Also rename
 195         the .mshark files using the same scheme.
 196
 197         * sunspider-compare-results: Relax the parsing rules so we can parse the
 198         results file exactly as it's emitted from sunspider.
 199
 200 2007-10-25  Adam Roben  <aroben@apple.com>
 201
 202         Strip carriage returns from results
 203
 204         Carriage returns were screwing up sunspider-compare-results on
 205         Windows.
 206
 207         Rubberstamped by Oliver.
 208
 209         * sunspider:
 210
 211 2007-10-24  Darin Adler  <darin@apple.com>
 212
 213         Reviewed by Eric.
 214
 215         * sunspider: Open the profile in shark after completing a run.
 216         * sunspider-compare-results: Tiny perl tweaks.
 217
 218 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 219
 220         Reviewed by Darin.
 221
 222         - Don't hardcode my path to testkjs
 223
 224         * sunspider-compare-results:
 225
 226 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 227
 228         Reviewed by Darin.
 229
 230         - Add a compare script to compare results, and improve formatting of standard results a bit.
 231
 232         * resources/sunspider-analyze-results.js:
 233         * resources/sunspider-compare-results.js: Added.
 234         * sunspider-compare-results: Added.
 235
 236 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 237
 238         Reviewed by Darin.
 239
 240         * resources/sunspider-analyze-results.js: Do the error range
 241         properly, using the t-distribution instead of 1.96 (which was
 242         based on the normal distribution).
 243
 244         * sunspider: Print results in a way that will be friendlier to the
 245         soon-to-come compare mode.
 246
 247 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
 248
 249         Reviewed by Review.
 250
 251         - add more tests (probably the final set for now)
 252
 253         Already balanced. Added date, regexp, control flow, and a few
 254         more string and object/array access tests.
 255
 256         * tests/LIST:
 257         * tests/access-fannkuch.js: Added.
 258         * tests/access-nbody.js: Added.
 259         * tests/controlflow-recursive.js: Added.
 260         * tests/date-format-tofte.js: Added.
 261         * tests/date-format-xparb.js: Added.
 262         * tests/regexp-dna.js: Added.
 263         * tests/string-validate-input.js: Added.
 264
 265 2007-10-21  Eric Seidel  <eric@webkit.org>
 266
 267         Reviewed by mjs.
 268
 269         * sunspider: make --shark only sample from testkjs
 270
 271 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 272
 273         Rubber stamped by Eric.
 274
 275         - added crypto tests (already balanced)
 276
 277         * tests/LIST:
 278         * tests/crypto-aes.js: Added.
 279         * tests/crypto-md5.js: Added.
 280         * tests/crypto-sha1.js: Added.
 281
 282 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 283
 284         Reviewed by Adam.
 285
 286         - rebalance new tests
 287
 288         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
 289
 290         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 291
 292         This balances them with the existing tests.
 293
 294         * tests/access-binary-trees.js:
 295         * tests/access-nsieve.js:
 296         * tests/bitops-nsieve-bits.js:
 297         * tests/math-partial-sums.js:
 298         * tests/math-spectral-norm.js:
 299         * tests/string-fasta.js:
 300
 301 2007-10-20  Eric Seidel  <eric@webkit.org>
 302
 303         Reviewed by darin.
 304
 305         Add improved argument handling to sunspider, including
 306         --runs=<number>, --tests=<pattern, --shark, and --help
 307         Also re-factor code into subroutines
 308
 309         * sunspider: improved argument handling
 310
 311 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 312
 313         Reviewed by Mark.
 314
 315         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
 316
 317         * TODO:
 318         * tests/LIST:
 319         * tests/access-binary-trees.js: Added.
 320         * tests/access-nsieve.js: Added.
 321         * tests/bitops-nsieve-bits.js: Added.
 322         * tests/math-partial-sums.js: Added.
 323         * tests/math-spectral-norm.js: Added.
 324         * tests/string-fasta.js: Added.
 325
 326 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 327
 328         Reviewed by Darin.
 329
 330         * TODO: Updated to note areas that are now well-covered.
 331
 332         - Change the tests and the driver to avoid leaving large object
 333         graphs hanging around, since that throws off the subsequent tests.
 334
 335         * resources/sunspider-standalone-driver.js:
 336         * tests/3d-cube.js:
 337         * tests/3d-morph.js:
 338         * tests/3d-raytrace.js:
 339         * tests/bitops-bitwise-and.js:
 340         * tests/string-base64.js:
 341         * tests/string-tagcloud.js:
 342
 343 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 344
 345         Reviewed by Nikolas.
 346
 347         - Rebalance test complexity.
 348
 349         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
 350
 351         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 352
 353         However, for some tests, IE7 was a huge outlier compared to other
 354         major browsers. For those tests, I used Firefox 2.0.0.8 on the
 355         same OS and hardware as a normalization basis.
 356
 357         * tests/3d-morph.js:
 358         * tests/3d-raytrace.js:
 359         * tests/bitops-3bit-bits-in-byte.js:
 360         * tests/bitops-bits-in-byte.js:
 361         * tests/bitops-bitwise-and.js:
 362         * tests/math-cordic.js:
 363         * tests/string-base64.js:
 364         * tests/string-tagcloud.js:
 365         * tests/string-unpack-code.js:
 366
 367 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 368
 369         Reviewed by Oliver.
 370
 371         - Fix the browser-hosted driver to be compatible with Safari 2
 372
 373         * hosted/sunspider-driver.html:
 374
 375 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 376
 377         Rubber stamped by Hyatt.
 378
 379         - Add BSD license notice to cordic test, the original author said to do so.
 380
 381         * tests/math-cordic.js:
 382
 383 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 384
 385         Reviewed by Eric.
 386
 387         - Support Windows and Windows browsers.
 388
 389         * hosted/sunspider-results.html: Display results correctly in IE.
 390         * sunspider: tweaks to work on Windows, and with spaces in the path.
 391
 392 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 393
 394         Rubber stamped by Oliver.
 395
 396         * tests/string-tagcloud.js: Remove stray print()
 397
 398 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
 399
 400         Reviewed by Oliver.
 401
 402         Add some string test cases.
 403
 404         * tests/string-base64.js: Added. Does base64 encoding and decoding.
 405         * tests/string-tagcloud.js: Added. Parses a JSON string and
 406         generates tagcloud HTML markup based on the data.
 407         * tests/string-unpack-code.js: Added. Unpack compressed versions of
 408         four of the most popular JavaScript libraries.
 409         * tests/LIST: Added the new tests.
 410
 411 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
 412
 413         Reviewed by Anders.
 414
 415         * TODO: Updated with info on how to do the stats properly.
 416
 417         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
 418         avoid overly slowing down the benchmark.
 419
 420         Added some 3D-related test cases.
 421
 422         * tests/3d-cube.js: Added.
 423         * tests/3d-morph.js: Added.
 424         * tests/3d-raytrace.js: Added.
 425         * tests/LIST:
 426
 427 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
 428
 429         Reviewed by Darin.
 430
 431         - New JavaScript benchmark
 432         http://bugs.webkit.org/show_bug.cgi?id=15515
 433
 434         A start on a new JavaScript benchmark, with standalone and
 435         browser-hosted drivers.
 436
 437         The test content is still incomplete and the drivers could use
 438         some more features but there is enough here to make a good start.
 439
 440         The drivers compute a 95% confidence interval on the mean for the
 441         whole test, each category, and each individual test to make it
 442         easier to tell whether differences are statistically
 443         significant. The confidence interval can be narrowed by running
 444         the test more times.
 445
 446         * TODO: Added. Remaining things that need to be done.
 447         * sunspider: Added. Perl script that acts as the standalone test driver.
 448         * resources/sunspider-standalone-driver.js: Added. JavaScript part
 449         of standalone test driver.
 450         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
 451         analysis code.
 452         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
 453         * make-hosted: Added. Script to generate browser-hosted tests.
 454         * hosted/sunspider.html: Added. Start page for browser-hosted test.
 455         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
 456         * hosted/sunspider-results.html: Added. Results page for browser-hosted
 457         tests.
 458         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
 459         tests.
 460         * tests/LIST: Added. List of tests to use.
 461         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
 462         * tests/bitops-bits-in-byte.js: Added. Ditto.
 463         * tests/bitops-bitwise-and.js: Added. Ditto.
 464         * tests/math-cordic.js: Added. Ditto.
 465