SunSpider/ChangeLog

   1 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
   2
   3         Reviewed by Oliver.
   4
   5         - Add some explanatory test and a dumb logo to SunSpider web hosted version
   6
   7         * hosted/sunspider-driver.html:
   8         * hosted/sunspider-results.html:
   9         * hosted/sunspider.css:
  10         * hosted/sunspider.html:
  11
  12 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
  13
  14         Reviewed by Oliver.
  15
  16         - Make the web-hosted version of SunSpider look a little bit less drab.
  17
  18         * hosted/sunspider-driver.html:
  19         * hosted/sunspider-results.html:
  20         * hosted/sunspider.css: Added.
  21         * hosted/sunspider.html:
  22         * resources/TEMPLATE.html:
  23
  24 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
  25
  26         Reviewed by Sam.
  27
  28         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
  29
  30         A sample of what new-style results would look like:
  31
  32         TEST                   COMPARISON            FROM                 TO             DETAILS
  33         ...
  34         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
  35         ...
  36             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
  37
  38         * resources/sunspider-compare-results.js:
  39
  40 2007-11-14  Eric Seidel  <eric@webkit.org>
  41
  42         Reviewed by Sam.
  43
  44         * sunspider: add --shark-cache for L2 Cache Miss profiling
  45
  46 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
  47
  48         Reviewed by Darin.
  49
  50         http://bugs.webkit.org/show_bug.cgi?id=15925
  51         SunSpider should check for Shark being installed
  52
  53         * sunspider: Explicitly test that shark command line tool is installed.
  54
  55 2007-11-04  Darin Adler  <darin@apple.com>
  56
  57         * hosted/sunspider.html: Tweaked formatting a bit.
  58
  59 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
  60
  61         Reviewed by Darin.
  62
  63         - Add compare mode and a description of the benchmark to the hosted version.
  64
  65         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
  66         * hosted/sunspider.html: Added intro text.
  67         * make-hosted: Copy new file.
  68         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
  69         a function.
  70         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
  71         expects.
  72         * sunspider-compare-results: Use sunspider-standalone-compare.js.
  73
  74 2007-11-02  Darin Adler  <darin@apple.com>
  75
  76         * sunspider: Changed "--base" to "--set-baseline".
  77
  78 2007-11-02  Darin Adler  <darin@apple.com>
  79
  80         Reviewed by Maciej.
  81
  82         * sunspider: Added a "--base" option that records the current test as a baseline.
  83         Moved the code to find the newest .mshark file into a function (since I needed to use
  84         it in the other script and I prefer to copy/paste a function).
  85
  86         * sunspider-compare-results: Added logic to compare the baseline with the last
  87         run when you don't pass any parameters.
  88
  89 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
  90
  91         Reviewed by Eric.
  92
  93         - Added list of ECMAScript3 features that SunSpider does not cover currently.
  94
  95         * UNCOVERED: Added.
  96
  97 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
  98
  99         Reviewed by Oliver.
 100
 101         - Give better error messages when sunspider-compare-results fails
 102
 103         * sunspider-compare-results:
 104
 105 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 106
 107         Reviewed by Darin.
 108
 109         - fix Date tests to work in command-line SpiderMonkey
 110
 111         * tests/date-format-tofte.js:
 112         * tests/date-format-xparb.js:
 113
 114 2007-10-29  Darin Adler  <darin@apple.com>
 115
 116         Reviewed by Maciej.
 117
 118         * sunspider: Added --shark20 option, to run Shark at its highest sample
 119         resolution instead of its default.
 120
 121 2007-10-28  Adam Roben  <aroben@apple.com>
 122
 123         Fix SunSpider on Windows
 124
 125         Reviewed by Eric.
 126
 127         * sunspider: Don't use colons in filenames.
 128
 129 2007-10-28  Darin Adler  <darin@apple.com>
 130
 131         Reviewed by Adam.
 132
 133         * resources/sunspider-analyze-results.js: Tweak the output format a little.
 134         Change so that when there's only one run we don't write out confidence
 135         intervals at all rather than writing "NaN" over and over again.
 136
 137         * sunspider: Use the time and date as a suffix on the results file, that way
 138         all the results are left behind in the tmp directory. This will make it easy
 139         for us to add features that compare past results in the future. Also rename
 140         the .mshark files using the same scheme.
 141
 142         * sunspider-compare-results: Relax the parsing rules so we can parse the
 143         results file exactly as it's emitted from sunspider.
 144
 145 2007-10-25  Adam Roben  <aroben@apple.com>
 146
 147         Strip carriage returns from results
 148
 149         Carriage returns were screwing up sunspider-compare-results on
 150         Windows.
 151
 152         Rubberstamped by Oliver.
 153
 154         * sunspider:
 155
 156 2007-10-24  Darin Adler  <darin@apple.com>
 157
 158         Reviewed by Eric.
 159
 160         * sunspider: Open the profile in shark after completing a run.
 161         * sunspider-compare-results: Tiny perl tweaks.
 162
 163 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 164
 165         Reviewed by Darin.
 166
 167         - Don't hardcode my path to testkjs
 168
 169         * sunspider-compare-results:
 170
 171 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 172
 173         Reviewed by Darin.
 174
 175         - Add a compare script to compare results, and improve formatting of standard results a bit.
 176
 177         * resources/sunspider-analyze-results.js:
 178         * resources/sunspider-compare-results.js: Added.
 179         * sunspider-compare-results: Added.
 180
 181 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 182
 183         Reviewed by Darin.
 184
 185         * resources/sunspider-analyze-results.js: Do the error range
 186         properly, using the t-distribution instead of 1.96 (which was
 187         based on the normal distribution).
 188
 189         * sunspider: Print results in a way that will be friendlier to the
 190         soon-to-come compare mode.
 191
 192 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
 193
 194         Reviewed by Review.
 195
 196         - add more tests (probably the final set for now)
 197
 198         Already balanced. Added date, regexp, control flow, and a few
 199         more string and object/array access tests.
 200
 201         * tests/LIST:
 202         * tests/access-fannkuch.js: Added.
 203         * tests/access-nbody.js: Added.
 204         * tests/controlflow-recursive.js: Added.
 205         * tests/date-format-tofte.js: Added.
 206         * tests/date-format-xparb.js: Added.
 207         * tests/regexp-dna.js: Added.
 208         * tests/string-validate-input.js: Added.
 209
 210 2007-10-21  Eric Seidel  <eric@webkit.org>
 211
 212         Reviewed by mjs.
 213
 214         * sunspider: make --shark only sample from testkjs
 215
 216 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 217
 218         Rubber stamped by Eric.
 219
 220         - added crypto tests (already balanced)
 221
 222         * tests/LIST:
 223         * tests/crypto-aes.js: Added.
 224         * tests/crypto-md5.js: Added.
 225         * tests/crypto-sha1.js: Added.
 226
 227 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 228
 229         Reviewed by Adam.
 230
 231         - rebalance new tests
 232
 233         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
 234
 235         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 236
 237         This balances them with the existing tests.
 238
 239         * tests/access-binary-trees.js:
 240         * tests/access-nsieve.js:
 241         * tests/bitops-nsieve-bits.js:
 242         * tests/math-partial-sums.js:
 243         * tests/math-spectral-norm.js:
 244         * tests/string-fasta.js:
 245
 246 2007-10-20  Eric Seidel  <eric@webkit.org>
 247
 248         Reviewed by darin.
 249
 250         Add improved argument handling to sunspider, including
 251         --runs=<number>, --tests=<pattern, --shark, and --help
 252         Also re-factor code into subroutines
 253
 254         * sunspider: improved argument handling
 255
 256 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 257
 258         Reviewed by Mark.
 259
 260         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
 261
 262         * TODO:
 263         * tests/LIST:
 264         * tests/access-binary-trees.js: Added.
 265         * tests/access-nsieve.js: Added.
 266         * tests/bitops-nsieve-bits.js: Added.
 267         * tests/math-partial-sums.js: Added.
 268         * tests/math-spectral-norm.js: Added.
 269         * tests/string-fasta.js: Added.
 270
 271 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 272
 273         Reviewed by Darin.
 274
 275         * TODO: Updated to note areas that are now well-covered.
 276
 277         - Change the tests and the driver to avoid leaving large object
 278         graphs hanging around, since that throws off the subsequent tests.
 279
 280         * resources/sunspider-standalone-driver.js:
 281         * tests/3d-cube.js:
 282         * tests/3d-morph.js:
 283         * tests/3d-raytrace.js:
 284         * tests/bitops-bitwise-and.js:
 285         * tests/string-base64.js:
 286         * tests/string-tagcloud.js:
 287
 288 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 289
 290         Reviewed by Nikolas.
 291
 292         - Rebalance test complexity.
 293
 294         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
 295
 296         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 297
 298         However, for some tests, IE7 was a huge outlier compared to other
 299         major browsers. For those tests, I used Firefox 2.0.0.8 on the
 300         same OS and hardware as a normalization basis.
 301
 302         * tests/3d-morph.js:
 303         * tests/3d-raytrace.js:
 304         * tests/bitops-3bit-bits-in-byte.js:
 305         * tests/bitops-bits-in-byte.js:
 306         * tests/bitops-bitwise-and.js:
 307         * tests/math-cordic.js:
 308         * tests/string-base64.js:
 309         * tests/string-tagcloud.js:
 310         * tests/string-unpack-code.js:
 311
 312 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 313
 314         Reviewed by Oliver.
 315
 316         - Fix the browser-hosted driver to be compatible with Safari 2
 317
 318         * hosted/sunspider-driver.html:
 319
 320 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 321
 322         Rubber stamped by Hyatt.
 323
 324         - Add BSD license notice to cordic test, the original author said to do so.
 325
 326         * tests/math-cordic.js:
 327
 328 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 329
 330         Reviewed by Eric.
 331
 332         - Support Windows and Windows browsers.
 333
 334         * hosted/sunspider-results.html: Display results correctly in IE.
 335         * sunspider: tweaks to work on Windows, and with spaces in the path.
 336
 337 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 338
 339         Rubber stamped by Oliver.
 340
 341         * tests/string-tagcloud.js: Remove stray print()
 342
 343 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
 344
 345         Reviewed by Oliver.
 346
 347         Add some string test cases.
 348
 349         * tests/string-base64.js: Added. Does base64 encoding and decoding.
 350         * tests/string-tagcloud.js: Added. Parses a JSON string and
 351         generates tagcloud HTML markup based on the data.
 352         * tests/string-unpack-code.js: Added. Unpack compressed versions of
 353         four of the most popular JavaScript libraries.
 354         * tests/LIST: Added the new tests.
 355
 356 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
 357
 358         Reviewed by Anders.
 359
 360         * TODO: Updated with info on how to do the stats properly.
 361
 362         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
 363         avoid overly slowing down the benchmark.
 364
 365         Added some 3D-related test cases.
 366
 367         * tests/3d-cube.js: Added.
 368         * tests/3d-morph.js: Added.
 369         * tests/3d-raytrace.js: Added.
 370         * tests/LIST:
 371
 372 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
 373
 374         Reviewed by Darin.
 375
 376         - New JavaScript benchmark
 377         http://bugs.webkit.org/show_bug.cgi?id=15515
 378
 379         A start on a new JavaScript benchmark, with standalone and
 380         browser-hosted drivers.
 381
 382         The test content is still incomplete and the drivers could use
 383         some more features but there is enough here to make a good start.
 384
 385         The drivers compute a 95% confidence interval on the mean for the
 386         whole test, each category, and each individual test to make it
 387         easier to tell whether differences are statistically
 388         significant. The confidence interval can be narrowed by running
 389         the test more times.
 390
 391         * TODO: Added. Remaining things that need to be done.
 392         * sunspider: Added. Perl script that acts as the standalone test driver.
 393         * resources/sunspider-standalone-driver.js: Added. JavaScript part
 394         of standalone test driver.
 395         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
 396         analysis code.
 397         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
 398         * make-hosted: Added. Script to generate browser-hosted tests.
 399         * hosted/sunspider.html: Added. Start page for browser-hosted test.
 400         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
 401         * hosted/sunspider-results.html: Added. Results page for browser-hosted
 402         tests.
 403         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
 404         tests.
 405         * tests/LIST: Added. List of tests to use.
 406         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
 407         * tests/bitops-bits-in-byte.js: Added. Ditto.
 408         * tests/bitops-bitwise-and.js: Added. Ditto.
 409         * tests/math-cordic.js: Added. Ditto.
 410