SunSpider/ChangeLog

   1 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
   2
   3         Reviewed by Eric.
   4
   5         - work around IE layout bugs
   6
   7         * hosted/sunspider.css:
   8
   9 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
  10
  11         Reviewed by Mark.
  12
  13         - fix compare feature
  14
  15         * hosted/sunspider-results.html:
  16
  17 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
  18
  19         Reviewed by Mark.
  20
  21         - Change the format of the output URL to be shorter, to meet IE constraints
  22
  23         * hosted/sunspider-driver.html:
  24         * hosted/sunspider-results.html:
  25
  26 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
  27
  28         Reviewed by Oliver.
  29
  30         - Add some explanatory test and a dumb logo to SunSpider web hosted version
  31
  32         * hosted/sunspider-driver.html:
  33         * hosted/sunspider-results.html:
  34         * hosted/sunspider.css:
  35         * hosted/sunspider.html:
  36
  37 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
  38
  39         Reviewed by Oliver.
  40
  41         - Make the web-hosted version of SunSpider look a little bit less drab.
  42
  43         * hosted/sunspider-driver.html:
  44         * hosted/sunspider-results.html:
  45         * hosted/sunspider.css: Added.
  46         * hosted/sunspider.html:
  47         * resources/TEMPLATE.html:
  48
  49 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
  50
  51         Reviewed by Sam.
  52
  53         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
  54
  55         A sample of what new-style results would look like:
  56
  57         TEST                   COMPARISON            FROM                 TO             DETAILS
  58         ...
  59         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
  60         ...
  61             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
  62
  63         * resources/sunspider-compare-results.js:
  64
  65 2007-11-14  Eric Seidel  <eric@webkit.org>
  66
  67         Reviewed by Sam.
  68
  69         * sunspider: add --shark-cache for L2 Cache Miss profiling
  70
  71 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
  72
  73         Reviewed by Darin.
  74
  75         http://bugs.webkit.org/show_bug.cgi?id=15925
  76         SunSpider should check for Shark being installed
  77
  78         * sunspider: Explicitly test that shark command line tool is installed.
  79
  80 2007-11-04  Darin Adler  <darin@apple.com>
  81
  82         * hosted/sunspider.html: Tweaked formatting a bit.
  83
  84 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
  85
  86         Reviewed by Darin.
  87
  88         - Add compare mode and a description of the benchmark to the hosted version.
  89
  90         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
  91         * hosted/sunspider.html: Added intro text.
  92         * make-hosted: Copy new file.
  93         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
  94         a function.
  95         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
  96         expects.
  97         * sunspider-compare-results: Use sunspider-standalone-compare.js.
  98
  99 2007-11-02  Darin Adler  <darin@apple.com>
 100
 101         * sunspider: Changed "--base" to "--set-baseline".
 102
 103 2007-11-02  Darin Adler  <darin@apple.com>
 104
 105         Reviewed by Maciej.
 106
 107         * sunspider: Added a "--base" option that records the current test as a baseline.
 108         Moved the code to find the newest .mshark file into a function (since I needed to use
 109         it in the other script and I prefer to copy/paste a function).
 110
 111         * sunspider-compare-results: Added logic to compare the baseline with the last
 112         run when you don't pass any parameters.
 113
 114 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 115
 116         Reviewed by Eric.
 117
 118         - Added list of ECMAScript3 features that SunSpider does not cover currently.
 119
 120         * UNCOVERED: Added.
 121
 122 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 123
 124         Reviewed by Oliver.
 125
 126         - Give better error messages when sunspider-compare-results fails
 127
 128         * sunspider-compare-results:
 129
 130 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 131
 132         Reviewed by Darin.
 133
 134         - fix Date tests to work in command-line SpiderMonkey
 135
 136         * tests/date-format-tofte.js:
 137         * tests/date-format-xparb.js:
 138
 139 2007-10-29  Darin Adler  <darin@apple.com>
 140
 141         Reviewed by Maciej.
 142
 143         * sunspider: Added --shark20 option, to run Shark at its highest sample
 144         resolution instead of its default.
 145
 146 2007-10-28  Adam Roben  <aroben@apple.com>
 147
 148         Fix SunSpider on Windows
 149
 150         Reviewed by Eric.
 151
 152         * sunspider: Don't use colons in filenames.
 153
 154 2007-10-28  Darin Adler  <darin@apple.com>
 155
 156         Reviewed by Adam.
 157
 158         * resources/sunspider-analyze-results.js: Tweak the output format a little.
 159         Change so that when there's only one run we don't write out confidence
 160         intervals at all rather than writing "NaN" over and over again.
 161
 162         * sunspider: Use the time and date as a suffix on the results file, that way
 163         all the results are left behind in the tmp directory. This will make it easy
 164         for us to add features that compare past results in the future. Also rename
 165         the .mshark files using the same scheme.
 166
 167         * sunspider-compare-results: Relax the parsing rules so we can parse the
 168         results file exactly as it's emitted from sunspider.
 169
 170 2007-10-25  Adam Roben  <aroben@apple.com>
 171
 172         Strip carriage returns from results
 173
 174         Carriage returns were screwing up sunspider-compare-results on
 175         Windows.
 176
 177         Rubberstamped by Oliver.
 178
 179         * sunspider:
 180
 181 2007-10-24  Darin Adler  <darin@apple.com>
 182
 183         Reviewed by Eric.
 184
 185         * sunspider: Open the profile in shark after completing a run.
 186         * sunspider-compare-results: Tiny perl tweaks.
 187
 188 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 189
 190         Reviewed by Darin.
 191
 192         - Don't hardcode my path to testkjs
 193
 194         * sunspider-compare-results:
 195
 196 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 197
 198         Reviewed by Darin.
 199
 200         - Add a compare script to compare results, and improve formatting of standard results a bit.
 201
 202         * resources/sunspider-analyze-results.js:
 203         * resources/sunspider-compare-results.js: Added.
 204         * sunspider-compare-results: Added.
 205
 206 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 207
 208         Reviewed by Darin.
 209
 210         * resources/sunspider-analyze-results.js: Do the error range
 211         properly, using the t-distribution instead of 1.96 (which was
 212         based on the normal distribution).
 213
 214         * sunspider: Print results in a way that will be friendlier to the
 215         soon-to-come compare mode.
 216
 217 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
 218
 219         Reviewed by Review.
 220
 221         - add more tests (probably the final set for now)
 222
 223         Already balanced. Added date, regexp, control flow, and a few
 224         more string and object/array access tests.
 225
 226         * tests/LIST:
 227         * tests/access-fannkuch.js: Added.
 228         * tests/access-nbody.js: Added.
 229         * tests/controlflow-recursive.js: Added.
 230         * tests/date-format-tofte.js: Added.
 231         * tests/date-format-xparb.js: Added.
 232         * tests/regexp-dna.js: Added.
 233         * tests/string-validate-input.js: Added.
 234
 235 2007-10-21  Eric Seidel  <eric@webkit.org>
 236
 237         Reviewed by mjs.
 238
 239         * sunspider: make --shark only sample from testkjs
 240
 241 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 242
 243         Rubber stamped by Eric.
 244
 245         - added crypto tests (already balanced)
 246
 247         * tests/LIST:
 248         * tests/crypto-aes.js: Added.
 249         * tests/crypto-md5.js: Added.
 250         * tests/crypto-sha1.js: Added.
 251
 252 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 253
 254         Reviewed by Adam.
 255
 256         - rebalance new tests
 257
 258         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
 259
 260         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 261
 262         This balances them with the existing tests.
 263
 264         * tests/access-binary-trees.js:
 265         * tests/access-nsieve.js:
 266         * tests/bitops-nsieve-bits.js:
 267         * tests/math-partial-sums.js:
 268         * tests/math-spectral-norm.js:
 269         * tests/string-fasta.js:
 270
 271 2007-10-20  Eric Seidel  <eric@webkit.org>
 272
 273         Reviewed by darin.
 274
 275         Add improved argument handling to sunspider, including
 276         --runs=<number>, --tests=<pattern, --shark, and --help
 277         Also re-factor code into subroutines
 278
 279         * sunspider: improved argument handling
 280
 281 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 282
 283         Reviewed by Mark.
 284
 285         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
 286
 287         * TODO:
 288         * tests/LIST:
 289         * tests/access-binary-trees.js: Added.
 290         * tests/access-nsieve.js: Added.
 291         * tests/bitops-nsieve-bits.js: Added.
 292         * tests/math-partial-sums.js: Added.
 293         * tests/math-spectral-norm.js: Added.
 294         * tests/string-fasta.js: Added.
 295
 296 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 297
 298         Reviewed by Darin.
 299
 300         * TODO: Updated to note areas that are now well-covered.
 301
 302         - Change the tests and the driver to avoid leaving large object
 303         graphs hanging around, since that throws off the subsequent tests.
 304
 305         * resources/sunspider-standalone-driver.js:
 306         * tests/3d-cube.js:
 307         * tests/3d-morph.js:
 308         * tests/3d-raytrace.js:
 309         * tests/bitops-bitwise-and.js:
 310         * tests/string-base64.js:
 311         * tests/string-tagcloud.js:
 312
 313 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 314
 315         Reviewed by Nikolas.
 316
 317         - Rebalance test complexity.
 318
 319         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
 320
 321         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 322
 323         However, for some tests, IE7 was a huge outlier compared to other
 324         major browsers. For those tests, I used Firefox 2.0.0.8 on the
 325         same OS and hardware as a normalization basis.
 326
 327         * tests/3d-morph.js:
 328         * tests/3d-raytrace.js:
 329         * tests/bitops-3bit-bits-in-byte.js:
 330         * tests/bitops-bits-in-byte.js:
 331         * tests/bitops-bitwise-and.js:
 332         * tests/math-cordic.js:
 333         * tests/string-base64.js:
 334         * tests/string-tagcloud.js:
 335         * tests/string-unpack-code.js:
 336
 337 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 338
 339         Reviewed by Oliver.
 340
 341         - Fix the browser-hosted driver to be compatible with Safari 2
 342
 343         * hosted/sunspider-driver.html:
 344
 345 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 346
 347         Rubber stamped by Hyatt.
 348
 349         - Add BSD license notice to cordic test, the original author said to do so.
 350
 351         * tests/math-cordic.js:
 352
 353 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 354
 355         Reviewed by Eric.
 356
 357         - Support Windows and Windows browsers.
 358
 359         * hosted/sunspider-results.html: Display results correctly in IE.
 360         * sunspider: tweaks to work on Windows, and with spaces in the path.
 361
 362 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 363
 364         Rubber stamped by Oliver.
 365
 366         * tests/string-tagcloud.js: Remove stray print()
 367
 368 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
 369
 370         Reviewed by Oliver.
 371
 372         Add some string test cases.
 373
 374         * tests/string-base64.js: Added. Does base64 encoding and decoding.
 375         * tests/string-tagcloud.js: Added. Parses a JSON string and
 376         generates tagcloud HTML markup based on the data.
 377         * tests/string-unpack-code.js: Added. Unpack compressed versions of
 378         four of the most popular JavaScript libraries.
 379         * tests/LIST: Added the new tests.
 380
 381 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
 382
 383         Reviewed by Anders.
 384
 385         * TODO: Updated with info on how to do the stats properly.
 386
 387         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
 388         avoid overly slowing down the benchmark.
 389
 390         Added some 3D-related test cases.
 391
 392         * tests/3d-cube.js: Added.
 393         * tests/3d-morph.js: Added.
 394         * tests/3d-raytrace.js: Added.
 395         * tests/LIST:
 396
 397 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
 398
 399         Reviewed by Darin.
 400
 401         - New JavaScript benchmark
 402         http://bugs.webkit.org/show_bug.cgi?id=15515
 403
 404         A start on a new JavaScript benchmark, with standalone and
 405         browser-hosted drivers.
 406
 407         The test content is still incomplete and the drivers could use
 408         some more features but there is enough here to make a good start.
 409
 410         The drivers compute a 95% confidence interval on the mean for the
 411         whole test, each category, and each individual test to make it
 412         easier to tell whether differences are statistically
 413         significant. The confidence interval can be narrowed by running
 414         the test more times.
 415
 416         * TODO: Added. Remaining things that need to be done.
 417         * sunspider: Added. Perl script that acts as the standalone test driver.
 418         * resources/sunspider-standalone-driver.js: Added. JavaScript part
 419         of standalone test driver.
 420         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
 421         analysis code.
 422         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
 423         * make-hosted: Added. Script to generate browser-hosted tests.
 424         * hosted/sunspider.html: Added. Start page for browser-hosted test.
 425         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
 426         * hosted/sunspider-results.html: Added. Results page for browser-hosted
 427         tests.
 428         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
 429         tests.
 430         * tests/LIST: Added. List of tests to use.
 431         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
 432         * tests/bitops-bits-in-byte.js: Added. Ditto.
 433         * tests/bitops-bitwise-and.js: Added. Ditto.
 434         * tests/math-cordic.js: Added. Ditto.
 435