SunSpider/ChangeLog

   1 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
   2
   3         Reviewed by Mark.
   4
   5         - fix compare feature
   6
   7         * hosted/sunspider-results.html:
   8
   9 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
  10
  11         Reviewed by Mark.
  12
  13         - Change the format of the output URL to be shorter, to meet IE constraints
  14
  15         * hosted/sunspider-driver.html:
  16         * hosted/sunspider-results.html:
  17
  18 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
  19
  20         Reviewed by Oliver.
  21
  22         - Add some explanatory test and a dumb logo to SunSpider web hosted version
  23
  24         * hosted/sunspider-driver.html:
  25         * hosted/sunspider-results.html:
  26         * hosted/sunspider.css:
  27         * hosted/sunspider.html:
  28
  29 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
  30
  31         Reviewed by Oliver.
  32
  33         - Make the web-hosted version of SunSpider look a little bit less drab.
  34
  35         * hosted/sunspider-driver.html:
  36         * hosted/sunspider-results.html:
  37         * hosted/sunspider.css: Added.
  38         * hosted/sunspider.html:
  39         * resources/TEMPLATE.html:
  40
  41 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
  42
  43         Reviewed by Sam.
  44
  45         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
  46
  47         A sample of what new-style results would look like:
  48
  49         TEST                   COMPARISON            FROM                 TO             DETAILS
  50         ...
  51         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
  52         ...
  53             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
  54
  55         * resources/sunspider-compare-results.js:
  56
  57 2007-11-14  Eric Seidel  <eric@webkit.org>
  58
  59         Reviewed by Sam.
  60
  61         * sunspider: add --shark-cache for L2 Cache Miss profiling
  62
  63 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
  64
  65         Reviewed by Darin.
  66
  67         http://bugs.webkit.org/show_bug.cgi?id=15925
  68         SunSpider should check for Shark being installed
  69
  70         * sunspider: Explicitly test that shark command line tool is installed.
  71
  72 2007-11-04  Darin Adler  <darin@apple.com>
  73
  74         * hosted/sunspider.html: Tweaked formatting a bit.
  75
  76 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
  77
  78         Reviewed by Darin.
  79
  80         - Add compare mode and a description of the benchmark to the hosted version.
  81
  82         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
  83         * hosted/sunspider.html: Added intro text.
  84         * make-hosted: Copy new file.
  85         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
  86         a function.
  87         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
  88         expects.
  89         * sunspider-compare-results: Use sunspider-standalone-compare.js.
  90
  91 2007-11-02  Darin Adler  <darin@apple.com>
  92
  93         * sunspider: Changed "--base" to "--set-baseline".
  94
  95 2007-11-02  Darin Adler  <darin@apple.com>
  96
  97         Reviewed by Maciej.
  98
  99         * sunspider: Added a "--base" option that records the current test as a baseline.
 100         Moved the code to find the newest .mshark file into a function (since I needed to use
 101         it in the other script and I prefer to copy/paste a function).
 102
 103         * sunspider-compare-results: Added logic to compare the baseline with the last
 104         run when you don't pass any parameters.
 105
 106 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 107
 108         Reviewed by Eric.
 109
 110         - Added list of ECMAScript3 features that SunSpider does not cover currently.
 111
 112         * UNCOVERED: Added.
 113
 114 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 115
 116         Reviewed by Oliver.
 117
 118         - Give better error messages when sunspider-compare-results fails
 119
 120         * sunspider-compare-results:
 121
 122 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 123
 124         Reviewed by Darin.
 125
 126         - fix Date tests to work in command-line SpiderMonkey
 127
 128         * tests/date-format-tofte.js:
 129         * tests/date-format-xparb.js:
 130
 131 2007-10-29  Darin Adler  <darin@apple.com>
 132
 133         Reviewed by Maciej.
 134
 135         * sunspider: Added --shark20 option, to run Shark at its highest sample
 136         resolution instead of its default.
 137
 138 2007-10-28  Adam Roben  <aroben@apple.com>
 139
 140         Fix SunSpider on Windows
 141
 142         Reviewed by Eric.
 143
 144         * sunspider: Don't use colons in filenames.
 145
 146 2007-10-28  Darin Adler  <darin@apple.com>
 147
 148         Reviewed by Adam.
 149
 150         * resources/sunspider-analyze-results.js: Tweak the output format a little.
 151         Change so that when there's only one run we don't write out confidence
 152         intervals at all rather than writing "NaN" over and over again.
 153
 154         * sunspider: Use the time and date as a suffix on the results file, that way
 155         all the results are left behind in the tmp directory. This will make it easy
 156         for us to add features that compare past results in the future. Also rename
 157         the .mshark files using the same scheme.
 158
 159         * sunspider-compare-results: Relax the parsing rules so we can parse the
 160         results file exactly as it's emitted from sunspider.
 161
 162 2007-10-25  Adam Roben  <aroben@apple.com>
 163
 164         Strip carriage returns from results
 165
 166         Carriage returns were screwing up sunspider-compare-results on
 167         Windows.
 168
 169         Rubberstamped by Oliver.
 170
 171         * sunspider:
 172
 173 2007-10-24  Darin Adler  <darin@apple.com>
 174
 175         Reviewed by Eric.
 176
 177         * sunspider: Open the profile in shark after completing a run.
 178         * sunspider-compare-results: Tiny perl tweaks.
 179
 180 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 181
 182         Reviewed by Darin.
 183
 184         - Don't hardcode my path to testkjs
 185
 186         * sunspider-compare-results:
 187
 188 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 189
 190         Reviewed by Darin.
 191
 192         - Add a compare script to compare results, and improve formatting of standard results a bit.
 193
 194         * resources/sunspider-analyze-results.js:
 195         * resources/sunspider-compare-results.js: Added.
 196         * sunspider-compare-results: Added.
 197
 198 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 199
 200         Reviewed by Darin.
 201
 202         * resources/sunspider-analyze-results.js: Do the error range
 203         properly, using the t-distribution instead of 1.96 (which was
 204         based on the normal distribution).
 205
 206         * sunspider: Print results in a way that will be friendlier to the
 207         soon-to-come compare mode.
 208
 209 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
 210
 211         Reviewed by Review.
 212
 213         - add more tests (probably the final set for now)
 214
 215         Already balanced. Added date, regexp, control flow, and a few
 216         more string and object/array access tests.
 217
 218         * tests/LIST:
 219         * tests/access-fannkuch.js: Added.
 220         * tests/access-nbody.js: Added.
 221         * tests/controlflow-recursive.js: Added.
 222         * tests/date-format-tofte.js: Added.
 223         * tests/date-format-xparb.js: Added.
 224         * tests/regexp-dna.js: Added.
 225         * tests/string-validate-input.js: Added.
 226
 227 2007-10-21  Eric Seidel  <eric@webkit.org>
 228
 229         Reviewed by mjs.
 230
 231         * sunspider: make --shark only sample from testkjs
 232
 233 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 234
 235         Rubber stamped by Eric.
 236
 237         - added crypto tests (already balanced)
 238
 239         * tests/LIST:
 240         * tests/crypto-aes.js: Added.
 241         * tests/crypto-md5.js: Added.
 242         * tests/crypto-sha1.js: Added.
 243
 244 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 245
 246         Reviewed by Adam.
 247
 248         - rebalance new tests
 249
 250         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
 251
 252         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 253
 254         This balances them with the existing tests.
 255
 256         * tests/access-binary-trees.js:
 257         * tests/access-nsieve.js:
 258         * tests/bitops-nsieve-bits.js:
 259         * tests/math-partial-sums.js:
 260         * tests/math-spectral-norm.js:
 261         * tests/string-fasta.js:
 262
 263 2007-10-20  Eric Seidel  <eric@webkit.org>
 264
 265         Reviewed by darin.
 266
 267         Add improved argument handling to sunspider, including
 268         --runs=<number>, --tests=<pattern, --shark, and --help
 269         Also re-factor code into subroutines
 270
 271         * sunspider: improved argument handling
 272
 273 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 274
 275         Reviewed by Mark.
 276
 277         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
 278
 279         * TODO:
 280         * tests/LIST:
 281         * tests/access-binary-trees.js: Added.
 282         * tests/access-nsieve.js: Added.
 283         * tests/bitops-nsieve-bits.js: Added.
 284         * tests/math-partial-sums.js: Added.
 285         * tests/math-spectral-norm.js: Added.
 286         * tests/string-fasta.js: Added.
 287
 288 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 289
 290         Reviewed by Darin.
 291
 292         * TODO: Updated to note areas that are now well-covered.
 293
 294         - Change the tests and the driver to avoid leaving large object
 295         graphs hanging around, since that throws off the subsequent tests.
 296
 297         * resources/sunspider-standalone-driver.js:
 298         * tests/3d-cube.js:
 299         * tests/3d-morph.js:
 300         * tests/3d-raytrace.js:
 301         * tests/bitops-bitwise-and.js:
 302         * tests/string-base64.js:
 303         * tests/string-tagcloud.js:
 304
 305 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 306
 307         Reviewed by Nikolas.
 308
 309         - Rebalance test complexity.
 310
 311         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
 312
 313         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 314
 315         However, for some tests, IE7 was a huge outlier compared to other
 316         major browsers. For those tests, I used Firefox 2.0.0.8 on the
 317         same OS and hardware as a normalization basis.
 318
 319         * tests/3d-morph.js:
 320         * tests/3d-raytrace.js:
 321         * tests/bitops-3bit-bits-in-byte.js:
 322         * tests/bitops-bits-in-byte.js:
 323         * tests/bitops-bitwise-and.js:
 324         * tests/math-cordic.js:
 325         * tests/string-base64.js:
 326         * tests/string-tagcloud.js:
 327         * tests/string-unpack-code.js:
 328
 329 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 330
 331         Reviewed by Oliver.
 332
 333         - Fix the browser-hosted driver to be compatible with Safari 2
 334
 335         * hosted/sunspider-driver.html:
 336
 337 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 338
 339         Rubber stamped by Hyatt.
 340
 341         - Add BSD license notice to cordic test, the original author said to do so.
 342
 343         * tests/math-cordic.js:
 344
 345 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 346
 347         Reviewed by Eric.
 348
 349         - Support Windows and Windows browsers.
 350
 351         * hosted/sunspider-results.html: Display results correctly in IE.
 352         * sunspider: tweaks to work on Windows, and with spaces in the path.
 353
 354 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 355
 356         Rubber stamped by Oliver.
 357
 358         * tests/string-tagcloud.js: Remove stray print()
 359
 360 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
 361
 362         Reviewed by Oliver.
 363
 364         Add some string test cases.
 365
 366         * tests/string-base64.js: Added. Does base64 encoding and decoding.
 367         * tests/string-tagcloud.js: Added. Parses a JSON string and
 368         generates tagcloud HTML markup based on the data.
 369         * tests/string-unpack-code.js: Added. Unpack compressed versions of
 370         four of the most popular JavaScript libraries.
 371         * tests/LIST: Added the new tests.
 372
 373 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
 374
 375         Reviewed by Anders.
 376
 377         * TODO: Updated with info on how to do the stats properly.
 378
 379         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
 380         avoid overly slowing down the benchmark.
 381
 382         Added some 3D-related test cases.
 383
 384         * tests/3d-cube.js: Added.
 385         * tests/3d-morph.js: Added.
 386         * tests/3d-raytrace.js: Added.
 387         * tests/LIST:
 388
 389 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
 390
 391         Reviewed by Darin.
 392
 393         - New JavaScript benchmark
 394         http://bugs.webkit.org/show_bug.cgi?id=15515
 395
 396         A start on a new JavaScript benchmark, with standalone and
 397         browser-hosted drivers.
 398
 399         The test content is still incomplete and the drivers could use
 400         some more features but there is enough here to make a good start.
 401
 402         The drivers compute a 95% confidence interval on the mean for the
 403         whole test, each category, and each individual test to make it
 404         easier to tell whether differences are statistically
 405         significant. The confidence interval can be narrowed by running
 406         the test more times.
 407
 408         * TODO: Added. Remaining things that need to be done.
 409         * sunspider: Added. Perl script that acts as the standalone test driver.
 410         * resources/sunspider-standalone-driver.js: Added. JavaScript part
 411         of standalone test driver.
 412         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
 413         analysis code.
 414         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
 415         * make-hosted: Added. Script to generate browser-hosted tests.
 416         * hosted/sunspider.html: Added. Start page for browser-hosted test.
 417         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
 418         * hosted/sunspider-results.html: Added. Results page for browser-hosted
 419         tests.
 420         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
 421         tests.
 422         * tests/LIST: Added. List of tests to use.
 423         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
 424         * tests/bitops-bits-in-byte.js: Added. Ditto.
 425         * tests/bitops-bitwise-and.js: Added. Ditto.
 426         * tests/math-cordic.js: Added. Ditto.
 427