SunSpider/ChangeLog

   1 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
   2
   3         Reviewed by Oliver.
   4
   5         - Make the web-hosted version of SunSpider look a little bit less drab.
   6
   7         * hosted/sunspider-driver.html:
   8         * hosted/sunspider-results.html:
   9         * hosted/sunspider.css: Added.
  10         * hosted/sunspider.html:
  11         * resources/TEMPLATE.html:
  12
  13 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
  14
  15         Reviewed by Sam.
  16
  17         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
  18
  19         A sample of what new-style results would look like:
  20
  21         TEST                   COMPARISON            FROM                 TO             DETAILS
  22         ...
  23         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
  24         ...
  25             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
  26
  27         * resources/sunspider-compare-results.js:
  28
  29 2007-11-14  Eric Seidel  <eric@webkit.org>
  30
  31         Reviewed by Sam.
  32
  33         * sunspider: add --shark-cache for L2 Cache Miss profiling
  34
  35 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
  36
  37         Reviewed by Darin.
  38
  39         http://bugs.webkit.org/show_bug.cgi?id=15925
  40         SunSpider should check for Shark being installed
  41
  42         * sunspider: Explicitly test that shark command line tool is installed.
  43
  44 2007-11-04  Darin Adler  <darin@apple.com>
  45
  46         * hosted/sunspider.html: Tweaked formatting a bit.
  47
  48 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
  49
  50         Reviewed by Darin.
  51
  52         - Add compare mode and a description of the benchmark to the hosted version.
  53
  54         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
  55         * hosted/sunspider.html: Added intro text.
  56         * make-hosted: Copy new file.
  57         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
  58         a function.
  59         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
  60         expects.
  61         * sunspider-compare-results: Use sunspider-standalone-compare.js.
  62
  63 2007-11-02  Darin Adler  <darin@apple.com>
  64
  65         * sunspider: Changed "--base" to "--set-baseline".
  66
  67 2007-11-02  Darin Adler  <darin@apple.com>
  68
  69         Reviewed by Maciej.
  70
  71         * sunspider: Added a "--base" option that records the current test as a baseline.
  72         Moved the code to find the newest .mshark file into a function (since I needed to use
  73         it in the other script and I prefer to copy/paste a function).
  74
  75         * sunspider-compare-results: Added logic to compare the baseline with the last
  76         run when you don't pass any parameters.
  77
  78 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
  79
  80         Reviewed by Eric.
  81
  82         - Added list of ECMAScript3 features that SunSpider does not cover currently.
  83
  84         * UNCOVERED: Added.
  85
  86 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
  87
  88         Reviewed by Oliver.
  89
  90         - Give better error messages when sunspider-compare-results fails
  91
  92         * sunspider-compare-results:
  93
  94 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
  95
  96         Reviewed by Darin.
  97
  98         - fix Date tests to work in command-line SpiderMonkey
  99
 100         * tests/date-format-tofte.js:
 101         * tests/date-format-xparb.js:
 102
 103 2007-10-29  Darin Adler  <darin@apple.com>
 104
 105         Reviewed by Maciej.
 106
 107         * sunspider: Added --shark20 option, to run Shark at its highest sample
 108         resolution instead of its default.
 109
 110 2007-10-28  Adam Roben  <aroben@apple.com>
 111
 112         Fix SunSpider on Windows
 113
 114         Reviewed by Eric.
 115
 116         * sunspider: Don't use colons in filenames.
 117
 118 2007-10-28  Darin Adler  <darin@apple.com>
 119
 120         Reviewed by Adam.
 121
 122         * resources/sunspider-analyze-results.js: Tweak the output format a little.
 123         Change so that when there's only one run we don't write out confidence
 124         intervals at all rather than writing "NaN" over and over again.
 125
 126         * sunspider: Use the time and date as a suffix on the results file, that way
 127         all the results are left behind in the tmp directory. This will make it easy
 128         for us to add features that compare past results in the future. Also rename
 129         the .mshark files using the same scheme.
 130
 131         * sunspider-compare-results: Relax the parsing rules so we can parse the
 132         results file exactly as it's emitted from sunspider.
 133
 134 2007-10-25  Adam Roben  <aroben@apple.com>
 135
 136         Strip carriage returns from results
 137
 138         Carriage returns were screwing up sunspider-compare-results on
 139         Windows.
 140
 141         Rubberstamped by Oliver.
 142
 143         * sunspider:
 144
 145 2007-10-24  Darin Adler  <darin@apple.com>
 146
 147         Reviewed by Eric.
 148
 149         * sunspider: Open the profile in shark after completing a run.
 150         * sunspider-compare-results: Tiny perl tweaks.
 151
 152 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 153
 154         Reviewed by Darin.
 155
 156         - Don't hardcode my path to testkjs
 157
 158         * sunspider-compare-results:
 159
 160 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 161
 162         Reviewed by Darin.
 163
 164         - Add a compare script to compare results, and improve formatting of standard results a bit.
 165
 166         * resources/sunspider-analyze-results.js:
 167         * resources/sunspider-compare-results.js: Added.
 168         * sunspider-compare-results: Added.
 169
 170 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 171
 172         Reviewed by Darin.
 173
 174         * resources/sunspider-analyze-results.js: Do the error range
 175         properly, using the t-distribution instead of 1.96 (which was
 176         based on the normal distribution).
 177
 178         * sunspider: Print results in a way that will be friendlier to the
 179         soon-to-come compare mode.
 180
 181 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
 182
 183         Reviewed by Review.
 184
 185         - add more tests (probably the final set for now)
 186
 187         Already balanced. Added date, regexp, control flow, and a few
 188         more string and object/array access tests.
 189
 190         * tests/LIST:
 191         * tests/access-fannkuch.js: Added.
 192         * tests/access-nbody.js: Added.
 193         * tests/controlflow-recursive.js: Added.
 194         * tests/date-format-tofte.js: Added.
 195         * tests/date-format-xparb.js: Added.
 196         * tests/regexp-dna.js: Added.
 197         * tests/string-validate-input.js: Added.
 198
 199 2007-10-21  Eric Seidel  <eric@webkit.org>
 200
 201         Reviewed by mjs.
 202
 203         * sunspider: make --shark only sample from testkjs
 204
 205 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 206
 207         Rubber stamped by Eric.
 208
 209         - added crypto tests (already balanced)
 210
 211         * tests/LIST:
 212         * tests/crypto-aes.js: Added.
 213         * tests/crypto-md5.js: Added.
 214         * tests/crypto-sha1.js: Added.
 215
 216 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 217
 218         Reviewed by Adam.
 219
 220         - rebalance new tests
 221
 222         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
 223
 224         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 225
 226         This balances them with the existing tests.
 227
 228         * tests/access-binary-trees.js:
 229         * tests/access-nsieve.js:
 230         * tests/bitops-nsieve-bits.js:
 231         * tests/math-partial-sums.js:
 232         * tests/math-spectral-norm.js:
 233         * tests/string-fasta.js:
 234
 235 2007-10-20  Eric Seidel  <eric@webkit.org>
 236
 237         Reviewed by darin.
 238
 239         Add improved argument handling to sunspider, including
 240         --runs=<number>, --tests=<pattern, --shark, and --help
 241         Also re-factor code into subroutines
 242
 243         * sunspider: improved argument handling
 244
 245 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 246
 247         Reviewed by Mark.
 248
 249         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
 250
 251         * TODO:
 252         * tests/LIST:
 253         * tests/access-binary-trees.js: Added.
 254         * tests/access-nsieve.js: Added.
 255         * tests/bitops-nsieve-bits.js: Added.
 256         * tests/math-partial-sums.js: Added.
 257         * tests/math-spectral-norm.js: Added.
 258         * tests/string-fasta.js: Added.
 259
 260 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 261
 262         Reviewed by Darin.
 263
 264         * TODO: Updated to note areas that are now well-covered.
 265
 266         - Change the tests and the driver to avoid leaving large object
 267         graphs hanging around, since that throws off the subsequent tests.
 268
 269         * resources/sunspider-standalone-driver.js:
 270         * tests/3d-cube.js:
 271         * tests/3d-morph.js:
 272         * tests/3d-raytrace.js:
 273         * tests/bitops-bitwise-and.js:
 274         * tests/string-base64.js:
 275         * tests/string-tagcloud.js:
 276
 277 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 278
 279         Reviewed by Nikolas.
 280
 281         - Rebalance test complexity.
 282
 283         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
 284
 285         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 286
 287         However, for some tests, IE7 was a huge outlier compared to other
 288         major browsers. For those tests, I used Firefox 2.0.0.8 on the
 289         same OS and hardware as a normalization basis.
 290
 291         * tests/3d-morph.js:
 292         * tests/3d-raytrace.js:
 293         * tests/bitops-3bit-bits-in-byte.js:
 294         * tests/bitops-bits-in-byte.js:
 295         * tests/bitops-bitwise-and.js:
 296         * tests/math-cordic.js:
 297         * tests/string-base64.js:
 298         * tests/string-tagcloud.js:
 299         * tests/string-unpack-code.js:
 300
 301 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 302
 303         Reviewed by Oliver.
 304
 305         - Fix the browser-hosted driver to be compatible with Safari 2
 306
 307         * hosted/sunspider-driver.html:
 308
 309 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 310
 311         Rubber stamped by Hyatt.
 312
 313         - Add BSD license notice to cordic test, the original author said to do so.
 314
 315         * tests/math-cordic.js:
 316
 317 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 318
 319         Reviewed by Eric.
 320
 321         - Support Windows and Windows browsers.
 322
 323         * hosted/sunspider-results.html: Display results correctly in IE.
 324         * sunspider: tweaks to work on Windows, and with spaces in the path.
 325
 326 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 327
 328         Rubber stamped by Oliver.
 329
 330         * tests/string-tagcloud.js: Remove stray print()
 331
 332 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
 333
 334         Reviewed by Oliver.
 335
 336         Add some string test cases.
 337
 338         * tests/string-base64.js: Added. Does base64 encoding and decoding.
 339         * tests/string-tagcloud.js: Added. Parses a JSON string and
 340         generates tagcloud HTML markup based on the data.
 341         * tests/string-unpack-code.js: Added. Unpack compressed versions of
 342         four of the most popular JavaScript libraries.
 343         * tests/LIST: Added the new tests.
 344
 345 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
 346
 347         Reviewed by Anders.
 348
 349         * TODO: Updated with info on how to do the stats properly.
 350
 351         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
 352         avoid overly slowing down the benchmark.
 353
 354         Added some 3D-related test cases.
 355
 356         * tests/3d-cube.js: Added.
 357         * tests/3d-morph.js: Added.
 358         * tests/3d-raytrace.js: Added.
 359         * tests/LIST:
 360
 361 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
 362
 363         Reviewed by Darin.
 364
 365         - New JavaScript benchmark
 366         http://bugs.webkit.org/show_bug.cgi?id=15515
 367
 368         A start on a new JavaScript benchmark, with standalone and
 369         browser-hosted drivers.
 370
 371         The test content is still incomplete and the drivers could use
 372         some more features but there is enough here to make a good start.
 373
 374         The drivers compute a 95% confidence interval on the mean for the
 375         whole test, each category, and each individual test to make it
 376         easier to tell whether differences are statistically
 377         significant. The confidence interval can be narrowed by running
 378         the test more times.
 379
 380         * TODO: Added. Remaining things that need to be done.
 381         * sunspider: Added. Perl script that acts as the standalone test driver.
 382         * resources/sunspider-standalone-driver.js: Added. JavaScript part
 383         of standalone test driver.
 384         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
 385         analysis code.
 386         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
 387         * make-hosted: Added. Script to generate browser-hosted tests.
 388         * hosted/sunspider.html: Added. Start page for browser-hosted test.
 389         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
 390         * hosted/sunspider-results.html: Added. Results page for browser-hosted
 391         tests.
 392         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
 393         tests.
 394         * tests/LIST: Added. List of tests to use.
 395         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
 396         * tests/bitops-bits-in-byte.js: Added. Ditto.
 397         * tests/bitops-bitwise-and.js: Added. Ditto.
 398         * tests/math-cordic.js: Added. Ditto.
 399