SunSpider/ChangeLog

   1 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
   2
   3         Reviewed by Eric.
   4
   5         - give an extra digit of precision for small differences.
   6
   7         * resources/sunspider-compare-results.js:
   8
   9 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
  10
  11         Reviewed by Eric.
  12
  13         - work around IE layout bugs
  14
  15         * hosted/sunspider.css:
  16
  17 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
  18
  19         Reviewed by Mark.
  20
  21         - fix compare feature
  22
  23         * hosted/sunspider-results.html:
  24
  25 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
  26
  27         Reviewed by Mark.
  28
  29         - Change the format of the output URL to be shorter, to meet IE constraints
  30
  31         * hosted/sunspider-driver.html:
  32         * hosted/sunspider-results.html:
  33
  34 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
  35
  36         Reviewed by Oliver.
  37
  38         - Add some explanatory test and a dumb logo to SunSpider web hosted version
  39
  40         * hosted/sunspider-driver.html:
  41         * hosted/sunspider-results.html:
  42         * hosted/sunspider.css:
  43         * hosted/sunspider.html:
  44
  45 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
  46
  47         Reviewed by Oliver.
  48
  49         - Make the web-hosted version of SunSpider look a little bit less drab.
  50
  51         * hosted/sunspider-driver.html:
  52         * hosted/sunspider-results.html:
  53         * hosted/sunspider.css: Added.
  54         * hosted/sunspider.html:
  55         * resources/TEMPLATE.html:
  56
  57 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
  58
  59         Reviewed by Sam.
  60
  61         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
  62
  63         A sample of what new-style results would look like:
  64
  65         TEST                   COMPARISON            FROM                 TO             DETAILS
  66         ...
  67         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
  68         ...
  69             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
  70
  71         * resources/sunspider-compare-results.js:
  72
  73 2007-11-14  Eric Seidel  <eric@webkit.org>
  74
  75         Reviewed by Sam.
  76
  77         * sunspider: add --shark-cache for L2 Cache Miss profiling
  78
  79 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
  80
  81         Reviewed by Darin.
  82
  83         http://bugs.webkit.org/show_bug.cgi?id=15925
  84         SunSpider should check for Shark being installed
  85
  86         * sunspider: Explicitly test that shark command line tool is installed.
  87
  88 2007-11-04  Darin Adler  <darin@apple.com>
  89
  90         * hosted/sunspider.html: Tweaked formatting a bit.
  91
  92 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
  93
  94         Reviewed by Darin.
  95
  96         - Add compare mode and a description of the benchmark to the hosted version.
  97
  98         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
  99         * hosted/sunspider.html: Added intro text.
 100         * make-hosted: Copy new file.
 101         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
 102         a function.
 103         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
 104         expects.
 105         * sunspider-compare-results: Use sunspider-standalone-compare.js.
 106
 107 2007-11-02  Darin Adler  <darin@apple.com>
 108
 109         * sunspider: Changed "--base" to "--set-baseline".
 110
 111 2007-11-02  Darin Adler  <darin@apple.com>
 112
 113         Reviewed by Maciej.
 114
 115         * sunspider: Added a "--base" option that records the current test as a baseline.
 116         Moved the code to find the newest .mshark file into a function (since I needed to use
 117         it in the other script and I prefer to copy/paste a function).
 118
 119         * sunspider-compare-results: Added logic to compare the baseline with the last
 120         run when you don't pass any parameters.
 121
 122 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 123
 124         Reviewed by Eric.
 125
 126         - Added list of ECMAScript3 features that SunSpider does not cover currently.
 127
 128         * UNCOVERED: Added.
 129
 130 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 131
 132         Reviewed by Oliver.
 133
 134         - Give better error messages when sunspider-compare-results fails
 135
 136         * sunspider-compare-results:
 137
 138 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 139
 140         Reviewed by Darin.
 141
 142         - fix Date tests to work in command-line SpiderMonkey
 143
 144         * tests/date-format-tofte.js:
 145         * tests/date-format-xparb.js:
 146
 147 2007-10-29  Darin Adler  <darin@apple.com>
 148
 149         Reviewed by Maciej.
 150
 151         * sunspider: Added --shark20 option, to run Shark at its highest sample
 152         resolution instead of its default.
 153
 154 2007-10-28  Adam Roben  <aroben@apple.com>
 155
 156         Fix SunSpider on Windows
 157
 158         Reviewed by Eric.
 159
 160         * sunspider: Don't use colons in filenames.
 161
 162 2007-10-28  Darin Adler  <darin@apple.com>
 163
 164         Reviewed by Adam.
 165
 166         * resources/sunspider-analyze-results.js: Tweak the output format a little.
 167         Change so that when there's only one run we don't write out confidence
 168         intervals at all rather than writing "NaN" over and over again.
 169
 170         * sunspider: Use the time and date as a suffix on the results file, that way
 171         all the results are left behind in the tmp directory. This will make it easy
 172         for us to add features that compare past results in the future. Also rename
 173         the .mshark files using the same scheme.
 174
 175         * sunspider-compare-results: Relax the parsing rules so we can parse the
 176         results file exactly as it's emitted from sunspider.
 177
 178 2007-10-25  Adam Roben  <aroben@apple.com>
 179
 180         Strip carriage returns from results
 181
 182         Carriage returns were screwing up sunspider-compare-results on
 183         Windows.
 184
 185         Rubberstamped by Oliver.
 186
 187         * sunspider:
 188
 189 2007-10-24  Darin Adler  <darin@apple.com>
 190
 191         Reviewed by Eric.
 192
 193         * sunspider: Open the profile in shark after completing a run.
 194         * sunspider-compare-results: Tiny perl tweaks.
 195
 196 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 197
 198         Reviewed by Darin.
 199
 200         - Don't hardcode my path to testkjs
 201
 202         * sunspider-compare-results:
 203
 204 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 205
 206         Reviewed by Darin.
 207
 208         - Add a compare script to compare results, and improve formatting of standard results a bit.
 209
 210         * resources/sunspider-analyze-results.js:
 211         * resources/sunspider-compare-results.js: Added.
 212         * sunspider-compare-results: Added.
 213
 214 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 215
 216         Reviewed by Darin.
 217
 218         * resources/sunspider-analyze-results.js: Do the error range
 219         properly, using the t-distribution instead of 1.96 (which was
 220         based on the normal distribution).
 221
 222         * sunspider: Print results in a way that will be friendlier to the
 223         soon-to-come compare mode.
 224
 225 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
 226
 227         Reviewed by Review.
 228
 229         - add more tests (probably the final set for now)
 230
 231         Already balanced. Added date, regexp, control flow, and a few
 232         more string and object/array access tests.
 233
 234         * tests/LIST:
 235         * tests/access-fannkuch.js: Added.
 236         * tests/access-nbody.js: Added.
 237         * tests/controlflow-recursive.js: Added.
 238         * tests/date-format-tofte.js: Added.
 239         * tests/date-format-xparb.js: Added.
 240         * tests/regexp-dna.js: Added.
 241         * tests/string-validate-input.js: Added.
 242
 243 2007-10-21  Eric Seidel  <eric@webkit.org>
 244
 245         Reviewed by mjs.
 246
 247         * sunspider: make --shark only sample from testkjs
 248
 249 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 250
 251         Rubber stamped by Eric.
 252
 253         - added crypto tests (already balanced)
 254
 255         * tests/LIST:
 256         * tests/crypto-aes.js: Added.
 257         * tests/crypto-md5.js: Added.
 258         * tests/crypto-sha1.js: Added.
 259
 260 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 261
 262         Reviewed by Adam.
 263
 264         - rebalance new tests
 265
 266         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
 267
 268         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 269
 270         This balances them with the existing tests.
 271
 272         * tests/access-binary-trees.js:
 273         * tests/access-nsieve.js:
 274         * tests/bitops-nsieve-bits.js:
 275         * tests/math-partial-sums.js:
 276         * tests/math-spectral-norm.js:
 277         * tests/string-fasta.js:
 278
 279 2007-10-20  Eric Seidel  <eric@webkit.org>
 280
 281         Reviewed by darin.
 282
 283         Add improved argument handling to sunspider, including
 284         --runs=<number>, --tests=<pattern, --shark, and --help
 285         Also re-factor code into subroutines
 286
 287         * sunspider: improved argument handling
 288
 289 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 290
 291         Reviewed by Mark.
 292
 293         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
 294
 295         * TODO:
 296         * tests/LIST:
 297         * tests/access-binary-trees.js: Added.
 298         * tests/access-nsieve.js: Added.
 299         * tests/bitops-nsieve-bits.js: Added.
 300         * tests/math-partial-sums.js: Added.
 301         * tests/math-spectral-norm.js: Added.
 302         * tests/string-fasta.js: Added.
 303
 304 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 305
 306         Reviewed by Darin.
 307
 308         * TODO: Updated to note areas that are now well-covered.
 309
 310         - Change the tests and the driver to avoid leaving large object
 311         graphs hanging around, since that throws off the subsequent tests.
 312
 313         * resources/sunspider-standalone-driver.js:
 314         * tests/3d-cube.js:
 315         * tests/3d-morph.js:
 316         * tests/3d-raytrace.js:
 317         * tests/bitops-bitwise-and.js:
 318         * tests/string-base64.js:
 319         * tests/string-tagcloud.js:
 320
 321 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 322
 323         Reviewed by Nikolas.
 324
 325         - Rebalance test complexity.
 326
 327         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
 328
 329         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 330
 331         However, for some tests, IE7 was a huge outlier compared to other
 332         major browsers. For those tests, I used Firefox 2.0.0.8 on the
 333         same OS and hardware as a normalization basis.
 334
 335         * tests/3d-morph.js:
 336         * tests/3d-raytrace.js:
 337         * tests/bitops-3bit-bits-in-byte.js:
 338         * tests/bitops-bits-in-byte.js:
 339         * tests/bitops-bitwise-and.js:
 340         * tests/math-cordic.js:
 341         * tests/string-base64.js:
 342         * tests/string-tagcloud.js:
 343         * tests/string-unpack-code.js:
 344
 345 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 346
 347         Reviewed by Oliver.
 348
 349         - Fix the browser-hosted driver to be compatible with Safari 2
 350
 351         * hosted/sunspider-driver.html:
 352
 353 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 354
 355         Rubber stamped by Hyatt.
 356
 357         - Add BSD license notice to cordic test, the original author said to do so.
 358
 359         * tests/math-cordic.js:
 360
 361 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 362
 363         Reviewed by Eric.
 364
 365         - Support Windows and Windows browsers.
 366
 367         * hosted/sunspider-results.html: Display results correctly in IE.
 368         * sunspider: tweaks to work on Windows, and with spaces in the path.
 369
 370 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 371
 372         Rubber stamped by Oliver.
 373
 374         * tests/string-tagcloud.js: Remove stray print()
 375
 376 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
 377
 378         Reviewed by Oliver.
 379
 380         Add some string test cases.
 381
 382         * tests/string-base64.js: Added. Does base64 encoding and decoding.
 383         * tests/string-tagcloud.js: Added. Parses a JSON string and
 384         generates tagcloud HTML markup based on the data.
 385         * tests/string-unpack-code.js: Added. Unpack compressed versions of
 386         four of the most popular JavaScript libraries.
 387         * tests/LIST: Added the new tests.
 388
 389 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
 390
 391         Reviewed by Anders.
 392
 393         * TODO: Updated with info on how to do the stats properly.
 394
 395         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
 396         avoid overly slowing down the benchmark.
 397
 398         Added some 3D-related test cases.
 399
 400         * tests/3d-cube.js: Added.
 401         * tests/3d-morph.js: Added.
 402         * tests/3d-raytrace.js: Added.
 403         * tests/LIST:
 404
 405 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
 406
 407         Reviewed by Darin.
 408
 409         - New JavaScript benchmark
 410         http://bugs.webkit.org/show_bug.cgi?id=15515
 411
 412         A start on a new JavaScript benchmark, with standalone and
 413         browser-hosted drivers.
 414
 415         The test content is still incomplete and the drivers could use
 416         some more features but there is enough here to make a good start.
 417
 418         The drivers compute a 95% confidence interval on the mean for the
 419         whole test, each category, and each individual test to make it
 420         easier to tell whether differences are statistically
 421         significant. The confidence interval can be narrowed by running
 422         the test more times.
 423
 424         * TODO: Added. Remaining things that need to be done.
 425         * sunspider: Added. Perl script that acts as the standalone test driver.
 426         * resources/sunspider-standalone-driver.js: Added. JavaScript part
 427         of standalone test driver.
 428         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
 429         analysis code.
 430         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
 431         * make-hosted: Added. Script to generate browser-hosted tests.
 432         * hosted/sunspider.html: Added. Start page for browser-hosted test.
 433         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
 434         * hosted/sunspider-results.html: Added. Results page for browser-hosted
 435         tests.
 436         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
 437         tests.
 438         * tests/LIST: Added. List of tests to use.
 439         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
 440         * tests/bitops-bits-in-byte.js: Added. Ditto.
 441         * tests/bitops-bitwise-and.js: Added. Ditto.
 442         * tests/math-cordic.js: Added. Ditto.
 443