SunSpider/ChangeLog

   1 2007-12-29  Alp Toker  <alp@atoker.com>
   2
   3         Fix typo.
   4
   5         * hosted/sunspider.html:
   6
   7 2007-12-29  Alp Toker  <alp@atoker.com>
   8
   9         Reviewed by Maciej.
  10
  11         Provide the current URL in the SunSpider results page. Useful for
  12         copying benchmark results out of browsers without a location bar or
  13         where the location bar can't handle long URLs.
  14
  15         * hosted/sunspider-results.html:
  16
  17 2007-12-20  Eric Seidel  <eric@webkit.org>
  18
  19         Reviewed by mjs.
  20
  21         * sunspider: increase default number of runs for more accurate numbers
  22
  23 2007-12-18  Mark Rowe  <mrowe@apple.com>
  24
  25         Fix typo.
  26
  27         * hosted/sunspider.html:
  28
  29 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
  30
  31         Reviewed by Eric.
  32
  33         - give an extra digit of precision for small differences.
  34
  35         * resources/sunspider-compare-results.js:
  36
  37 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
  38
  39         Reviewed by Eric.
  40
  41         - work around IE layout bugs
  42
  43         * hosted/sunspider.css:
  44
  45 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
  46
  47         Reviewed by Mark.
  48
  49         - fix compare feature
  50
  51         * hosted/sunspider-results.html:
  52
  53 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
  54
  55         Reviewed by Mark.
  56
  57         - Change the format of the output URL to be shorter, to meet IE constraints
  58
  59         * hosted/sunspider-driver.html:
  60         * hosted/sunspider-results.html:
  61
  62 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
  63
  64         Reviewed by Oliver.
  65
  66         - Add some explanatory test and a dumb logo to SunSpider web hosted version
  67
  68         * hosted/sunspider-driver.html:
  69         * hosted/sunspider-results.html:
  70         * hosted/sunspider.css:
  71         * hosted/sunspider.html:
  72
  73 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
  74
  75         Reviewed by Oliver.
  76
  77         - Make the web-hosted version of SunSpider look a little bit less drab.
  78
  79         * hosted/sunspider-driver.html:
  80         * hosted/sunspider-results.html:
  81         * hosted/sunspider.css: Added.
  82         * hosted/sunspider.html:
  83         * resources/TEMPLATE.html:
  84
  85 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
  86
  87         Reviewed by Sam.
  88
  89         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
  90
  91         A sample of what new-style results would look like:
  92
  93         TEST                   COMPARISON            FROM                 TO             DETAILS
  94         ...
  95         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
  96         ...
  97             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
  98
  99         * resources/sunspider-compare-results.js:
 100
 101 2007-11-14  Eric Seidel  <eric@webkit.org>
 102
 103         Reviewed by Sam.
 104
 105         * sunspider: add --shark-cache for L2 Cache Miss profiling
 106
 107 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
 108
 109         Reviewed by Darin.
 110
 111         http://bugs.webkit.org/show_bug.cgi?id=15925
 112         SunSpider should check for Shark being installed
 113
 114         * sunspider: Explicitly test that shark command line tool is installed.
 115
 116 2007-11-04  Darin Adler  <darin@apple.com>
 117
 118         * hosted/sunspider.html: Tweaked formatting a bit.
 119
 120 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
 121
 122         Reviewed by Darin.
 123
 124         - Add compare mode and a description of the benchmark to the hosted version.
 125
 126         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
 127         * hosted/sunspider.html: Added intro text.
 128         * make-hosted: Copy new file.
 129         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
 130         a function.
 131         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
 132         expects.
 133         * sunspider-compare-results: Use sunspider-standalone-compare.js.
 134
 135 2007-11-02  Darin Adler  <darin@apple.com>
 136
 137         * sunspider: Changed "--base" to "--set-baseline".
 138
 139 2007-11-02  Darin Adler  <darin@apple.com>
 140
 141         Reviewed by Maciej.
 142
 143         * sunspider: Added a "--base" option that records the current test as a baseline.
 144         Moved the code to find the newest .mshark file into a function (since I needed to use
 145         it in the other script and I prefer to copy/paste a function).
 146
 147         * sunspider-compare-results: Added logic to compare the baseline with the last
 148         run when you don't pass any parameters.
 149
 150 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 151
 152         Reviewed by Eric.
 153
 154         - Added list of ECMAScript3 features that SunSpider does not cover currently.
 155
 156         * UNCOVERED: Added.
 157
 158 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 159
 160         Reviewed by Oliver.
 161
 162         - Give better error messages when sunspider-compare-results fails
 163
 164         * sunspider-compare-results:
 165
 166 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 167
 168         Reviewed by Darin.
 169
 170         - fix Date tests to work in command-line SpiderMonkey
 171
 172         * tests/date-format-tofte.js:
 173         * tests/date-format-xparb.js:
 174
 175 2007-10-29  Darin Adler  <darin@apple.com>
 176
 177         Reviewed by Maciej.
 178
 179         * sunspider: Added --shark20 option, to run Shark at its highest sample
 180         resolution instead of its default.
 181
 182 2007-10-28  Adam Roben  <aroben@apple.com>
 183
 184         Fix SunSpider on Windows
 185
 186         Reviewed by Eric.
 187
 188         * sunspider: Don't use colons in filenames.
 189
 190 2007-10-28  Darin Adler  <darin@apple.com>
 191
 192         Reviewed by Adam.
 193
 194         * resources/sunspider-analyze-results.js: Tweak the output format a little.
 195         Change so that when there's only one run we don't write out confidence
 196         intervals at all rather than writing "NaN" over and over again.
 197
 198         * sunspider: Use the time and date as a suffix on the results file, that way
 199         all the results are left behind in the tmp directory. This will make it easy
 200         for us to add features that compare past results in the future. Also rename
 201         the .mshark files using the same scheme.
 202
 203         * sunspider-compare-results: Relax the parsing rules so we can parse the
 204         results file exactly as it's emitted from sunspider.
 205
 206 2007-10-25  Adam Roben  <aroben@apple.com>
 207
 208         Strip carriage returns from results
 209
 210         Carriage returns were screwing up sunspider-compare-results on
 211         Windows.
 212
 213         Rubberstamped by Oliver.
 214
 215         * sunspider:
 216
 217 2007-10-24  Darin Adler  <darin@apple.com>
 218
 219         Reviewed by Eric.
 220
 221         * sunspider: Open the profile in shark after completing a run.
 222         * sunspider-compare-results: Tiny perl tweaks.
 223
 224 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 225
 226         Reviewed by Darin.
 227
 228         - Don't hardcode my path to testkjs
 229
 230         * sunspider-compare-results:
 231
 232 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 233
 234         Reviewed by Darin.
 235
 236         - Add a compare script to compare results, and improve formatting of standard results a bit.
 237
 238         * resources/sunspider-analyze-results.js:
 239         * resources/sunspider-compare-results.js: Added.
 240         * sunspider-compare-results: Added.
 241
 242 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 243
 244         Reviewed by Darin.
 245
 246         * resources/sunspider-analyze-results.js: Do the error range
 247         properly, using the t-distribution instead of 1.96 (which was
 248         based on the normal distribution).
 249
 250         * sunspider: Print results in a way that will be friendlier to the
 251         soon-to-come compare mode.
 252
 253 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
 254
 255         Reviewed by Review.
 256
 257         - add more tests (probably the final set for now)
 258
 259         Already balanced. Added date, regexp, control flow, and a few
 260         more string and object/array access tests.
 261
 262         * tests/LIST:
 263         * tests/access-fannkuch.js: Added.
 264         * tests/access-nbody.js: Added.
 265         * tests/controlflow-recursive.js: Added.
 266         * tests/date-format-tofte.js: Added.
 267         * tests/date-format-xparb.js: Added.
 268         * tests/regexp-dna.js: Added.
 269         * tests/string-validate-input.js: Added.
 270
 271 2007-10-21  Eric Seidel  <eric@webkit.org>
 272
 273         Reviewed by mjs.
 274
 275         * sunspider: make --shark only sample from testkjs
 276
 277 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 278
 279         Rubber stamped by Eric.
 280
 281         - added crypto tests (already balanced)
 282
 283         * tests/LIST:
 284         * tests/crypto-aes.js: Added.
 285         * tests/crypto-md5.js: Added.
 286         * tests/crypto-sha1.js: Added.
 287
 288 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 289
 290         Reviewed by Adam.
 291
 292         - rebalance new tests
 293
 294         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
 295
 296         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 297
 298         This balances them with the existing tests.
 299
 300         * tests/access-binary-trees.js:
 301         * tests/access-nsieve.js:
 302         * tests/bitops-nsieve-bits.js:
 303         * tests/math-partial-sums.js:
 304         * tests/math-spectral-norm.js:
 305         * tests/string-fasta.js:
 306
 307 2007-10-20  Eric Seidel  <eric@webkit.org>
 308
 309         Reviewed by darin.
 310
 311         Add improved argument handling to sunspider, including
 312         --runs=<number>, --tests=<pattern, --shark, and --help
 313         Also re-factor code into subroutines
 314
 315         * sunspider: improved argument handling
 316
 317 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 318
 319         Reviewed by Mark.
 320
 321         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
 322
 323         * TODO:
 324         * tests/LIST:
 325         * tests/access-binary-trees.js: Added.
 326         * tests/access-nsieve.js: Added.
 327         * tests/bitops-nsieve-bits.js: Added.
 328         * tests/math-partial-sums.js: Added.
 329         * tests/math-spectral-norm.js: Added.
 330         * tests/string-fasta.js: Added.
 331
 332 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 333
 334         Reviewed by Darin.
 335
 336         * TODO: Updated to note areas that are now well-covered.
 337
 338         - Change the tests and the driver to avoid leaving large object
 339         graphs hanging around, since that throws off the subsequent tests.
 340
 341         * resources/sunspider-standalone-driver.js:
 342         * tests/3d-cube.js:
 343         * tests/3d-morph.js:
 344         * tests/3d-raytrace.js:
 345         * tests/bitops-bitwise-and.js:
 346         * tests/string-base64.js:
 347         * tests/string-tagcloud.js:
 348
 349 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 350
 351         Reviewed by Nikolas.
 352
 353         - Rebalance test complexity.
 354
 355         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
 356
 357         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 358
 359         However, for some tests, IE7 was a huge outlier compared to other
 360         major browsers. For those tests, I used Firefox 2.0.0.8 on the
 361         same OS and hardware as a normalization basis.
 362
 363         * tests/3d-morph.js:
 364         * tests/3d-raytrace.js:
 365         * tests/bitops-3bit-bits-in-byte.js:
 366         * tests/bitops-bits-in-byte.js:
 367         * tests/bitops-bitwise-and.js:
 368         * tests/math-cordic.js:
 369         * tests/string-base64.js:
 370         * tests/string-tagcloud.js:
 371         * tests/string-unpack-code.js:
 372
 373 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 374
 375         Reviewed by Oliver.
 376
 377         - Fix the browser-hosted driver to be compatible with Safari 2
 378
 379         * hosted/sunspider-driver.html:
 380
 381 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 382
 383         Rubber stamped by Hyatt.
 384
 385         - Add BSD license notice to cordic test, the original author said to do so.
 386
 387         * tests/math-cordic.js:
 388
 389 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 390
 391         Reviewed by Eric.
 392
 393         - Support Windows and Windows browsers.
 394
 395         * hosted/sunspider-results.html: Display results correctly in IE.
 396         * sunspider: tweaks to work on Windows, and with spaces in the path.
 397
 398 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 399
 400         Rubber stamped by Oliver.
 401
 402         * tests/string-tagcloud.js: Remove stray print()
 403
 404 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
 405
 406         Reviewed by Oliver.
 407
 408         Add some string test cases.
 409
 410         * tests/string-base64.js: Added. Does base64 encoding and decoding.
 411         * tests/string-tagcloud.js: Added. Parses a JSON string and
 412         generates tagcloud HTML markup based on the data.
 413         * tests/string-unpack-code.js: Added. Unpack compressed versions of
 414         four of the most popular JavaScript libraries.
 415         * tests/LIST: Added the new tests.
 416
 417 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
 418
 419         Reviewed by Anders.
 420
 421         * TODO: Updated with info on how to do the stats properly.
 422
 423         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
 424         avoid overly slowing down the benchmark.
 425
 426         Added some 3D-related test cases.
 427
 428         * tests/3d-cube.js: Added.
 429         * tests/3d-morph.js: Added.
 430         * tests/3d-raytrace.js: Added.
 431         * tests/LIST:
 432
 433 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
 434
 435         Reviewed by Darin.
 436
 437         - New JavaScript benchmark
 438         http://bugs.webkit.org/show_bug.cgi?id=15515
 439
 440         A start on a new JavaScript benchmark, with standalone and
 441         browser-hosted drivers.
 442
 443         The test content is still incomplete and the drivers could use
 444         some more features but there is enough here to make a good start.
 445
 446         The drivers compute a 95% confidence interval on the mean for the
 447         whole test, each category, and each individual test to make it
 448         easier to tell whether differences are statistically
 449         significant. The confidence interval can be narrowed by running
 450         the test more times.
 451
 452         * TODO: Added. Remaining things that need to be done.
 453         * sunspider: Added. Perl script that acts as the standalone test driver.
 454         * resources/sunspider-standalone-driver.js: Added. JavaScript part
 455         of standalone test driver.
 456         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
 457         analysis code.
 458         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
 459         * make-hosted: Added. Script to generate browser-hosted tests.
 460         * hosted/sunspider.html: Added. Start page for browser-hosted test.
 461         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
 462         * hosted/sunspider-results.html: Added. Results page for browser-hosted
 463         tests.
 464         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
 465         tests.
 466         * tests/LIST: Added. List of tests to use.
 467         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
 468         * tests/bitops-bits-in-byte.js: Added. Ditto.
 469         * tests/bitops-bitwise-and.js: Added. Ditto.
 470         * tests/math-cordic.js: Added. Ditto.
 471