SunSpider/ChangeLog

   1 2007-12-18  Mark Rowe  <mrowe@apple.com>
   2
   3         Fix typo.
   4
   5         * hosted/sunspider.html:
   6
   7 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
   8
   9         Reviewed by Eric.
  10
  11         - give an extra digit of precision for small differences.
  12
  13         * resources/sunspider-compare-results.js:
  14
  15 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
  16
  17         Reviewed by Eric.
  18
  19         - work around IE layout bugs
  20
  21         * hosted/sunspider.css:
  22
  23 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
  24
  25         Reviewed by Mark.
  26
  27         - fix compare feature
  28
  29         * hosted/sunspider-results.html:
  30
  31 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
  32
  33         Reviewed by Mark.
  34
  35         - Change the format of the output URL to be shorter, to meet IE constraints
  36
  37         * hosted/sunspider-driver.html:
  38         * hosted/sunspider-results.html:
  39
  40 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
  41
  42         Reviewed by Oliver.
  43
  44         - Add some explanatory test and a dumb logo to SunSpider web hosted version
  45
  46         * hosted/sunspider-driver.html:
  47         * hosted/sunspider-results.html:
  48         * hosted/sunspider.css:
  49         * hosted/sunspider.html:
  50
  51 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
  52
  53         Reviewed by Oliver.
  54
  55         - Make the web-hosted version of SunSpider look a little bit less drab.
  56
  57         * hosted/sunspider-driver.html:
  58         * hosted/sunspider-results.html:
  59         * hosted/sunspider.css: Added.
  60         * hosted/sunspider.html:
  61         * resources/TEMPLATE.html:
  62
  63 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
  64
  65         Reviewed by Sam.
  66
  67         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
  68
  69         A sample of what new-style results would look like:
  70
  71         TEST                   COMPARISON            FROM                 TO             DETAILS
  72         ...
  73         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
  74         ...
  75             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
  76
  77         * resources/sunspider-compare-results.js:
  78
  79 2007-11-14  Eric Seidel  <eric@webkit.org>
  80
  81         Reviewed by Sam.
  82
  83         * sunspider: add --shark-cache for L2 Cache Miss profiling
  84
  85 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
  86
  87         Reviewed by Darin.
  88
  89         http://bugs.webkit.org/show_bug.cgi?id=15925
  90         SunSpider should check for Shark being installed
  91
  92         * sunspider: Explicitly test that shark command line tool is installed.
  93
  94 2007-11-04  Darin Adler  <darin@apple.com>
  95
  96         * hosted/sunspider.html: Tweaked formatting a bit.
  97
  98 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
  99
 100         Reviewed by Darin.
 101
 102         - Add compare mode and a description of the benchmark to the hosted version.
 103
 104         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
 105         * hosted/sunspider.html: Added intro text.
 106         * make-hosted: Copy new file.
 107         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
 108         a function.
 109         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
 110         expects.
 111         * sunspider-compare-results: Use sunspider-standalone-compare.js.
 112
 113 2007-11-02  Darin Adler  <darin@apple.com>
 114
 115         * sunspider: Changed "--base" to "--set-baseline".
 116
 117 2007-11-02  Darin Adler  <darin@apple.com>
 118
 119         Reviewed by Maciej.
 120
 121         * sunspider: Added a "--base" option that records the current test as a baseline.
 122         Moved the code to find the newest .mshark file into a function (since I needed to use
 123         it in the other script and I prefer to copy/paste a function).
 124
 125         * sunspider-compare-results: Added logic to compare the baseline with the last
 126         run when you don't pass any parameters.
 127
 128 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 129
 130         Reviewed by Eric.
 131
 132         - Added list of ECMAScript3 features that SunSpider does not cover currently.
 133
 134         * UNCOVERED: Added.
 135
 136 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 137
 138         Reviewed by Oliver.
 139
 140         - Give better error messages when sunspider-compare-results fails
 141
 142         * sunspider-compare-results:
 143
 144 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 145
 146         Reviewed by Darin.
 147
 148         - fix Date tests to work in command-line SpiderMonkey
 149
 150         * tests/date-format-tofte.js:
 151         * tests/date-format-xparb.js:
 152
 153 2007-10-29  Darin Adler  <darin@apple.com>
 154
 155         Reviewed by Maciej.
 156
 157         * sunspider: Added --shark20 option, to run Shark at its highest sample
 158         resolution instead of its default.
 159
 160 2007-10-28  Adam Roben  <aroben@apple.com>
 161
 162         Fix SunSpider on Windows
 163
 164         Reviewed by Eric.
 165
 166         * sunspider: Don't use colons in filenames.
 167
 168 2007-10-28  Darin Adler  <darin@apple.com>
 169
 170         Reviewed by Adam.
 171
 172         * resources/sunspider-analyze-results.js: Tweak the output format a little.
 173         Change so that when there's only one run we don't write out confidence
 174         intervals at all rather than writing "NaN" over and over again.
 175
 176         * sunspider: Use the time and date as a suffix on the results file, that way
 177         all the results are left behind in the tmp directory. This will make it easy
 178         for us to add features that compare past results in the future. Also rename
 179         the .mshark files using the same scheme.
 180
 181         * sunspider-compare-results: Relax the parsing rules so we can parse the
 182         results file exactly as it's emitted from sunspider.
 183
 184 2007-10-25  Adam Roben  <aroben@apple.com>
 185
 186         Strip carriage returns from results
 187
 188         Carriage returns were screwing up sunspider-compare-results on
 189         Windows.
 190
 191         Rubberstamped by Oliver.
 192
 193         * sunspider:
 194
 195 2007-10-24  Darin Adler  <darin@apple.com>
 196
 197         Reviewed by Eric.
 198
 199         * sunspider: Open the profile in shark after completing a run.
 200         * sunspider-compare-results: Tiny perl tweaks.
 201
 202 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 203
 204         Reviewed by Darin.
 205
 206         - Don't hardcode my path to testkjs
 207
 208         * sunspider-compare-results:
 209
 210 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 211
 212         Reviewed by Darin.
 213
 214         - Add a compare script to compare results, and improve formatting of standard results a bit.
 215
 216         * resources/sunspider-analyze-results.js:
 217         * resources/sunspider-compare-results.js: Added.
 218         * sunspider-compare-results: Added.
 219
 220 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 221
 222         Reviewed by Darin.
 223
 224         * resources/sunspider-analyze-results.js: Do the error range
 225         properly, using the t-distribution instead of 1.96 (which was
 226         based on the normal distribution).
 227
 228         * sunspider: Print results in a way that will be friendlier to the
 229         soon-to-come compare mode.
 230
 231 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
 232
 233         Reviewed by Review.
 234
 235         - add more tests (probably the final set for now)
 236
 237         Already balanced. Added date, regexp, control flow, and a few
 238         more string and object/array access tests.
 239
 240         * tests/LIST:
 241         * tests/access-fannkuch.js: Added.
 242         * tests/access-nbody.js: Added.
 243         * tests/controlflow-recursive.js: Added.
 244         * tests/date-format-tofte.js: Added.
 245         * tests/date-format-xparb.js: Added.
 246         * tests/regexp-dna.js: Added.
 247         * tests/string-validate-input.js: Added.
 248
 249 2007-10-21  Eric Seidel  <eric@webkit.org>
 250
 251         Reviewed by mjs.
 252
 253         * sunspider: make --shark only sample from testkjs
 254
 255 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 256
 257         Rubber stamped by Eric.
 258
 259         - added crypto tests (already balanced)
 260
 261         * tests/LIST:
 262         * tests/crypto-aes.js: Added.
 263         * tests/crypto-md5.js: Added.
 264         * tests/crypto-sha1.js: Added.
 265
 266 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 267
 268         Reviewed by Adam.
 269
 270         - rebalance new tests
 271
 272         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
 273
 274         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 275
 276         This balances them with the existing tests.
 277
 278         * tests/access-binary-trees.js:
 279         * tests/access-nsieve.js:
 280         * tests/bitops-nsieve-bits.js:
 281         * tests/math-partial-sums.js:
 282         * tests/math-spectral-norm.js:
 283         * tests/string-fasta.js:
 284
 285 2007-10-20  Eric Seidel  <eric@webkit.org>
 286
 287         Reviewed by darin.
 288
 289         Add improved argument handling to sunspider, including
 290         --runs=<number>, --tests=<pattern, --shark, and --help
 291         Also re-factor code into subroutines
 292
 293         * sunspider: improved argument handling
 294
 295 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 296
 297         Reviewed by Mark.
 298
 299         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
 300
 301         * TODO:
 302         * tests/LIST:
 303         * tests/access-binary-trees.js: Added.
 304         * tests/access-nsieve.js: Added.
 305         * tests/bitops-nsieve-bits.js: Added.
 306         * tests/math-partial-sums.js: Added.
 307         * tests/math-spectral-norm.js: Added.
 308         * tests/string-fasta.js: Added.
 309
 310 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 311
 312         Reviewed by Darin.
 313
 314         * TODO: Updated to note areas that are now well-covered.
 315
 316         - Change the tests and the driver to avoid leaving large object
 317         graphs hanging around, since that throws off the subsequent tests.
 318
 319         * resources/sunspider-standalone-driver.js:
 320         * tests/3d-cube.js:
 321         * tests/3d-morph.js:
 322         * tests/3d-raytrace.js:
 323         * tests/bitops-bitwise-and.js:
 324         * tests/string-base64.js:
 325         * tests/string-tagcloud.js:
 326
 327 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 328
 329         Reviewed by Nikolas.
 330
 331         - Rebalance test complexity.
 332
 333         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
 334
 335         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 336
 337         However, for some tests, IE7 was a huge outlier compared to other
 338         major browsers. For those tests, I used Firefox 2.0.0.8 on the
 339         same OS and hardware as a normalization basis.
 340
 341         * tests/3d-morph.js:
 342         * tests/3d-raytrace.js:
 343         * tests/bitops-3bit-bits-in-byte.js:
 344         * tests/bitops-bits-in-byte.js:
 345         * tests/bitops-bitwise-and.js:
 346         * tests/math-cordic.js:
 347         * tests/string-base64.js:
 348         * tests/string-tagcloud.js:
 349         * tests/string-unpack-code.js:
 350
 351 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 352
 353         Reviewed by Oliver.
 354
 355         - Fix the browser-hosted driver to be compatible with Safari 2
 356
 357         * hosted/sunspider-driver.html:
 358
 359 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 360
 361         Rubber stamped by Hyatt.
 362
 363         - Add BSD license notice to cordic test, the original author said to do so.
 364
 365         * tests/math-cordic.js:
 366
 367 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 368
 369         Reviewed by Eric.
 370
 371         - Support Windows and Windows browsers.
 372
 373         * hosted/sunspider-results.html: Display results correctly in IE.
 374         * sunspider: tweaks to work on Windows, and with spaces in the path.
 375
 376 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 377
 378         Rubber stamped by Oliver.
 379
 380         * tests/string-tagcloud.js: Remove stray print()
 381
 382 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
 383
 384         Reviewed by Oliver.
 385
 386         Add some string test cases.
 387
 388         * tests/string-base64.js: Added. Does base64 encoding and decoding.
 389         * tests/string-tagcloud.js: Added. Parses a JSON string and
 390         generates tagcloud HTML markup based on the data.
 391         * tests/string-unpack-code.js: Added. Unpack compressed versions of
 392         four of the most popular JavaScript libraries.
 393         * tests/LIST: Added the new tests.
 394
 395 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
 396
 397         Reviewed by Anders.
 398
 399         * TODO: Updated with info on how to do the stats properly.
 400
 401         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
 402         avoid overly slowing down the benchmark.
 403
 404         Added some 3D-related test cases.
 405
 406         * tests/3d-cube.js: Added.
 407         * tests/3d-morph.js: Added.
 408         * tests/3d-raytrace.js: Added.
 409         * tests/LIST:
 410
 411 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
 412
 413         Reviewed by Darin.
 414
 415         - New JavaScript benchmark
 416         http://bugs.webkit.org/show_bug.cgi?id=15515
 417
 418         A start on a new JavaScript benchmark, with standalone and
 419         browser-hosted drivers.
 420
 421         The test content is still incomplete and the drivers could use
 422         some more features but there is enough here to make a good start.
 423
 424         The drivers compute a 95% confidence interval on the mean for the
 425         whole test, each category, and each individual test to make it
 426         easier to tell whether differences are statistically
 427         significant. The confidence interval can be narrowed by running
 428         the test more times.
 429
 430         * TODO: Added. Remaining things that need to be done.
 431         * sunspider: Added. Perl script that acts as the standalone test driver.
 432         * resources/sunspider-standalone-driver.js: Added. JavaScript part
 433         of standalone test driver.
 434         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
 435         analysis code.
 436         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
 437         * make-hosted: Added. Script to generate browser-hosted tests.
 438         * hosted/sunspider.html: Added. Start page for browser-hosted test.
 439         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
 440         * hosted/sunspider-results.html: Added. Results page for browser-hosted
 441         tests.
 442         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
 443         tests.
 444         * tests/LIST: Added. List of tests to use.
 445         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
 446         * tests/bitops-bits-in-byte.js: Added. Ditto.
 447         * tests/bitops-bitwise-and.js: Added. Ditto.
 448         * tests/math-cordic.js: Added. Ditto.
 449