SunSpider/ChangeLog

   1 2007-11-04  Darin Adler  <darin@apple.com>
   2
   3         * hosted/sunspider.html: Tweaked formatting a bit.
   4
   5 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
   6
   7         Reviewed by Darin.
   8
   9         - Add compare mode and a description of the benchmark to the hosted version.
  10
  11         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
  12         * hosted/sunspider.html: Added intro text.
  13         * make-hosted: Copy new file.
  14         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
  15         a function.
  16         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
  17         expects.
  18         * sunspider-compare-results: Use sunspider-standalone-compare.js.
  19
  20 2007-11-02  Darin Adler  <darin@apple.com>
  21
  22         * sunspider: Changed "--base" to "--set-baseline".
  23
  24 2007-11-02  Darin Adler  <darin@apple.com>
  25
  26         Reviewed by Maciej.
  27
  28         * sunspider: Added a "--base" option that records the current test as a baseline.
  29         Moved the code to find the newest .mshark file into a function (since I needed to use
  30         it in the other script and I prefer to copy/paste a function).
  31
  32         * sunspider-compare-results: Added logic to compare the baseline with the last
  33         run when you don't pass any parameters.
  34
  35 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
  36
  37         Reviewed by Eric.
  38
  39         - Added list of ECMAScript3 features that SunSpider does not cover currently.
  40
  41         * UNCOVERED: Added.
  42
  43 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
  44
  45         Reviewed by Oliver.
  46
  47         - Give better error messages when sunspider-compare-results fails
  48
  49         * sunspider-compare-results:
  50
  51 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
  52
  53         Reviewed by Darin.
  54
  55         - fix Date tests to work in command-line SpiderMonkey
  56
  57         * tests/date-format-tofte.js:
  58         * tests/date-format-xparb.js:
  59
  60 2007-10-29  Darin Adler  <darin@apple.com>
  61
  62         Reviewed by Maciej.
  63
  64         * sunspider: Added --shark20 option, to run Shark at its highest sample
  65         resolution instead of its default.
  66
  67 2007-10-28  Adam Roben  <aroben@apple.com>
  68
  69         Fix SunSpider on Windows
  70
  71         Reviewed by Eric.
  72
  73         * sunspider: Don't use colons in filenames.
  74
  75 2007-10-28  Darin Adler  <darin@apple.com>
  76
  77         Reviewed by Adam.
  78
  79         * resources/sunspider-analyze-results.js: Tweak the output format a little.
  80         Change so that when there's only one run we don't write out confidence
  81         intervals at all rather than writing "NaN" over and over again.
  82
  83         * sunspider: Use the time and date as a suffix on the results file, that way
  84         all the results are left behind in the tmp directory. This will make it easy
  85         for us to add features that compare past results in the future. Also rename
  86         the .mshark files using the same scheme.
  87
  88         * sunspider-compare-results: Relax the parsing rules so we can parse the
  89         results file exactly as it's emitted from sunspider.
  90
  91 2007-10-25  Adam Roben  <aroben@apple.com>
  92
  93         Strip carriage returns from results
  94
  95         Carriage returns were screwing up sunspider-compare-results on
  96         Windows.
  97
  98         Rubberstamped by Oliver.
  99
 100         * sunspider:
 101
 102 2007-10-24  Darin Adler  <darin@apple.com>
 103
 104         Reviewed by Eric.
 105
 106         * sunspider: Open the profile in shark after completing a run.
 107         * sunspider-compare-results: Tiny perl tweaks.
 108
 109 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 110
 111         Reviewed by Darin.
 112
 113         - Don't hardcode my path to testkjs
 114
 115         * sunspider-compare-results:
 116
 117 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 118
 119         Reviewed by Darin.
 120
 121         - Add a compare script to compare results, and improve formatting of standard results a bit.
 122
 123         * resources/sunspider-analyze-results.js:
 124         * resources/sunspider-compare-results.js: Added.
 125         * sunspider-compare-results: Added.
 126
 127 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 128
 129         Reviewed by Darin.
 130
 131         * resources/sunspider-analyze-results.js: Do the error range
 132         properly, using the t-distribution instead of 1.96 (which was
 133         based on the normal distribution).
 134
 135         * sunspider: Print results in a way that will be friendlier to the
 136         soon-to-come compare mode.
 137
 138 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
 139
 140         Reviewed by Review.
 141
 142         - add more tests (probably the final set for now)
 143
 144         Already balanced. Added date, regexp, control flow, and a few
 145         more string and object/array access tests.
 146
 147         * tests/LIST:
 148         * tests/access-fannkuch.js: Added.
 149         * tests/access-nbody.js: Added.
 150         * tests/controlflow-recursive.js: Added.
 151         * tests/date-format-tofte.js: Added.
 152         * tests/date-format-xparb.js: Added.
 153         * tests/regexp-dna.js: Added.
 154         * tests/string-validate-input.js: Added.
 155
 156 2007-10-21  Eric Seidel  <eric@webkit.org>
 157
 158         Reviewed by mjs.
 159
 160         * sunspider: make --shark only sample from testkjs
 161
 162 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 163
 164         Rubber stamped by Eric.
 165
 166         - added crypto tests (already balanced)
 167
 168         * tests/LIST:
 169         * tests/crypto-aes.js: Added.
 170         * tests/crypto-md5.js: Added.
 171         * tests/crypto-sha1.js: Added.
 172
 173 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 174
 175         Reviewed by Adam.
 176
 177         - rebalance new tests
 178
 179         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
 180
 181         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 182
 183         This balances them with the existing tests.
 184
 185         * tests/access-binary-trees.js:
 186         * tests/access-nsieve.js:
 187         * tests/bitops-nsieve-bits.js:
 188         * tests/math-partial-sums.js:
 189         * tests/math-spectral-norm.js:
 190         * tests/string-fasta.js:
 191
 192 2007-10-20  Eric Seidel  <eric@webkit.org>
 193
 194         Reviewed by darin.
 195
 196         Add improved argument handling to sunspider, including
 197         --runs=<number>, --tests=<pattern, --shark, and --help
 198         Also re-factor code into subroutines
 199
 200         * sunspider: improved argument handling
 201
 202 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 203
 204         Reviewed by Mark.
 205
 206         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
 207
 208         * TODO:
 209         * tests/LIST:
 210         * tests/access-binary-trees.js: Added.
 211         * tests/access-nsieve.js: Added.
 212         * tests/bitops-nsieve-bits.js: Added.
 213         * tests/math-partial-sums.js: Added.
 214         * tests/math-spectral-norm.js: Added.
 215         * tests/string-fasta.js: Added.
 216
 217 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 218
 219         Reviewed by Darin.
 220
 221         * TODO: Updated to note areas that are now well-covered.
 222
 223         - Change the tests and the driver to avoid leaving large object
 224         graphs hanging around, since that throws off the subsequent tests.
 225
 226         * resources/sunspider-standalone-driver.js:
 227         * tests/3d-cube.js:
 228         * tests/3d-morph.js:
 229         * tests/3d-raytrace.js:
 230         * tests/bitops-bitwise-and.js:
 231         * tests/string-base64.js:
 232         * tests/string-tagcloud.js:
 233
 234 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 235
 236         Reviewed by Nikolas.
 237
 238         - Rebalance test complexity.
 239
 240         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
 241
 242         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 243
 244         However, for some tests, IE7 was a huge outlier compared to other
 245         major browsers. For those tests, I used Firefox 2.0.0.8 on the
 246         same OS and hardware as a normalization basis.
 247
 248         * tests/3d-morph.js:
 249         * tests/3d-raytrace.js:
 250         * tests/bitops-3bit-bits-in-byte.js:
 251         * tests/bitops-bits-in-byte.js:
 252         * tests/bitops-bitwise-and.js:
 253         * tests/math-cordic.js:
 254         * tests/string-base64.js:
 255         * tests/string-tagcloud.js:
 256         * tests/string-unpack-code.js:
 257
 258 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 259
 260         Reviewed by Oliver.
 261
 262         - Fix the browser-hosted driver to be compatible with Safari 2
 263
 264         * hosted/sunspider-driver.html:
 265
 266 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 267
 268         Rubber stamped by Hyatt.
 269
 270         - Add BSD license notice to cordic test, the original author said to do so.
 271
 272         * tests/math-cordic.js:
 273
 274 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 275
 276         Reviewed by Eric.
 277
 278         - Support Windows and Windows browsers.
 279
 280         * hosted/sunspider-results.html: Display results correctly in IE.
 281         * sunspider: tweaks to work on Windows, and with spaces in the path.
 282
 283 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 284
 285         Rubber stamped by Oliver.
 286
 287         * tests/string-tagcloud.js: Remove stray print()
 288
 289 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
 290
 291         Reviewed by Oliver.
 292
 293         Add some string test cases.
 294
 295         * tests/string-base64.js: Added. Does base64 encoding and decoding.
 296         * tests/string-tagcloud.js: Added. Parses a JSON string and
 297         generates tagcloud HTML markup based on the data.
 298         * tests/string-unpack-code.js: Added. Unpack compressed versions of
 299         four of the most popular JavaScript libraries.
 300         * tests/LIST: Added the new tests.
 301
 302 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
 303
 304         Reviewed by Anders.
 305
 306         * TODO: Updated with info on how to do the stats properly.
 307
 308         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
 309         avoid overly slowing down the benchmark.
 310
 311         Added some 3D-related test cases.
 312
 313         * tests/3d-cube.js: Added.
 314         * tests/3d-morph.js: Added.
 315         * tests/3d-raytrace.js: Added.
 316         * tests/LIST:
 317
 318 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
 319
 320         Reviewed by Darin.
 321
 322         - New JavaScript benchmark
 323         http://bugs.webkit.org/show_bug.cgi?id=15515
 324
 325         A start on a new JavaScript benchmark, with standalone and
 326         browser-hosted drivers.
 327
 328         The test content is still incomplete and the drivers could use
 329         some more features but there is enough here to make a good start.
 330
 331         The drivers compute a 95% confidence interval on the mean for the
 332         whole test, each category, and each individual test to make it
 333         easier to tell whether differences are statistically
 334         significant. The confidence interval can be narrowed by running
 335         the test more times.
 336
 337         * TODO: Added. Remaining things that need to be done.
 338         * sunspider: Added. Perl script that acts as the standalone test driver.
 339         * resources/sunspider-standalone-driver.js: Added. JavaScript part
 340         of standalone test driver.
 341         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
 342         analysis code.
 343         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
 344         * make-hosted: Added. Script to generate browser-hosted tests.
 345         * hosted/sunspider.html: Added. Start page for browser-hosted test.
 346         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
 347         * hosted/sunspider-results.html: Added. Results page for browser-hosted
 348         tests.
 349         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
 350         tests.
 351         * tests/LIST: Added. List of tests to use.
 352         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
 353         * tests/bitops-bits-in-byte.js: Added. Ditto.
 354         * tests/bitops-bitwise-and.js: Added. Ditto.
 355         * tests/math-cordic.js: Added. Ditto.
 356