SunSpider/ChangeLog

   1 2007-11-02  Darin Adler  <darin@apple.com>
   2
   3         * sunspider: Changed "--base" to "--set-baseline".
   4
   5 2007-11-02  Darin Adler  <darin@apple.com>
   6
   7         Reviewed by Maciej.
   8
   9         * sunspider: Added a "--base" option that records the current test as a baseline.
  10         Moved the code to find the newest .mshark file into a function (since I needed to use
  11         it in the other script and I prefer to copy/paste a function).
  12
  13         * sunspider-compare-results: Added logic to compare the baseline with the last
  14         run when you don't pass any parameters.
  15
  16 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
  17
  18         Reviewed by Eric.
  19
  20         - Added list of ECMAScript3 features that SunSpider does not cover currently.
  21
  22         * UNCOVERED: Added.
  23
  24 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
  25
  26         Reviewed by Oliver.
  27
  28         - Give better error messages when sunspider-compare-results fails
  29
  30         * sunspider-compare-results:
  31
  32 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
  33
  34         Reviewed by Darin.
  35
  36         - fix Date tests to work in command-line SpiderMonkey
  37
  38         * tests/date-format-tofte.js:
  39         * tests/date-format-xparb.js:
  40
  41 2007-10-29  Darin Adler  <darin@apple.com>
  42
  43         Reviewed by Maciej.
  44
  45         * sunspider: Added --shark20 option, to run Shark at its highest sample
  46         resolution instead of its default.
  47
  48 2007-10-28  Adam Roben  <aroben@apple.com>
  49
  50         Fix SunSpider on Windows
  51
  52         Reviewed by Eric.
  53
  54         * sunspider: Don't use colons in filenames.
  55
  56 2007-10-28  Darin Adler  <darin@apple.com>
  57
  58         Reviewed by Adam.
  59
  60         * resources/sunspider-analyze-results.js: Tweak the output format a little.
  61         Change so that when there's only one run we don't write out confidence
  62         intervals at all rather than writing "NaN" over and over again.
  63
  64         * sunspider: Use the time and date as a suffix on the results file, that way
  65         all the results are left behind in the tmp directory. This will make it easy
  66         for us to add features that compare past results in the future. Also rename
  67         the .mshark files using the same scheme.
  68
  69         * sunspider-compare-results: Relax the parsing rules so we can parse the
  70         results file exactly as it's emitted from sunspider.
  71
  72 2007-10-25  Adam Roben  <aroben@apple.com>
  73
  74         Strip carriage returns from results
  75
  76         Carriage returns were screwing up sunspider-compare-results on
  77         Windows.
  78
  79         Rubberstamped by Oliver.
  80
  81         * sunspider:
  82
  83 2007-10-24  Darin Adler  <darin@apple.com>
  84
  85         Reviewed by Eric.
  86
  87         * sunspider: Open the profile in shark after completing a run.
  88         * sunspider-compare-results: Tiny perl tweaks.
  89
  90 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
  91
  92         Reviewed by Darin.
  93
  94         - Don't hardcode my path to testkjs
  95
  96         * sunspider-compare-results:
  97
  98 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
  99
 100         Reviewed by Darin.
 101
 102         - Add a compare script to compare results, and improve formatting of standard results a bit.
 103
 104         * resources/sunspider-analyze-results.js:
 105         * resources/sunspider-compare-results.js: Added.
 106         * sunspider-compare-results: Added.
 107
 108 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 109
 110         Reviewed by Darin.
 111
 112         * resources/sunspider-analyze-results.js: Do the error range
 113         properly, using the t-distribution instead of 1.96 (which was
 114         based on the normal distribution).
 115
 116         * sunspider: Print results in a way that will be friendlier to the
 117         soon-to-come compare mode.
 118
 119 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
 120
 121         Reviewed by Review.
 122
 123         - add more tests (probably the final set for now)
 124
 125         Already balanced. Added date, regexp, control flow, and a few
 126         more string and object/array access tests.
 127
 128         * tests/LIST:
 129         * tests/access-fannkuch.js: Added.
 130         * tests/access-nbody.js: Added.
 131         * tests/controlflow-recursive.js: Added.
 132         * tests/date-format-tofte.js: Added.
 133         * tests/date-format-xparb.js: Added.
 134         * tests/regexp-dna.js: Added.
 135         * tests/string-validate-input.js: Added.
 136
 137 2007-10-21  Eric Seidel  <eric@webkit.org>
 138
 139         Reviewed by mjs.
 140
 141         * sunspider: make --shark only sample from testkjs
 142
 143 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 144
 145         Rubber stamped by Eric.
 146
 147         - added crypto tests (already balanced)
 148
 149         * tests/LIST:
 150         * tests/crypto-aes.js: Added.
 151         * tests/crypto-md5.js: Added.
 152         * tests/crypto-sha1.js: Added.
 153
 154 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 155
 156         Reviewed by Adam.
 157
 158         - rebalance new tests
 159
 160         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
 161
 162         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 163
 164         This balances them with the existing tests.
 165
 166         * tests/access-binary-trees.js:
 167         * tests/access-nsieve.js:
 168         * tests/bitops-nsieve-bits.js:
 169         * tests/math-partial-sums.js:
 170         * tests/math-spectral-norm.js:
 171         * tests/string-fasta.js:
 172
 173 2007-10-20  Eric Seidel  <eric@webkit.org>
 174
 175         Reviewed by darin.
 176
 177         Add improved argument handling to sunspider, including
 178         --runs=<number>, --tests=<pattern, --shark, and --help
 179         Also re-factor code into subroutines
 180
 181         * sunspider: improved argument handling
 182
 183 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 184
 185         Reviewed by Mark.
 186
 187         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
 188
 189         * TODO:
 190         * tests/LIST:
 191         * tests/access-binary-trees.js: Added.
 192         * tests/access-nsieve.js: Added.
 193         * tests/bitops-nsieve-bits.js: Added.
 194         * tests/math-partial-sums.js: Added.
 195         * tests/math-spectral-norm.js: Added.
 196         * tests/string-fasta.js: Added.
 197
 198 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 199
 200         Reviewed by Darin.
 201
 202         * TODO: Updated to note areas that are now well-covered.
 203
 204         - Change the tests and the driver to avoid leaving large object
 205         graphs hanging around, since that throws off the subsequent tests.
 206
 207         * resources/sunspider-standalone-driver.js:
 208         * tests/3d-cube.js:
 209         * tests/3d-morph.js:
 210         * tests/3d-raytrace.js:
 211         * tests/bitops-bitwise-and.js:
 212         * tests/string-base64.js:
 213         * tests/string-tagcloud.js:
 214
 215 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 216
 217         Reviewed by Nikolas.
 218
 219         - Rebalance test complexity.
 220
 221         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
 222
 223         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 224
 225         However, for some tests, IE7 was a huge outlier compared to other
 226         major browsers. For those tests, I used Firefox 2.0.0.8 on the
 227         same OS and hardware as a normalization basis.
 228
 229         * tests/3d-morph.js:
 230         * tests/3d-raytrace.js:
 231         * tests/bitops-3bit-bits-in-byte.js:
 232         * tests/bitops-bits-in-byte.js:
 233         * tests/bitops-bitwise-and.js:
 234         * tests/math-cordic.js:
 235         * tests/string-base64.js:
 236         * tests/string-tagcloud.js:
 237         * tests/string-unpack-code.js:
 238
 239 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 240
 241         Reviewed by Oliver.
 242
 243         - Fix the browser-hosted driver to be compatible with Safari 2
 244
 245         * hosted/sunspider-driver.html:
 246
 247 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 248
 249         Rubber stamped by Hyatt.
 250
 251         - Add BSD license notice to cordic test, the original author said to do so.
 252
 253         * tests/math-cordic.js:
 254
 255 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 256
 257         Reviewed by Eric.
 258
 259         - Support Windows and Windows browsers.
 260
 261         * hosted/sunspider-results.html: Display results correctly in IE.
 262         * sunspider: tweaks to work on Windows, and with spaces in the path.
 263
 264 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 265
 266         Rubber stamped by Oliver.
 267
 268         * tests/string-tagcloud.js: Remove stray print()
 269
 270 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
 271
 272         Reviewed by Oliver.
 273
 274         Add some string test cases.
 275
 276         * tests/string-base64.js: Added. Does base64 encoding and decoding.
 277         * tests/string-tagcloud.js: Added. Parses a JSON string and
 278         generates tagcloud HTML markup based on the data.
 279         * tests/string-unpack-code.js: Added. Unpack compressed versions of
 280         four of the most popular JavaScript libraries.
 281         * tests/LIST: Added the new tests.
 282
 283 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
 284
 285         Reviewed by Anders.
 286
 287         * TODO: Updated with info on how to do the stats properly.
 288
 289         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
 290         avoid overly slowing down the benchmark.
 291
 292         Added some 3D-related test cases.
 293
 294         * tests/3d-cube.js: Added.
 295         * tests/3d-morph.js: Added.
 296         * tests/3d-raytrace.js: Added.
 297         * tests/LIST:
 298
 299 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
 300
 301         Reviewed by Darin.
 302
 303         - New JavaScript benchmark
 304         http://bugs.webkit.org/show_bug.cgi?id=15515
 305
 306         A start on a new JavaScript benchmark, with standalone and
 307         browser-hosted drivers.
 308
 309         The test content is still incomplete and the drivers could use
 310         some more features but there is enough here to make a good start.
 311
 312         The drivers compute a 95% confidence interval on the mean for the
 313         whole test, each category, and each individual test to make it
 314         easier to tell whether differences are statistically
 315         significant. The confidence interval can be narrowed by running
 316         the test more times.
 317
 318         * TODO: Added. Remaining things that need to be done.
 319         * sunspider: Added. Perl script that acts as the standalone test driver.
 320         * resources/sunspider-standalone-driver.js: Added. JavaScript part
 321         of standalone test driver.
 322         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
 323         analysis code.
 324         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
 325         * make-hosted: Added. Script to generate browser-hosted tests.
 326         * hosted/sunspider.html: Added. Start page for browser-hosted test.
 327         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
 328         * hosted/sunspider-results.html: Added. Results page for browser-hosted
 329         tests.
 330         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
 331         tests.
 332         * tests/LIST: Added. List of tests to use.
 333         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
 334         * tests/bitops-bits-in-byte.js: Added. Ditto.
 335         * tests/bitops-bitwise-and.js: Added. Ditto.
 336         * tests/math-cordic.js: Added. Ditto.
 337