SunSpider/ChangeLog

   1 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
   2
   3         Reviewed by Eric.
   4
   5         - Added list of ECMAScript3 features that SunSpider does not cover currently.
   6
   7         * UNCOVERED: Added.
   8
   9 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
  10
  11         Reviewed by Oliver.
  12
  13         - Give better error messages when sunspider-compare-results fails
  14
  15         * sunspider-compare-results:
  16
  17 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
  18
  19         Reviewed by Darin.
  20
  21         - fix Date tests to work in command-line SpiderMonkey
  22
  23         * tests/date-format-tofte.js:
  24         * tests/date-format-xparb.js:
  25
  26 2007-10-29  Darin Adler  <darin@apple.com>
  27
  28         Reviewed by Maciej.
  29
  30         * sunspider: Added --shark20 option, to run Shark at its highest sample
  31         resolution instead of its default.
  32
  33 2007-10-28  Adam Roben  <aroben@apple.com>
  34
  35         Fix SunSpider on Windows
  36
  37         Reviewed by Eric.
  38
  39         * sunspider: Don't use colons in filenames.
  40
  41 2007-10-28  Darin Adler  <darin@apple.com>
  42
  43         Reviewed by Adam.
  44
  45         * resources/sunspider-analyze-results.js: Tweak the output format a little.
  46         Change so that when there's only one run we don't write out confidence
  47         intervals at all rather than writing "NaN" over and over again.
  48
  49         * sunspider: Use the time and date as a suffix on the results file, that way
  50         all the results are left behind in the tmp directory. This will make it easy
  51         for us to add features that compare past results in the future. Also rename
  52         the .mshark files using the same scheme.
  53
  54         * sunspider-compare-results: Relax the parsing rules so we can parse the
  55         results file exactly as it's emitted from sunspider.
  56
  57 2007-10-25  Adam Roben  <aroben@apple.com>
  58
  59         Strip carriage returns from results
  60
  61         Carriage returns were screwing up sunspider-compare-results on
  62         Windows.
  63
  64         Rubberstamped by Oliver.
  65
  66         * sunspider:
  67
  68 2007-10-24  Darin Adler  <darin@apple.com>
  69
  70         Reviewed by Eric.
  71
  72         * sunspider: Open the profile in shark after completing a run.
  73         * sunspider-compare-results: Tiny perl tweaks.
  74
  75 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
  76
  77         Reviewed by Darin.
  78
  79         - Don't hardcode my path to testkjs
  80
  81         * sunspider-compare-results:
  82
  83 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
  84
  85         Reviewed by Darin.
  86
  87         - Add a compare script to compare results, and improve formatting of standard results a bit.
  88
  89         * resources/sunspider-analyze-results.js:
  90         * resources/sunspider-compare-results.js: Added.
  91         * sunspider-compare-results: Added.
  92
  93 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
  94
  95         Reviewed by Darin.
  96
  97         * resources/sunspider-analyze-results.js: Do the error range
  98         properly, using the t-distribution instead of 1.96 (which was
  99         based on the normal distribution).
 100
 101         * sunspider: Print results in a way that will be friendlier to the
 102         soon-to-come compare mode.
 103
 104 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
 105
 106         Reviewed by Review.
 107
 108         - add more tests (probably the final set for now)
 109
 110         Already balanced. Added date, regexp, control flow, and a few
 111         more string and object/array access tests.
 112
 113         * tests/LIST:
 114         * tests/access-fannkuch.js: Added.
 115         * tests/access-nbody.js: Added.
 116         * tests/controlflow-recursive.js: Added.
 117         * tests/date-format-tofte.js: Added.
 118         * tests/date-format-xparb.js: Added.
 119         * tests/regexp-dna.js: Added.
 120         * tests/string-validate-input.js: Added.
 121
 122 2007-10-21  Eric Seidel  <eric@webkit.org>
 123
 124         Reviewed by mjs.
 125
 126         * sunspider: make --shark only sample from testkjs
 127
 128 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 129
 130         Rubber stamped by Eric.
 131
 132         - added crypto tests (already balanced)
 133
 134         * tests/LIST:
 135         * tests/crypto-aes.js: Added.
 136         * tests/crypto-md5.js: Added.
 137         * tests/crypto-sha1.js: Added.
 138
 139 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 140
 141         Reviewed by Adam.
 142
 143         - rebalance new tests
 144
 145         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
 146
 147         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 148
 149         This balances them with the existing tests.
 150
 151         * tests/access-binary-trees.js:
 152         * tests/access-nsieve.js:
 153         * tests/bitops-nsieve-bits.js:
 154         * tests/math-partial-sums.js:
 155         * tests/math-spectral-norm.js:
 156         * tests/string-fasta.js:
 157
 158 2007-10-20  Eric Seidel  <eric@webkit.org>
 159
 160         Reviewed by darin.
 161
 162         Add improved argument handling to sunspider, including
 163         --runs=<number>, --tests=<pattern, --shark, and --help
 164         Also re-factor code into subroutines
 165
 166         * sunspider: improved argument handling
 167
 168 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 169
 170         Reviewed by Mark.
 171
 172         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
 173
 174         * TODO:
 175         * tests/LIST:
 176         * tests/access-binary-trees.js: Added.
 177         * tests/access-nsieve.js: Added.
 178         * tests/bitops-nsieve-bits.js: Added.
 179         * tests/math-partial-sums.js: Added.
 180         * tests/math-spectral-norm.js: Added.
 181         * tests/string-fasta.js: Added.
 182
 183 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 184
 185         Reviewed by Darin.
 186
 187         * TODO: Updated to note areas that are now well-covered.
 188
 189         - Change the tests and the driver to avoid leaving large object
 190         graphs hanging around, since that throws off the subsequent tests.
 191
 192         * resources/sunspider-standalone-driver.js:
 193         * tests/3d-cube.js:
 194         * tests/3d-morph.js:
 195         * tests/3d-raytrace.js:
 196         * tests/bitops-bitwise-and.js:
 197         * tests/string-base64.js:
 198         * tests/string-tagcloud.js:
 199
 200 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 201
 202         Reviewed by Nikolas.
 203
 204         - Rebalance test complexity.
 205
 206         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
 207
 208         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 209
 210         However, for some tests, IE7 was a huge outlier compared to other
 211         major browsers. For those tests, I used Firefox 2.0.0.8 on the
 212         same OS and hardware as a normalization basis.
 213
 214         * tests/3d-morph.js:
 215         * tests/3d-raytrace.js:
 216         * tests/bitops-3bit-bits-in-byte.js:
 217         * tests/bitops-bits-in-byte.js:
 218         * tests/bitops-bitwise-and.js:
 219         * tests/math-cordic.js:
 220         * tests/string-base64.js:
 221         * tests/string-tagcloud.js:
 222         * tests/string-unpack-code.js:
 223
 224 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 225
 226         Reviewed by Oliver.
 227
 228         - Fix the browser-hosted driver to be compatible with Safari 2
 229
 230         * hosted/sunspider-driver.html:
 231
 232 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 233
 234         Rubber stamped by Hyatt.
 235
 236         - Add BSD license notice to cordic test, the original author said to do so.
 237
 238         * tests/math-cordic.js:
 239
 240 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 241
 242         Reviewed by Eric.
 243
 244         - Support Windows and Windows browsers.
 245
 246         * hosted/sunspider-results.html: Display results correctly in IE.
 247         * sunspider: tweaks to work on Windows, and with spaces in the path.
 248
 249 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 250
 251         Rubber stamped by Oliver.
 252
 253         * tests/string-tagcloud.js: Remove stray print()
 254
 255 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
 256
 257         Reviewed by Oliver.
 258
 259         Add some string test cases.
 260
 261         * tests/string-base64.js: Added. Does base64 encoding and decoding.
 262         * tests/string-tagcloud.js: Added. Parses a JSON string and
 263         generates tagcloud HTML markup based on the data.
 264         * tests/string-unpack-code.js: Added. Unpack compressed versions of
 265         four of the most popular JavaScript libraries.
 266         * tests/LIST: Added the new tests.
 267
 268 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
 269
 270         Reviewed by Anders.
 271
 272         * TODO: Updated with info on how to do the stats properly.
 273
 274         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
 275         avoid overly slowing down the benchmark.
 276
 277         Added some 3D-related test cases.
 278
 279         * tests/3d-cube.js: Added.
 280         * tests/3d-morph.js: Added.
 281         * tests/3d-raytrace.js: Added.
 282         * tests/LIST:
 283
 284 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
 285
 286         Reviewed by Darin.
 287
 288         - New JavaScript benchmark
 289         http://bugs.webkit.org/show_bug.cgi?id=15515
 290
 291         A start on a new JavaScript benchmark, with standalone and
 292         browser-hosted drivers.
 293
 294         The test content is still incomplete and the drivers could use
 295         some more features but there is enough here to make a good start.
 296
 297         The drivers compute a 95% confidence interval on the mean for the
 298         whole test, each category, and each individual test to make it
 299         easier to tell whether differences are statistically
 300         significant. The confidence interval can be narrowed by running
 301         the test more times.
 302
 303         * TODO: Added. Remaining things that need to be done.
 304         * sunspider: Added. Perl script that acts as the standalone test driver.
 305         * resources/sunspider-standalone-driver.js: Added. JavaScript part
 306         of standalone test driver.
 307         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
 308         analysis code.
 309         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
 310         * make-hosted: Added. Script to generate browser-hosted tests.
 311         * hosted/sunspider.html: Added. Start page for browser-hosted test.
 312         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
 313         * hosted/sunspider-results.html: Added. Results page for browser-hosted
 314         tests.
 315         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
 316         tests.
 317         * tests/LIST: Added. List of tests to use.
 318         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
 319         * tests/bitops-bits-in-byte.js: Added. Ditto.
 320         * tests/bitops-bitwise-and.js: Added. Ditto.
 321         * tests/math-cordic.js: Added. Ditto.
 322