SunSpider/ChangeLog

   1 2007-10-28  Adam Roben  <aroben@apple.com>
   2
   3         Fix SunSpider on Windows
   4
   5         Reviewed by Eric.
   6
   7         * sunspider: Don't use colons in filenames.
   8
   9 2007-10-28  Darin Adler  <darin@apple.com>
  10
  11         Reviewed by Adam.
  12
  13         * resources/sunspider-analyze-results.js: Tweak the output format a little.
  14         Change so that when there's only one run we don't write out confidence
  15         intervals at all rather than writing "NaN" over and over again.
  16
  17         * sunspider: Use the time and date as a suffix on the results file, that way
  18         all the results are left behind in the tmp directory. This will make it easy
  19         for us to add features that compare past results in the future. Also rename
  20         the .mshark files using the same scheme.
  21
  22         * sunspider-compare-results: Relax the parsing rules so we can parse the
  23         results file exactly as it's emitted from sunspider.
  24
  25 2007-10-25  Adam Roben  <aroben@apple.com>
  26
  27         Strip carriage returns from results
  28
  29         Carriage returns were screwing up sunspider-compare-results on
  30         Windows.
  31
  32         Rubberstamped by Oliver.
  33
  34         * sunspider:
  35
  36 2007-10-24  Darin Adler  <darin@apple.com>
  37
  38         Reviewed by Eric.
  39
  40         * sunspider: Open the profile in shark after completing a run.
  41         * sunspider-compare-results: Tiny perl tweaks.
  42
  43 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
  44
  45         Reviewed by Darin.
  46
  47         - Don't hardcode my path to testkjs
  48
  49         * sunspider-compare-results:
  50
  51 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
  52
  53         Reviewed by Darin.
  54
  55         - Add a compare script to compare results, and improve formatting of standard results a bit.
  56
  57         * resources/sunspider-analyze-results.js:
  58         * resources/sunspider-compare-results.js: Added.
  59         * sunspider-compare-results: Added.
  60
  61 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
  62
  63         Reviewed by Darin.
  64
  65         * resources/sunspider-analyze-results.js: Do the error range
  66         properly, using the t-distribution instead of 1.96 (which was
  67         based on the normal distribution).
  68
  69         * sunspider: Print results in a way that will be friendlier to the
  70         soon-to-come compare mode.
  71
  72 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
  73
  74         Reviewed by Review.
  75
  76         - add more tests (probably the final set for now)
  77
  78         Already balanced. Added date, regexp, control flow, and a few
  79         more string and object/array access tests.
  80
  81         * tests/LIST:
  82         * tests/access-fannkuch.js: Added.
  83         * tests/access-nbody.js: Added.
  84         * tests/controlflow-recursive.js: Added.
  85         * tests/date-format-tofte.js: Added.
  86         * tests/date-format-xparb.js: Added.
  87         * tests/regexp-dna.js: Added.
  88         * tests/string-validate-input.js: Added.
  89
  90 2007-10-21  Eric Seidel  <eric@webkit.org>
  91
  92         Reviewed by mjs.
  93
  94         * sunspider: make --shark only sample from testkjs
  95
  96 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
  97
  98         Rubber stamped by Eric.
  99
 100         - added crypto tests (already balanced)
 101
 102         * tests/LIST:
 103         * tests/crypto-aes.js: Added.
 104         * tests/crypto-md5.js: Added.
 105         * tests/crypto-sha1.js: Added.
 106
 107 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 108
 109         Reviewed by Adam.
 110
 111         - rebalance new tests
 112
 113         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
 114
 115         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 116
 117         This balances them with the existing tests.
 118
 119         * tests/access-binary-trees.js:
 120         * tests/access-nsieve.js:
 121         * tests/bitops-nsieve-bits.js:
 122         * tests/math-partial-sums.js:
 123         * tests/math-spectral-norm.js:
 124         * tests/string-fasta.js:
 125
 126 2007-10-20  Eric Seidel  <eric@webkit.org>
 127
 128         Reviewed by darin.
 129
 130         Add improved argument handling to sunspider, including
 131         --runs=<number>, --tests=<pattern, --shark, and --help
 132         Also re-factor code into subroutines
 133
 134         * sunspider: improved argument handling
 135
 136 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 137
 138         Reviewed by Mark.
 139
 140         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
 141
 142         * TODO:
 143         * tests/LIST:
 144         * tests/access-binary-trees.js: Added.
 145         * tests/access-nsieve.js: Added.
 146         * tests/bitops-nsieve-bits.js: Added.
 147         * tests/math-partial-sums.js: Added.
 148         * tests/math-spectral-norm.js: Added.
 149         * tests/string-fasta.js: Added.
 150
 151 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 152
 153         Reviewed by Darin.
 154
 155         * TODO: Updated to note areas that are now well-covered.
 156
 157         - Change the tests and the driver to avoid leaving large object
 158         graphs hanging around, since that throws off the subsequent tests.
 159
 160         * resources/sunspider-standalone-driver.js:
 161         * tests/3d-cube.js:
 162         * tests/3d-morph.js:
 163         * tests/3d-raytrace.js:
 164         * tests/bitops-bitwise-and.js:
 165         * tests/string-base64.js:
 166         * tests/string-tagcloud.js:
 167
 168 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 169
 170         Reviewed by Nikolas.
 171
 172         - Rebalance test complexity.
 173
 174         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
 175
 176         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 177
 178         However, for some tests, IE7 was a huge outlier compared to other
 179         major browsers. For those tests, I used Firefox 2.0.0.8 on the
 180         same OS and hardware as a normalization basis.
 181
 182         * tests/3d-morph.js:
 183         * tests/3d-raytrace.js:
 184         * tests/bitops-3bit-bits-in-byte.js:
 185         * tests/bitops-bits-in-byte.js:
 186         * tests/bitops-bitwise-and.js:
 187         * tests/math-cordic.js:
 188         * tests/string-base64.js:
 189         * tests/string-tagcloud.js:
 190         * tests/string-unpack-code.js:
 191
 192 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 193
 194         Reviewed by Oliver.
 195
 196         - Fix the browser-hosted driver to be compatible with Safari 2
 197
 198         * hosted/sunspider-driver.html:
 199
 200 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 201
 202         Rubber stamped by Hyatt.
 203
 204         - Add BSD license notice to cordic test, the original author said to do so.
 205
 206         * tests/math-cordic.js:
 207
 208 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 209
 210         Reviewed by Eric.
 211
 212         - Support Windows and Windows browsers.
 213
 214         * hosted/sunspider-results.html: Display results correctly in IE.
 215         * sunspider: tweaks to work on Windows, and with spaces in the path.
 216
 217 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 218
 219         Rubber stamped by Oliver.
 220
 221         * tests/string-tagcloud.js: Remove stray print()
 222
 223 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
 224
 225         Reviewed by Oliver.
 226
 227         Add some string test cases.
 228
 229         * tests/string-base64.js: Added. Does base64 encoding and decoding.
 230         * tests/string-tagcloud.js: Added. Parses a JSON string and
 231         generates tagcloud HTML markup based on the data.
 232         * tests/string-unpack-code.js: Added. Unpack compressed versions of
 233         four of the most popular JavaScript libraries.
 234         * tests/LIST: Added the new tests.
 235
 236 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
 237
 238         Reviewed by Anders.
 239
 240         * TODO: Updated with info on how to do the stats properly.
 241
 242         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
 243         avoid overly slowing down the benchmark.
 244
 245         Added some 3D-related test cases.
 246
 247         * tests/3d-cube.js: Added.
 248         * tests/3d-morph.js: Added.
 249         * tests/3d-raytrace.js: Added.
 250         * tests/LIST:
 251
 252 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
 253
 254         Reviewed by Darin.
 255
 256         - New JavaScript benchmark
 257         http://bugs.webkit.org/show_bug.cgi?id=15515
 258
 259         A start on a new JavaScript benchmark, with standalone and
 260         browser-hosted drivers.
 261
 262         The test content is still incomplete and the drivers could use
 263         some more features but there is enough here to make a good start.
 264
 265         The drivers compute a 95% confidence interval on the mean for the
 266         whole test, each category, and each individual test to make it
 267         easier to tell whether differences are statistically
 268         significant. The confidence interval can be narrowed by running
 269         the test more times.
 270
 271         * TODO: Added. Remaining things that need to be done.
 272         * sunspider: Added. Perl script that acts as the standalone test driver.
 273         * resources/sunspider-standalone-driver.js: Added. JavaScript part
 274         of standalone test driver.
 275         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
 276         analysis code.
 277         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
 278         * make-hosted: Added. Script to generate browser-hosted tests.
 279         * hosted/sunspider.html: Added. Start page for browser-hosted test.
 280         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
 281         * hosted/sunspider-results.html: Added. Results page for browser-hosted
 282         tests.
 283         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
 284         tests.
 285         * tests/LIST: Added. List of tests to use.
 286         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
 287         * tests/bitops-bits-in-byte.js: Added. Ditto.
 288         * tests/bitops-bitwise-and.js: Added. Ditto.
 289         * tests/math-cordic.js: Added. Ditto.
 290