SunSpider/ChangeLog

   1 2007-10-28  Darin Adler  <darin@apple.com>
   2
   3         Reviewed by Adam.
   4
   5         * resources/sunspider-analyze-results.js: Tweak the output format a little.
   6         Change so that when there's only one run we don't write out confidence
   7         intervals at all rather than writing "NaN" over and over again.
   8
   9         * sunspider: Use the time and date as a suffix on the results file, that way
  10         all the results are left behind in the tmp directory. This will make it easy
  11         for us to add features that compare past results in the future. Also rename
  12         the .mshark files using the same scheme.
  13
  14         * sunspider-compare-results: Relax the parsing rules so we can parse the
  15         results file exactly as it's emitted from sunspider.
  16
  17 2007-10-25  Adam Roben  <aroben@apple.com>
  18
  19         Strip carriage returns from results
  20
  21         Carriage returns were screwing up sunspider-compare-results on
  22         Windows.
  23
  24         Rubberstamped by Oliver.
  25
  26         * sunspider:
  27
  28 2007-10-24  Darin Adler  <darin@apple.com>
  29
  30         Reviewed by Eric.
  31
  32         * sunspider: Open the profile in shark after completing a run.
  33         * sunspider-compare-results: Tiny perl tweaks.
  34
  35 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
  36
  37         Reviewed by Darin.
  38
  39         - Don't hardcode my path to testkjs
  40
  41         * sunspider-compare-results:
  42
  43 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
  44
  45         Reviewed by Darin.
  46
  47         - Add a compare script to compare results, and improve formatting of standard results a bit.
  48
  49         * resources/sunspider-analyze-results.js:
  50         * resources/sunspider-compare-results.js: Added.
  51         * sunspider-compare-results: Added.
  52
  53 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
  54
  55         Reviewed by Darin.
  56
  57         * resources/sunspider-analyze-results.js: Do the error range
  58         properly, using the t-distribution instead of 1.96 (which was
  59         based on the normal distribution).
  60
  61         * sunspider: Print results in a way that will be friendlier to the
  62         soon-to-come compare mode.
  63
  64 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
  65
  66         Reviewed by Review.
  67
  68         - add more tests (probably the final set for now)
  69
  70         Already balanced. Added date, regexp, control flow, and a few
  71         more string and object/array access tests.
  72
  73         * tests/LIST:
  74         * tests/access-fannkuch.js: Added.
  75         * tests/access-nbody.js: Added.
  76         * tests/controlflow-recursive.js: Added.
  77         * tests/date-format-tofte.js: Added.
  78         * tests/date-format-xparb.js: Added.
  79         * tests/regexp-dna.js: Added.
  80         * tests/string-validate-input.js: Added.
  81
  82 2007-10-21  Eric Seidel  <eric@webkit.org>
  83
  84         Reviewed by mjs.
  85
  86         * sunspider: make --shark only sample from testkjs
  87
  88 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
  89
  90         Rubber stamped by Eric.
  91
  92         - added crypto tests (already balanced)
  93
  94         * tests/LIST:
  95         * tests/crypto-aes.js: Added.
  96         * tests/crypto-md5.js: Added.
  97         * tests/crypto-sha1.js: Added.
  98
  99 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 100
 101         Reviewed by Adam.
 102
 103         - rebalance new tests
 104
 105         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
 106
 107         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 108
 109         This balances them with the existing tests.
 110
 111         * tests/access-binary-trees.js:
 112         * tests/access-nsieve.js:
 113         * tests/bitops-nsieve-bits.js:
 114         * tests/math-partial-sums.js:
 115         * tests/math-spectral-norm.js:
 116         * tests/string-fasta.js:
 117
 118 2007-10-20  Eric Seidel  <eric@webkit.org>
 119
 120         Reviewed by darin.
 121
 122         Add improved argument handling to sunspider, including
 123         --runs=<number>, --tests=<pattern, --shark, and --help
 124         Also re-factor code into subroutines
 125
 126         * sunspider: improved argument handling
 127
 128 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 129
 130         Reviewed by Mark.
 131
 132         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
 133
 134         * TODO:
 135         * tests/LIST:
 136         * tests/access-binary-trees.js: Added.
 137         * tests/access-nsieve.js: Added.
 138         * tests/bitops-nsieve-bits.js: Added.
 139         * tests/math-partial-sums.js: Added.
 140         * tests/math-spectral-norm.js: Added.
 141         * tests/string-fasta.js: Added.
 142
 143 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 144
 145         Reviewed by Darin.
 146
 147         * TODO: Updated to note areas that are now well-covered.
 148
 149         - Change the tests and the driver to avoid leaving large object
 150         graphs hanging around, since that throws off the subsequent tests.
 151
 152         * resources/sunspider-standalone-driver.js:
 153         * tests/3d-cube.js:
 154         * tests/3d-morph.js:
 155         * tests/3d-raytrace.js:
 156         * tests/bitops-bitwise-and.js:
 157         * tests/string-base64.js:
 158         * tests/string-tagcloud.js:
 159
 160 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 161
 162         Reviewed by Nikolas.
 163
 164         - Rebalance test complexity.
 165
 166         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
 167
 168         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 169
 170         However, for some tests, IE7 was a huge outlier compared to other
 171         major browsers. For those tests, I used Firefox 2.0.0.8 on the
 172         same OS and hardware as a normalization basis.
 173
 174         * tests/3d-morph.js:
 175         * tests/3d-raytrace.js:
 176         * tests/bitops-3bit-bits-in-byte.js:
 177         * tests/bitops-bits-in-byte.js:
 178         * tests/bitops-bitwise-and.js:
 179         * tests/math-cordic.js:
 180         * tests/string-base64.js:
 181         * tests/string-tagcloud.js:
 182         * tests/string-unpack-code.js:
 183
 184 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 185
 186         Reviewed by Oliver.
 187
 188         - Fix the browser-hosted driver to be compatible with Safari 2
 189
 190         * hosted/sunspider-driver.html:
 191
 192 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 193
 194         Rubber stamped by Hyatt.
 195
 196         - Add BSD license notice to cordic test, the original author said to do so.
 197
 198         * tests/math-cordic.js:
 199
 200 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 201
 202         Reviewed by Eric.
 203
 204         - Support Windows and Windows browsers.
 205
 206         * hosted/sunspider-results.html: Display results correctly in IE.
 207         * sunspider: tweaks to work on Windows, and with spaces in the path.
 208
 209 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 210
 211         Rubber stamped by Oliver.
 212
 213         * tests/string-tagcloud.js: Remove stray print()
 214
 215 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
 216
 217         Reviewed by Oliver.
 218
 219         Add some string test cases.
 220
 221         * tests/string-base64.js: Added. Does base64 encoding and decoding.
 222         * tests/string-tagcloud.js: Added. Parses a JSON string and
 223         generates tagcloud HTML markup based on the data.
 224         * tests/string-unpack-code.js: Added. Unpack compressed versions of
 225         four of the most popular JavaScript libraries.
 226         * tests/LIST: Added the new tests.
 227
 228 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
 229
 230         Reviewed by Anders.
 231
 232         * TODO: Updated with info on how to do the stats properly.
 233
 234         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
 235         avoid overly slowing down the benchmark.
 236
 237         Added some 3D-related test cases.
 238
 239         * tests/3d-cube.js: Added.
 240         * tests/3d-morph.js: Added.
 241         * tests/3d-raytrace.js: Added.
 242         * tests/LIST:
 243
 244 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
 245
 246         Reviewed by Darin.
 247
 248         - New JavaScript benchmark
 249         http://bugs.webkit.org/show_bug.cgi?id=15515
 250
 251         A start on a new JavaScript benchmark, with standalone and
 252         browser-hosted drivers.
 253
 254         The test content is still incomplete and the drivers could use
 255         some more features but there is enough here to make a good start.
 256
 257         The drivers compute a 95% confidence interval on the mean for the
 258         whole test, each category, and each individual test to make it
 259         easier to tell whether differences are statistically
 260         significant. The confidence interval can be narrowed by running
 261         the test more times.
 262
 263         * TODO: Added. Remaining things that need to be done.
 264         * sunspider: Added. Perl script that acts as the standalone test driver.
 265         * resources/sunspider-standalone-driver.js: Added. JavaScript part
 266         of standalone test driver.
 267         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
 268         analysis code.
 269         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
 270         * make-hosted: Added. Script to generate browser-hosted tests.
 271         * hosted/sunspider.html: Added. Start page for browser-hosted test.
 272         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
 273         * hosted/sunspider-results.html: Added. Results page for browser-hosted
 274         tests.
 275         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
 276         tests.
 277         * tests/LIST: Added. List of tests to use.
 278         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
 279         * tests/bitops-bits-in-byte.js: Added. Ditto.
 280         * tests/bitops-bitwise-and.js: Added. Ditto.
 281         * tests/math-cordic.js: Added. Ditto.
 282