SunSpider/ChangeLog

   1 2007-10-29  Darin Adler  <darin@apple.com>
   2
   3         Reviewed by Maciej.
   4
   5         * sunspider: Added --shark20 option, to run Shark at its highest sample
   6         resolution instead of its default.
   7
   8 2007-10-28  Adam Roben  <aroben@apple.com>
   9
  10         Fix SunSpider on Windows
  11
  12         Reviewed by Eric.
  13
  14         * sunspider: Don't use colons in filenames.
  15
  16 2007-10-28  Darin Adler  <darin@apple.com>
  17
  18         Reviewed by Adam.
  19
  20         * resources/sunspider-analyze-results.js: Tweak the output format a little.
  21         Change so that when there's only one run we don't write out confidence
  22         intervals at all rather than writing "NaN" over and over again.
  23
  24         * sunspider: Use the time and date as a suffix on the results file, that way
  25         all the results are left behind in the tmp directory. This will make it easy
  26         for us to add features that compare past results in the future. Also rename
  27         the .mshark files using the same scheme.
  28
  29         * sunspider-compare-results: Relax the parsing rules so we can parse the
  30         results file exactly as it's emitted from sunspider.
  31
  32 2007-10-25  Adam Roben  <aroben@apple.com>
  33
  34         Strip carriage returns from results
  35
  36         Carriage returns were screwing up sunspider-compare-results on
  37         Windows.
  38
  39         Rubberstamped by Oliver.
  40
  41         * sunspider:
  42
  43 2007-10-24  Darin Adler  <darin@apple.com>
  44
  45         Reviewed by Eric.
  46
  47         * sunspider: Open the profile in shark after completing a run.
  48         * sunspider-compare-results: Tiny perl tweaks.
  49
  50 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
  51
  52         Reviewed by Darin.
  53
  54         - Don't hardcode my path to testkjs
  55
  56         * sunspider-compare-results:
  57
  58 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
  59
  60         Reviewed by Darin.
  61
  62         - Add a compare script to compare results, and improve formatting of standard results a bit.
  63
  64         * resources/sunspider-analyze-results.js:
  65         * resources/sunspider-compare-results.js: Added.
  66         * sunspider-compare-results: Added.
  67
  68 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
  69
  70         Reviewed by Darin.
  71
  72         * resources/sunspider-analyze-results.js: Do the error range
  73         properly, using the t-distribution instead of 1.96 (which was
  74         based on the normal distribution).
  75
  76         * sunspider: Print results in a way that will be friendlier to the
  77         soon-to-come compare mode.
  78
  79 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
  80
  81         Reviewed by Review.
  82
  83         - add more tests (probably the final set for now)
  84
  85         Already balanced. Added date, regexp, control flow, and a few
  86         more string and object/array access tests.
  87
  88         * tests/LIST:
  89         * tests/access-fannkuch.js: Added.
  90         * tests/access-nbody.js: Added.
  91         * tests/controlflow-recursive.js: Added.
  92         * tests/date-format-tofte.js: Added.
  93         * tests/date-format-xparb.js: Added.
  94         * tests/regexp-dna.js: Added.
  95         * tests/string-validate-input.js: Added.
  96
  97 2007-10-21  Eric Seidel  <eric@webkit.org>
  98
  99         Reviewed by mjs.
 100
 101         * sunspider: make --shark only sample from testkjs
 102
 103 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 104
 105         Rubber stamped by Eric.
 106
 107         - added crypto tests (already balanced)
 108
 109         * tests/LIST:
 110         * tests/crypto-aes.js: Added.
 111         * tests/crypto-md5.js: Added.
 112         * tests/crypto-sha1.js: Added.
 113
 114 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 115
 116         Reviewed by Adam.
 117
 118         - rebalance new tests
 119
 120         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
 121
 122         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 123
 124         This balances them with the existing tests.
 125
 126         * tests/access-binary-trees.js:
 127         * tests/access-nsieve.js:
 128         * tests/bitops-nsieve-bits.js:
 129         * tests/math-partial-sums.js:
 130         * tests/math-spectral-norm.js:
 131         * tests/string-fasta.js:
 132
 133 2007-10-20  Eric Seidel  <eric@webkit.org>
 134
 135         Reviewed by darin.
 136
 137         Add improved argument handling to sunspider, including
 138         --runs=<number>, --tests=<pattern, --shark, and --help
 139         Also re-factor code into subroutines
 140
 141         * sunspider: improved argument handling
 142
 143 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 144
 145         Reviewed by Mark.
 146
 147         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
 148
 149         * TODO:
 150         * tests/LIST:
 151         * tests/access-binary-trees.js: Added.
 152         * tests/access-nsieve.js: Added.
 153         * tests/bitops-nsieve-bits.js: Added.
 154         * tests/math-partial-sums.js: Added.
 155         * tests/math-spectral-norm.js: Added.
 156         * tests/string-fasta.js: Added.
 157
 158 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 159
 160         Reviewed by Darin.
 161
 162         * TODO: Updated to note areas that are now well-covered.
 163
 164         - Change the tests and the driver to avoid leaving large object
 165         graphs hanging around, since that throws off the subsequent tests.
 166
 167         * resources/sunspider-standalone-driver.js:
 168         * tests/3d-cube.js:
 169         * tests/3d-morph.js:
 170         * tests/3d-raytrace.js:
 171         * tests/bitops-bitwise-and.js:
 172         * tests/string-base64.js:
 173         * tests/string-tagcloud.js:
 174
 175 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 176
 177         Reviewed by Nikolas.
 178
 179         - Rebalance test complexity.
 180
 181         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
 182
 183         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 184
 185         However, for some tests, IE7 was a huge outlier compared to other
 186         major browsers. For those tests, I used Firefox 2.0.0.8 on the
 187         same OS and hardware as a normalization basis.
 188
 189         * tests/3d-morph.js:
 190         * tests/3d-raytrace.js:
 191         * tests/bitops-3bit-bits-in-byte.js:
 192         * tests/bitops-bits-in-byte.js:
 193         * tests/bitops-bitwise-and.js:
 194         * tests/math-cordic.js:
 195         * tests/string-base64.js:
 196         * tests/string-tagcloud.js:
 197         * tests/string-unpack-code.js:
 198
 199 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 200
 201         Reviewed by Oliver.
 202
 203         - Fix the browser-hosted driver to be compatible with Safari 2
 204
 205         * hosted/sunspider-driver.html:
 206
 207 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 208
 209         Rubber stamped by Hyatt.
 210
 211         - Add BSD license notice to cordic test, the original author said to do so.
 212
 213         * tests/math-cordic.js:
 214
 215 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 216
 217         Reviewed by Eric.
 218
 219         - Support Windows and Windows browsers.
 220
 221         * hosted/sunspider-results.html: Display results correctly in IE.
 222         * sunspider: tweaks to work on Windows, and with spaces in the path.
 223
 224 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 225
 226         Rubber stamped by Oliver.
 227
 228         * tests/string-tagcloud.js: Remove stray print()
 229
 230 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
 231
 232         Reviewed by Oliver.
 233
 234         Add some string test cases.
 235
 236         * tests/string-base64.js: Added. Does base64 encoding and decoding.
 237         * tests/string-tagcloud.js: Added. Parses a JSON string and
 238         generates tagcloud HTML markup based on the data.
 239         * tests/string-unpack-code.js: Added. Unpack compressed versions of
 240         four of the most popular JavaScript libraries.
 241         * tests/LIST: Added the new tests.
 242
 243 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
 244
 245         Reviewed by Anders.
 246
 247         * TODO: Updated with info on how to do the stats properly.
 248
 249         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
 250         avoid overly slowing down the benchmark.
 251
 252         Added some 3D-related test cases.
 253
 254         * tests/3d-cube.js: Added.
 255         * tests/3d-morph.js: Added.
 256         * tests/3d-raytrace.js: Added.
 257         * tests/LIST:
 258
 259 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
 260
 261         Reviewed by Darin.
 262
 263         - New JavaScript benchmark
 264         http://bugs.webkit.org/show_bug.cgi?id=15515
 265
 266         A start on a new JavaScript benchmark, with standalone and
 267         browser-hosted drivers.
 268
 269         The test content is still incomplete and the drivers could use
 270         some more features but there is enough here to make a good start.
 271
 272         The drivers compute a 95% confidence interval on the mean for the
 273         whole test, each category, and each individual test to make it
 274         easier to tell whether differences are statistically
 275         significant. The confidence interval can be narrowed by running
 276         the test more times.
 277
 278         * TODO: Added. Remaining things that need to be done.
 279         * sunspider: Added. Perl script that acts as the standalone test driver.
 280         * resources/sunspider-standalone-driver.js: Added. JavaScript part
 281         of standalone test driver.
 282         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
 283         analysis code.
 284         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
 285         * make-hosted: Added. Script to generate browser-hosted tests.
 286         * hosted/sunspider.html: Added. Start page for browser-hosted test.
 287         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
 288         * hosted/sunspider-results.html: Added. Results page for browser-hosted
 289         tests.
 290         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
 291         tests.
 292         * tests/LIST: Added. List of tests to use.
 293         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
 294         * tests/bitops-bits-in-byte.js: Added. Ditto.
 295         * tests/bitops-bitwise-and.js: Added. Ditto.
 296         * tests/math-cordic.js: Added. Ditto.
 297