SunSpider/ChangeLog

   1 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
   2
   3         Reviewed by Darin.
   4
   5         http://bugs.webkit.org/show_bug.cgi?id=15925
   6         SunSpider should check for Shark being installed
   7
   8         * sunspider: Explicitly test that shark command line tool is installed.
   9
  10 2007-11-04  Darin Adler  <darin@apple.com>
  11
  12         * hosted/sunspider.html: Tweaked formatting a bit.
  13
  14 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
  15
  16         Reviewed by Darin.
  17
  18         - Add compare mode and a description of the benchmark to the hosted version.
  19
  20         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
  21         * hosted/sunspider.html: Added intro text.
  22         * make-hosted: Copy new file.
  23         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
  24         a function.
  25         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
  26         expects.
  27         * sunspider-compare-results: Use sunspider-standalone-compare.js.
  28
  29 2007-11-02  Darin Adler  <darin@apple.com>
  30
  31         * sunspider: Changed "--base" to "--set-baseline".
  32
  33 2007-11-02  Darin Adler  <darin@apple.com>
  34
  35         Reviewed by Maciej.
  36
  37         * sunspider: Added a "--base" option that records the current test as a baseline.
  38         Moved the code to find the newest .mshark file into a function (since I needed to use
  39         it in the other script and I prefer to copy/paste a function).
  40
  41         * sunspider-compare-results: Added logic to compare the baseline with the last
  42         run when you don't pass any parameters.
  43
  44 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
  45
  46         Reviewed by Eric.
  47
  48         - Added list of ECMAScript3 features that SunSpider does not cover currently.
  49
  50         * UNCOVERED: Added.
  51
  52 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
  53
  54         Reviewed by Oliver.
  55
  56         - Give better error messages when sunspider-compare-results fails
  57
  58         * sunspider-compare-results:
  59
  60 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
  61
  62         Reviewed by Darin.
  63
  64         - fix Date tests to work in command-line SpiderMonkey
  65
  66         * tests/date-format-tofte.js:
  67         * tests/date-format-xparb.js:
  68
  69 2007-10-29  Darin Adler  <darin@apple.com>
  70
  71         Reviewed by Maciej.
  72
  73         * sunspider: Added --shark20 option, to run Shark at its highest sample
  74         resolution instead of its default.
  75
  76 2007-10-28  Adam Roben  <aroben@apple.com>
  77
  78         Fix SunSpider on Windows
  79
  80         Reviewed by Eric.
  81
  82         * sunspider: Don't use colons in filenames.
  83
  84 2007-10-28  Darin Adler  <darin@apple.com>
  85
  86         Reviewed by Adam.
  87
  88         * resources/sunspider-analyze-results.js: Tweak the output format a little.
  89         Change so that when there's only one run we don't write out confidence
  90         intervals at all rather than writing "NaN" over and over again.
  91
  92         * sunspider: Use the time and date as a suffix on the results file, that way
  93         all the results are left behind in the tmp directory. This will make it easy
  94         for us to add features that compare past results in the future. Also rename
  95         the .mshark files using the same scheme.
  96
  97         * sunspider-compare-results: Relax the parsing rules so we can parse the
  98         results file exactly as it's emitted from sunspider.
  99
 100 2007-10-25  Adam Roben  <aroben@apple.com>
 101
 102         Strip carriage returns from results
 103
 104         Carriage returns were screwing up sunspider-compare-results on
 105         Windows.
 106
 107         Rubberstamped by Oliver.
 108
 109         * sunspider:
 110
 111 2007-10-24  Darin Adler  <darin@apple.com>
 112
 113         Reviewed by Eric.
 114
 115         * sunspider: Open the profile in shark after completing a run.
 116         * sunspider-compare-results: Tiny perl tweaks.
 117
 118 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 119
 120         Reviewed by Darin.
 121
 122         - Don't hardcode my path to testkjs
 123
 124         * sunspider-compare-results:
 125
 126 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 127
 128         Reviewed by Darin.
 129
 130         - Add a compare script to compare results, and improve formatting of standard results a bit.
 131
 132         * resources/sunspider-analyze-results.js:
 133         * resources/sunspider-compare-results.js: Added.
 134         * sunspider-compare-results: Added.
 135
 136 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 137
 138         Reviewed by Darin.
 139
 140         * resources/sunspider-analyze-results.js: Do the error range
 141         properly, using the t-distribution instead of 1.96 (which was
 142         based on the normal distribution).
 143
 144         * sunspider: Print results in a way that will be friendlier to the
 145         soon-to-come compare mode.
 146
 147 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
 148
 149         Reviewed by Review.
 150
 151         - add more tests (probably the final set for now)
 152
 153         Already balanced. Added date, regexp, control flow, and a few
 154         more string and object/array access tests.
 155
 156         * tests/LIST:
 157         * tests/access-fannkuch.js: Added.
 158         * tests/access-nbody.js: Added.
 159         * tests/controlflow-recursive.js: Added.
 160         * tests/date-format-tofte.js: Added.
 161         * tests/date-format-xparb.js: Added.
 162         * tests/regexp-dna.js: Added.
 163         * tests/string-validate-input.js: Added.
 164
 165 2007-10-21  Eric Seidel  <eric@webkit.org>
 166
 167         Reviewed by mjs.
 168
 169         * sunspider: make --shark only sample from testkjs
 170
 171 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 172
 173         Rubber stamped by Eric.
 174
 175         - added crypto tests (already balanced)
 176
 177         * tests/LIST:
 178         * tests/crypto-aes.js: Added.
 179         * tests/crypto-md5.js: Added.
 180         * tests/crypto-sha1.js: Added.
 181
 182 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
 183
 184         Reviewed by Adam.
 185
 186         - rebalance new tests
 187
 188         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
 189
 190         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 191
 192         This balances them with the existing tests.
 193
 194         * tests/access-binary-trees.js:
 195         * tests/access-nsieve.js:
 196         * tests/bitops-nsieve-bits.js:
 197         * tests/math-partial-sums.js:
 198         * tests/math-spectral-norm.js:
 199         * tests/string-fasta.js:
 200
 201 2007-10-20  Eric Seidel  <eric@webkit.org>
 202
 203         Reviewed by darin.
 204
 205         Add improved argument handling to sunspider, including
 206         --runs=<number>, --tests=<pattern, --shark, and --help
 207         Also re-factor code into subroutines
 208
 209         * sunspider: improved argument handling
 210
 211 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 212
 213         Reviewed by Mark.
 214
 215         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
 216
 217         * TODO:
 218         * tests/LIST:
 219         * tests/access-binary-trees.js: Added.
 220         * tests/access-nsieve.js: Added.
 221         * tests/bitops-nsieve-bits.js: Added.
 222         * tests/math-partial-sums.js: Added.
 223         * tests/math-spectral-norm.js: Added.
 224         * tests/string-fasta.js: Added.
 225
 226 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 227
 228         Reviewed by Darin.
 229
 230         * TODO: Updated to note areas that are now well-covered.
 231
 232         - Change the tests and the driver to avoid leaving large object
 233         graphs hanging around, since that throws off the subsequent tests.
 234
 235         * resources/sunspider-standalone-driver.js:
 236         * tests/3d-cube.js:
 237         * tests/3d-morph.js:
 238         * tests/3d-raytrace.js:
 239         * tests/bitops-bitwise-and.js:
 240         * tests/string-base64.js:
 241         * tests/string-tagcloud.js:
 242
 243 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 244
 245         Reviewed by Nikolas.
 246
 247         - Rebalance test complexity.
 248
 249         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
 250
 251         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
 252
 253         However, for some tests, IE7 was a huge outlier compared to other
 254         major browsers. For those tests, I used Firefox 2.0.0.8 on the
 255         same OS and hardware as a normalization basis.
 256
 257         * tests/3d-morph.js:
 258         * tests/3d-raytrace.js:
 259         * tests/bitops-3bit-bits-in-byte.js:
 260         * tests/bitops-bits-in-byte.js:
 261         * tests/bitops-bitwise-and.js:
 262         * tests/math-cordic.js:
 263         * tests/string-base64.js:
 264         * tests/string-tagcloud.js:
 265         * tests/string-unpack-code.js:
 266
 267 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 268
 269         Reviewed by Oliver.
 270
 271         - Fix the browser-hosted driver to be compatible with Safari 2
 272
 273         * hosted/sunspider-driver.html:
 274
 275 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
 276
 277         Rubber stamped by Hyatt.
 278
 279         - Add BSD license notice to cordic test, the original author said to do so.
 280
 281         * tests/math-cordic.js:
 282
 283 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 284
 285         Reviewed by Eric.
 286
 287         - Support Windows and Windows browsers.
 288
 289         * hosted/sunspider-results.html: Display results correctly in IE.
 290         * sunspider: tweaks to work on Windows, and with spaces in the path.
 291
 292 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
 293
 294         Rubber stamped by Oliver.
 295
 296         * tests/string-tagcloud.js: Remove stray print()
 297
 298 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
 299
 300         Reviewed by Oliver.
 301
 302         Add some string test cases.
 303
 304         * tests/string-base64.js: Added. Does base64 encoding and decoding.
 305         * tests/string-tagcloud.js: Added. Parses a JSON string and
 306         generates tagcloud HTML markup based on the data.
 307         * tests/string-unpack-code.js: Added. Unpack compressed versions of
 308         four of the most popular JavaScript libraries.
 309         * tests/LIST: Added the new tests.
 310
 311 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
 312
 313         Reviewed by Anders.
 314
 315         * TODO: Updated with info on how to do the stats properly.
 316
 317         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
 318         avoid overly slowing down the benchmark.
 319
 320         Added some 3D-related test cases.
 321
 322         * tests/3d-cube.js: Added.
 323         * tests/3d-morph.js: Added.
 324         * tests/3d-raytrace.js: Added.
 325         * tests/LIST:
 326
 327 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
 328
 329         Reviewed by Darin.
 330
 331         - New JavaScript benchmark
 332         http://bugs.webkit.org/show_bug.cgi?id=15515
 333
 334         A start on a new JavaScript benchmark, with standalone and
 335         browser-hosted drivers.
 336
 337         The test content is still incomplete and the drivers could use
 338         some more features but there is enough here to make a good start.
 339
 340         The drivers compute a 95% confidence interval on the mean for the
 341         whole test, each category, and each individual test to make it
 342         easier to tell whether differences are statistically
 343         significant. The confidence interval can be narrowed by running
 344         the test more times.
 345
 346         * TODO: Added. Remaining things that need to be done.
 347         * sunspider: Added. Perl script that acts as the standalone test driver.
 348         * resources/sunspider-standalone-driver.js: Added. JavaScript part
 349         of standalone test driver.
 350         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
 351         analysis code.
 352         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
 353         * make-hosted: Added. Script to generate browser-hosted tests.
 354         * hosted/sunspider.html: Added. Start page for browser-hosted test.
 355         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
 356         * hosted/sunspider-results.html: Added. Results page for browser-hosted
 357         tests.
 358         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
 359         tests.
 360         * tests/LIST: Added. List of tests to use.
 361         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
 362         * tests/bitops-bits-in-byte.js: Added. Ditto.
 363         * tests/bitops-bitwise-and.js: Added. Ditto.
 364         * tests/math-cordic.js: Added. Ditto.
 365