1 2008-09-02 Cameron Zwarich <cwzwarich@uwaterloo.ca>
5 Remove a local change that was accidentally committed in r36034.
7 * resources/sunspider-standalone-driver.js:
9 2008-09-02 Cameron Zwarich <cwzwarich@uwaterloo.ca>
11 Rubber-stamped by Oliver Hunt.
13 Make SunSpider work with JavaScript shells that don't take filenames
14 after a '-f' argument.
18 2008-06-07 Cameron Zwarich <cwzwarich@uwaterloo.ca>
22 Remove the --squirrelfish option from SunSpider, as it is no longer needed.
25 * tests/LIST-SQUIRRELFISH: Removed.
27 2008-06-03 Alexey Proskuryakov <ap@webkit.org>
29 Rubber-stamped by Mark Rowe.
31 Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
35 2008-04-15 Maciej Stachowiak <mjs@apple.com>
37 Rubber stamped by Oliver.
39 - add newly working tests to squirrelfish list
41 * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
43 2008-04-14 Maciej Stachowiak <mjs@apple.com>
45 Rubber stamped by Oliver.
47 - move ubench tests properly.
49 * tests/ubench: Added.
50 * tests/ubench/function-closure.js: Added.
51 * tests/ubench/function-empty.js: Added.
52 * tests/ubench/function-missing-args.js: Added.
53 * tests/ubench/function-sum.js: Added.
54 * tests/ubench/loop-empty-resolve.js: Added.
55 * tests/ubench/loop-empty.js: Added.
56 * tests/ubench/loop-sum.js: Added.
58 2008-04-14 Maciej Stachowiak <mjs@apple.com>
60 Rubber stamped by Oliver.
62 * sunspider: Don't force --runs=1 in squirrelfish mode any more.
64 2008-04-14 Maciej Stachowiak <mjs@apple.com>
68 - make --squirrelfish mode use the real harness, but a reduced set of tests
70 Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
73 * tests/LIST-SQUIRRELFISH:
74 * tests/LIST-UBENCH: Added.
75 * tests/squirrelfish: Moved to tests/ubench
77 2008-04-14 Geoffrey Garen <ggaren@apple.com>
79 Reviewed by Maciej Stachowiak.
81 Removed an unused test.
83 * tests/squirrelfish/loop-resolve.js: Removed.
85 2008-04-10 Maciej Stachowiak <mjs@apple.com>
89 Add newly runnable tests to --squirrelfish mode.
91 * tests/LIST-SQUIRRELFISH:
93 2008-04-06 Geoffrey Garen <ggaren@apple.com>
95 Reviewed by Maciej Stachowiak.
97 A little more love for --squirrelfish mode:
99 Fixed a misplaced sort. Results properly sort now.
101 Added a test list just for squirrelfish, and updated it to include
102 all currently passing SunSpider tests.
104 Fixed the pruning regexp to match 3d-morph.js.
108 2008-03-26 Geoffrey Garen <ggaren@apple.com>
110 Reviewed by Oliver Hunt.
112 --squirrelfish mode: pared down tests for squirrelfish to chew on.
116 * tests/squirrelfish: Added.
117 * tests/squirrelfish/function-closure.js: Added.
118 * tests/squirrelfish/function-empty.js: Added.
119 * tests/squirrelfish/function-missing-args.js: Added.
120 * tests/squirrelfish/function-sum.js: Added.
121 * tests/squirrelfish/loop-empty-resolve.js: Added.
122 * tests/squirrelfish/loop-empty.js: Added.
123 * tests/squirrelfish/loop-resolve.js: Added.
124 * tests/squirrelfish/loop-sum.js: Added.
126 2008-03-21 Alexey Proskuryakov <ap@webkit.org>
130 * sunspider: Pause update daemon on Darwin to make results reliable
131 again (workaround for <rdar://problem/5811127>).
133 2008-02-07 Eric Seidel <eric@webkit.org>
137 * sunspider: add an error message when ./tests/LIST is not found
139 2007-12-29 Alp Toker <alp@atoker.com>
143 * hosted/sunspider.html:
145 2007-12-29 Alp Toker <alp@atoker.com>
149 Provide the current URL in the SunSpider results page. Useful for
150 copying benchmark results out of browsers without a location bar or
151 where the location bar can't handle long URLs.
153 * hosted/sunspider-results.html:
155 2007-12-20 Eric Seidel <eric@webkit.org>
159 * sunspider: increase default number of runs for more accurate numbers
161 2007-12-18 Mark Rowe <mrowe@apple.com>
165 * hosted/sunspider.html:
167 2007-12-18 Maciej Stachowiak <mjs@apple.com>
171 - give an extra digit of precision for small differences.
173 * resources/sunspider-compare-results.js:
175 2007-12-17 Maciej Stachowiak <mjs@apple.com>
179 - work around IE layout bugs
181 * hosted/sunspider.css:
183 2007-12-17 Maciej Stachowiak <mjs@apple.com>
187 - fix compare feature
189 * hosted/sunspider-results.html:
191 2007-12-17 Maciej Stachowiak <mjs@apple.com>
195 - Change the format of the output URL to be shorter, to meet IE constraints
197 * hosted/sunspider-driver.html:
198 * hosted/sunspider-results.html:
200 2007-12-13 Maciej Stachowiak <mjs@apple.com>
204 - Add some explanatory test and a dumb logo to SunSpider web hosted version
206 * hosted/sunspider-driver.html:
207 * hosted/sunspider-results.html:
208 * hosted/sunspider.css:
209 * hosted/sunspider.html:
211 2007-12-13 Maciej Stachowiak <mjs@apple.com>
215 - Make the web-hosted version of SunSpider look a little bit less drab.
217 * hosted/sunspider-driver.html:
218 * hosted/sunspider-results.html:
219 * hosted/sunspider.css: Added.
220 * hosted/sunspider.html:
221 * resources/TEMPLATE.html:
223 2007-12-12 Maciej Stachowiak <mjs@apple.com>
227 - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
229 A sample of what new-style results would look like:
231 TEST COMPARISON FROM TO DETAILS
233 ** TOTAL **: 1.41x as fast 6613.6ms +/- 0.1% 4703.0ms +/- 0.1% significant
235 3bit-bits-in-byte: *1.36x as slow* 88.8ms +/- 0.6% 121.1ms +/- 0.6% significant
237 * resources/sunspider-compare-results.js:
239 2007-11-14 Eric Seidel <eric@webkit.org>
243 * sunspider: add --shark-cache for L2 Cache Miss profiling
245 2007-11-09 Alexey Proskuryakov <ap@webkit.org>
249 http://bugs.webkit.org/show_bug.cgi?id=15925
250 SunSpider should check for Shark being installed
252 * sunspider: Explicitly test that shark command line tool is installed.
254 2007-11-04 Darin Adler <darin@apple.com>
256 * hosted/sunspider.html: Tweaked formatting a bit.
258 2007-11-03 Maciej Stachowiak <mjs@apple.com>
262 - Add compare mode and a description of the benchmark to the hosted version.
264 * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
265 * hosted/sunspider.html: Added intro text.
266 * make-hosted: Copy new file.
267 * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
269 * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
271 * sunspider-compare-results: Use sunspider-standalone-compare.js.
273 2007-11-02 Darin Adler <darin@apple.com>
275 * sunspider: Changed "--base" to "--set-baseline".
277 2007-11-02 Darin Adler <darin@apple.com>
281 * sunspider: Added a "--base" option that records the current test as a baseline.
282 Moved the code to find the newest .mshark file into a function (since I needed to use
283 it in the other script and I prefer to copy/paste a function).
285 * sunspider-compare-results: Added logic to compare the baseline with the last
286 run when you don't pass any parameters.
288 2007-10-29 Maciej Stachowiak <mjs@apple.com>
292 - Added list of ECMAScript3 features that SunSpider does not cover currently.
296 2007-10-29 Maciej Stachowiak <mjs@apple.com>
300 - Give better error messages when sunspider-compare-results fails
302 * sunspider-compare-results:
304 2007-10-29 Maciej Stachowiak <mjs@apple.com>
308 - fix Date tests to work in command-line SpiderMonkey
310 * tests/date-format-tofte.js:
311 * tests/date-format-xparb.js:
313 2007-10-29 Darin Adler <darin@apple.com>
317 * sunspider: Added --shark20 option, to run Shark at its highest sample
318 resolution instead of its default.
320 2007-10-28 Adam Roben <aroben@apple.com>
322 Fix SunSpider on Windows
326 * sunspider: Don't use colons in filenames.
328 2007-10-28 Darin Adler <darin@apple.com>
332 * resources/sunspider-analyze-results.js: Tweak the output format a little.
333 Change so that when there's only one run we don't write out confidence
334 intervals at all rather than writing "NaN" over and over again.
336 * sunspider: Use the time and date as a suffix on the results file, that way
337 all the results are left behind in the tmp directory. This will make it easy
338 for us to add features that compare past results in the future. Also rename
339 the .mshark files using the same scheme.
341 * sunspider-compare-results: Relax the parsing rules so we can parse the
342 results file exactly as it's emitted from sunspider.
344 2007-10-25 Adam Roben <aroben@apple.com>
346 Strip carriage returns from results
348 Carriage returns were screwing up sunspider-compare-results on
351 Rubberstamped by Oliver.
355 2007-10-24 Darin Adler <darin@apple.com>
359 * sunspider: Open the profile in shark after completing a run.
360 * sunspider-compare-results: Tiny perl tweaks.
362 2007-10-23 Maciej Stachowiak <mjs@apple.com>
366 - Don't hardcode my path to testkjs
368 * sunspider-compare-results:
370 2007-10-23 Maciej Stachowiak <mjs@apple.com>
374 - Add a compare script to compare results, and improve formatting of standard results a bit.
376 * resources/sunspider-analyze-results.js:
377 * resources/sunspider-compare-results.js: Added.
378 * sunspider-compare-results: Added.
380 2007-10-23 Maciej Stachowiak <mjs@apple.com>
384 * resources/sunspider-analyze-results.js: Do the error range
385 properly, using the t-distribution instead of 1.96 (which was
386 based on the normal distribution).
388 * sunspider: Print results in a way that will be friendlier to the
389 soon-to-come compare mode.
391 2007-10-22 Maciej Stachowiak <mjs@apple.com>
395 - add more tests (probably the final set for now)
397 Already balanced. Added date, regexp, control flow, and a few
398 more string and object/array access tests.
401 * tests/access-fannkuch.js: Added.
402 * tests/access-nbody.js: Added.
403 * tests/controlflow-recursive.js: Added.
404 * tests/date-format-tofte.js: Added.
405 * tests/date-format-xparb.js: Added.
406 * tests/regexp-dna.js: Added.
407 * tests/string-validate-input.js: Added.
409 2007-10-21 Eric Seidel <eric@webkit.org>
413 * sunspider: make --shark only sample from testkjs
415 2007-10-21 Maciej Stachowiak <mjs@apple.com>
417 Rubber stamped by Eric.
419 - added crypto tests (already balanced)
422 * tests/crypto-aes.js: Added.
423 * tests/crypto-md5.js: Added.
424 * tests/crypto-sha1.js: Added.
426 2007-10-21 Maciej Stachowiak <mjs@apple.com>
430 - rebalance new tests
432 Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
434 Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
436 This balances them with the existing tests.
438 * tests/access-binary-trees.js:
439 * tests/access-nsieve.js:
440 * tests/bitops-nsieve-bits.js:
441 * tests/math-partial-sums.js:
442 * tests/math-spectral-norm.js:
443 * tests/string-fasta.js:
445 2007-10-20 Eric Seidel <eric@webkit.org>
449 Add improved argument handling to sunspider, including
450 --runs=<number>, --tests=<pattern, --shark, and --help
451 Also re-factor code into subroutines
453 * sunspider: improved argument handling
455 2007-10-20 Maciej Stachowiak <mjs@apple.com>
459 - Add more new tests, mostly from the computer language shootout. Not normalized yet.
463 * tests/access-binary-trees.js: Added.
464 * tests/access-nsieve.js: Added.
465 * tests/bitops-nsieve-bits.js: Added.
466 * tests/math-partial-sums.js: Added.
467 * tests/math-spectral-norm.js: Added.
468 * tests/string-fasta.js: Added.
470 2007-10-20 Maciej Stachowiak <mjs@apple.com>
474 * TODO: Updated to note areas that are now well-covered.
476 - Change the tests and the driver to avoid leaving large object
477 graphs hanging around, since that throws off the subsequent tests.
479 * resources/sunspider-standalone-driver.js:
482 * tests/3d-raytrace.js:
483 * tests/bitops-bitwise-and.js:
484 * tests/string-base64.js:
485 * tests/string-tagcloud.js:
487 2007-10-20 Maciej Stachowiak <mjs@apple.com>
491 - Rebalance test complexity.
493 Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
495 Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
497 However, for some tests, IE7 was a huge outlier compared to other
498 major browsers. For those tests, I used Firefox 2.0.0.8 on the
499 same OS and hardware as a normalization basis.
502 * tests/3d-raytrace.js:
503 * tests/bitops-3bit-bits-in-byte.js:
504 * tests/bitops-bits-in-byte.js:
505 * tests/bitops-bitwise-and.js:
506 * tests/math-cordic.js:
507 * tests/string-base64.js:
508 * tests/string-tagcloud.js:
509 * tests/string-unpack-code.js:
511 2007-10-20 Maciej Stachowiak <mjs@apple.com>
515 - Fix the browser-hosted driver to be compatible with Safari 2
517 * hosted/sunspider-driver.html:
519 2007-10-20 Maciej Stachowiak <mjs@apple.com>
521 Rubber stamped by Hyatt.
523 - Add BSD license notice to cordic test, the original author said to do so.
525 * tests/math-cordic.js:
527 2007-10-18 Maciej Stachowiak <mjs@apple.com>
531 - Support Windows and Windows browsers.
533 * hosted/sunspider-results.html: Display results correctly in IE.
534 * sunspider: tweaks to work on Windows, and with spaces in the path.
536 2007-10-18 Maciej Stachowiak <mjs@apple.com>
538 Rubber stamped by Oliver.
540 * tests/string-tagcloud.js: Remove stray print()
542 2007-10-17 Maciej Stachowiak <mjs@apple.com>
546 Add some string test cases.
548 * tests/string-base64.js: Added. Does base64 encoding and decoding.
549 * tests/string-tagcloud.js: Added. Parses a JSON string and
550 generates tagcloud HTML markup based on the data.
551 * tests/string-unpack-code.js: Added. Unpack compressed versions of
552 four of the most popular JavaScript libraries.
553 * tests/LIST: Added the new tests.
555 2007-10-16 Maciej Stachowiak <mjs@apple.com>
559 * TODO: Updated with info on how to do the stats properly.
561 * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
562 avoid overly slowing down the benchmark.
564 Added some 3D-related test cases.
566 * tests/3d-cube.js: Added.
567 * tests/3d-morph.js: Added.
568 * tests/3d-raytrace.js: Added.
571 2007-10-14 Maciej Stachowiak <mjs@apple.com>
575 - New JavaScript benchmark
576 http://bugs.webkit.org/show_bug.cgi?id=15515
578 A start on a new JavaScript benchmark, with standalone and
579 browser-hosted drivers.
581 The test content is still incomplete and the drivers could use
582 some more features but there is enough here to make a good start.
584 The drivers compute a 95% confidence interval on the mean for the
585 whole test, each category, and each individual test to make it
586 easier to tell whether differences are statistically
587 significant. The confidence interval can be narrowed by running
590 * TODO: Added. Remaining things that need to be done.
591 * sunspider: Added. Perl script that acts as the standalone test driver.
592 * resources/sunspider-standalone-driver.js: Added. JavaScript part
593 of standalone test driver.
594 * resources/sunspider-analyze-results.js: Added. JavaScript statistical
596 * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
597 * make-hosted: Added. Script to generate browser-hosted tests.
598 * hosted/sunspider.html: Added. Start page for browser-hosted test.
599 * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
600 * hosted/sunspider-results.html: Added. Results page for browser-hosted
602 * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
604 * tests/LIST: Added. List of tests to use.
605 * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
606 * tests/bitops-bits-in-byte.js: Added. Ditto.
607 * tests/bitops-bitwise-and.js: Added. Ditto.
608 * tests/math-cordic.js: Added. Ditto.