1 2007-11-09 Alexey Proskuryakov <ap@webkit.org>
5 http://bugs.webkit.org/show_bug.cgi?id=15925
6 SunSpider should check for Shark being installed
8 * sunspider: Explicitly test that shark command line tool is installed.
10 2007-11-04 Darin Adler <darin@apple.com>
12 * hosted/sunspider.html: Tweaked formatting a bit.
14 2007-11-03 Maciej Stachowiak <mjs@apple.com>
18 - Add compare mode and a description of the benchmark to the hosted version.
20 * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
21 * hosted/sunspider.html: Added intro text.
22 * make-hosted: Copy new file.
23 * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
25 * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
27 * sunspider-compare-results: Use sunspider-standalone-compare.js.
29 2007-11-02 Darin Adler <darin@apple.com>
31 * sunspider: Changed "--base" to "--set-baseline".
33 2007-11-02 Darin Adler <darin@apple.com>
37 * sunspider: Added a "--base" option that records the current test as a baseline.
38 Moved the code to find the newest .mshark file into a function (since I needed to use
39 it in the other script and I prefer to copy/paste a function).
41 * sunspider-compare-results: Added logic to compare the baseline with the last
42 run when you don't pass any parameters.
44 2007-10-29 Maciej Stachowiak <mjs@apple.com>
48 - Added list of ECMAScript3 features that SunSpider does not cover currently.
52 2007-10-29 Maciej Stachowiak <mjs@apple.com>
56 - Give better error messages when sunspider-compare-results fails
58 * sunspider-compare-results:
60 2007-10-29 Maciej Stachowiak <mjs@apple.com>
64 - fix Date tests to work in command-line SpiderMonkey
66 * tests/date-format-tofte.js:
67 * tests/date-format-xparb.js:
69 2007-10-29 Darin Adler <darin@apple.com>
73 * sunspider: Added --shark20 option, to run Shark at its highest sample
74 resolution instead of its default.
76 2007-10-28 Adam Roben <aroben@apple.com>
78 Fix SunSpider on Windows
82 * sunspider: Don't use colons in filenames.
84 2007-10-28 Darin Adler <darin@apple.com>
88 * resources/sunspider-analyze-results.js: Tweak the output format a little.
89 Change so that when there's only one run we don't write out confidence
90 intervals at all rather than writing "NaN" over and over again.
92 * sunspider: Use the time and date as a suffix on the results file, that way
93 all the results are left behind in the tmp directory. This will make it easy
94 for us to add features that compare past results in the future. Also rename
95 the .mshark files using the same scheme.
97 * sunspider-compare-results: Relax the parsing rules so we can parse the
98 results file exactly as it's emitted from sunspider.
100 2007-10-25 Adam Roben <aroben@apple.com>
102 Strip carriage returns from results
104 Carriage returns were screwing up sunspider-compare-results on
107 Rubberstamped by Oliver.
111 2007-10-24 Darin Adler <darin@apple.com>
115 * sunspider: Open the profile in shark after completing a run.
116 * sunspider-compare-results: Tiny perl tweaks.
118 2007-10-23 Maciej Stachowiak <mjs@apple.com>
122 - Don't hardcode my path to testkjs
124 * sunspider-compare-results:
126 2007-10-23 Maciej Stachowiak <mjs@apple.com>
130 - Add a compare script to compare results, and improve formatting of standard results a bit.
132 * resources/sunspider-analyze-results.js:
133 * resources/sunspider-compare-results.js: Added.
134 * sunspider-compare-results: Added.
136 2007-10-23 Maciej Stachowiak <mjs@apple.com>
140 * resources/sunspider-analyze-results.js: Do the error range
141 properly, using the t-distribution instead of 1.96 (which was
142 based on the normal distribution).
144 * sunspider: Print results in a way that will be friendlier to the
145 soon-to-come compare mode.
147 2007-10-22 Maciej Stachowiak <mjs@apple.com>
151 - add more tests (probably the final set for now)
153 Already balanced. Added date, regexp, control flow, and a few
154 more string and object/array access tests.
157 * tests/access-fannkuch.js: Added.
158 * tests/access-nbody.js: Added.
159 * tests/controlflow-recursive.js: Added.
160 * tests/date-format-tofte.js: Added.
161 * tests/date-format-xparb.js: Added.
162 * tests/regexp-dna.js: Added.
163 * tests/string-validate-input.js: Added.
165 2007-10-21 Eric Seidel <eric@webkit.org>
169 * sunspider: make --shark only sample from testkjs
171 2007-10-21 Maciej Stachowiak <mjs@apple.com>
173 Rubber stamped by Eric.
175 - added crypto tests (already balanced)
178 * tests/crypto-aes.js: Added.
179 * tests/crypto-md5.js: Added.
180 * tests/crypto-sha1.js: Added.
182 2007-10-21 Maciej Stachowiak <mjs@apple.com>
186 - rebalance new tests
188 Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
190 Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
192 This balances them with the existing tests.
194 * tests/access-binary-trees.js:
195 * tests/access-nsieve.js:
196 * tests/bitops-nsieve-bits.js:
197 * tests/math-partial-sums.js:
198 * tests/math-spectral-norm.js:
199 * tests/string-fasta.js:
201 2007-10-20 Eric Seidel <eric@webkit.org>
205 Add improved argument handling to sunspider, including
206 --runs=<number>, --tests=<pattern, --shark, and --help
207 Also re-factor code into subroutines
209 * sunspider: improved argument handling
211 2007-10-20 Maciej Stachowiak <mjs@apple.com>
215 - Add more new tests, mostly from the computer language shootout. Not normalized yet.
219 * tests/access-binary-trees.js: Added.
220 * tests/access-nsieve.js: Added.
221 * tests/bitops-nsieve-bits.js: Added.
222 * tests/math-partial-sums.js: Added.
223 * tests/math-spectral-norm.js: Added.
224 * tests/string-fasta.js: Added.
226 2007-10-20 Maciej Stachowiak <mjs@apple.com>
230 * TODO: Updated to note areas that are now well-covered.
232 - Change the tests and the driver to avoid leaving large object
233 graphs hanging around, since that throws off the subsequent tests.
235 * resources/sunspider-standalone-driver.js:
238 * tests/3d-raytrace.js:
239 * tests/bitops-bitwise-and.js:
240 * tests/string-base64.js:
241 * tests/string-tagcloud.js:
243 2007-10-20 Maciej Stachowiak <mjs@apple.com>
247 - Rebalance test complexity.
249 Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
251 Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
253 However, for some tests, IE7 was a huge outlier compared to other
254 major browsers. For those tests, I used Firefox 2.0.0.8 on the
255 same OS and hardware as a normalization basis.
258 * tests/3d-raytrace.js:
259 * tests/bitops-3bit-bits-in-byte.js:
260 * tests/bitops-bits-in-byte.js:
261 * tests/bitops-bitwise-and.js:
262 * tests/math-cordic.js:
263 * tests/string-base64.js:
264 * tests/string-tagcloud.js:
265 * tests/string-unpack-code.js:
267 2007-10-20 Maciej Stachowiak <mjs@apple.com>
271 - Fix the browser-hosted driver to be compatible with Safari 2
273 * hosted/sunspider-driver.html:
275 2007-10-20 Maciej Stachowiak <mjs@apple.com>
277 Rubber stamped by Hyatt.
279 - Add BSD license notice to cordic test, the original author said to do so.
281 * tests/math-cordic.js:
283 2007-10-18 Maciej Stachowiak <mjs@apple.com>
287 - Support Windows and Windows browsers.
289 * hosted/sunspider-results.html: Display results correctly in IE.
290 * sunspider: tweaks to work on Windows, and with spaces in the path.
292 2007-10-18 Maciej Stachowiak <mjs@apple.com>
294 Rubber stamped by Oliver.
296 * tests/string-tagcloud.js: Remove stray print()
298 2007-10-17 Maciej Stachowiak <mjs@apple.com>
302 Add some string test cases.
304 * tests/string-base64.js: Added. Does base64 encoding and decoding.
305 * tests/string-tagcloud.js: Added. Parses a JSON string and
306 generates tagcloud HTML markup based on the data.
307 * tests/string-unpack-code.js: Added. Unpack compressed versions of
308 four of the most popular JavaScript libraries.
309 * tests/LIST: Added the new tests.
311 2007-10-16 Maciej Stachowiak <mjs@apple.com>
315 * TODO: Updated with info on how to do the stats properly.
317 * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
318 avoid overly slowing down the benchmark.
320 Added some 3D-related test cases.
322 * tests/3d-cube.js: Added.
323 * tests/3d-morph.js: Added.
324 * tests/3d-raytrace.js: Added.
327 2007-10-14 Maciej Stachowiak <mjs@apple.com>
331 - New JavaScript benchmark
332 http://bugs.webkit.org/show_bug.cgi?id=15515
334 A start on a new JavaScript benchmark, with standalone and
335 browser-hosted drivers.
337 The test content is still incomplete and the drivers could use
338 some more features but there is enough here to make a good start.
340 The drivers compute a 95% confidence interval on the mean for the
341 whole test, each category, and each individual test to make it
342 easier to tell whether differences are statistically
343 significant. The confidence interval can be narrowed by running
346 * TODO: Added. Remaining things that need to be done.
347 * sunspider: Added. Perl script that acts as the standalone test driver.
348 * resources/sunspider-standalone-driver.js: Added. JavaScript part
349 of standalone test driver.
350 * resources/sunspider-analyze-results.js: Added. JavaScript statistical
352 * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
353 * make-hosted: Added. Script to generate browser-hosted tests.
354 * hosted/sunspider.html: Added. Start page for browser-hosted test.
355 * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
356 * hosted/sunspider-results.html: Added. Results page for browser-hosted
358 * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
360 * tests/LIST: Added. List of tests to use.
361 * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
362 * tests/bitops-bits-in-byte.js: Added. Ditto.
363 * tests/bitops-bitwise-and.js: Added. Ditto.
364 * tests/math-cordic.js: Added. Ditto.