d795426741b8d13a8ab9a404913b5ed8dfebca61
[WebKit-https.git] / SunSpider / ChangeLog
1 2007-12-18  Mark Rowe  <mrowe@apple.com>
2
3         Fix typo.
4
5         * hosted/sunspider.html:
6
7 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
8
9         Reviewed by Eric.
10         
11         - give an extra digit of precision for small differences.
12
13         * resources/sunspider-compare-results.js:
14
15 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
16
17         Reviewed by Eric.
18         
19         - work around IE layout bugs
20
21         * hosted/sunspider.css:
22
23 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
24
25         Reviewed by Mark.
26         
27         - fix compare feature
28
29         * hosted/sunspider-results.html:
30
31 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
32
33         Reviewed by Mark.
34         
35         - Change the format of the output URL to be shorter, to meet IE constraints
36
37         * hosted/sunspider-driver.html:
38         * hosted/sunspider-results.html:
39
40 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
41
42         Reviewed by Oliver.
43         
44         - Add some explanatory test and a dumb logo to SunSpider web hosted version
45
46         * hosted/sunspider-driver.html:
47         * hosted/sunspider-results.html:
48         * hosted/sunspider.css:
49         * hosted/sunspider.html:
50
51 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
52
53         Reviewed by Oliver.
54         
55         - Make the web-hosted version of SunSpider look a little bit less drab.
56
57         * hosted/sunspider-driver.html:
58         * hosted/sunspider-results.html:
59         * hosted/sunspider.css: Added.
60         * hosted/sunspider.html:
61         * resources/TEMPLATE.html:
62
63 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
64
65         Reviewed by Sam.
66         
67         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
68         
69         A sample of what new-style results would look like:
70         
71         TEST                   COMPARISON            FROM                 TO             DETAILS
72         ...
73         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
74         ...
75             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
76         
77         * resources/sunspider-compare-results.js:
78
79 2007-11-14  Eric Seidel  <eric@webkit.org>
80
81         Reviewed by Sam.
82
83         * sunspider: add --shark-cache for L2 Cache Miss profiling
84
85 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
86
87         Reviewed by Darin.
88
89         http://bugs.webkit.org/show_bug.cgi?id=15925
90         SunSpider should check for Shark being installed
91
92         * sunspider: Explicitly test that shark command line tool is installed.
93
94 2007-11-04  Darin Adler  <darin@apple.com>
95
96         * hosted/sunspider.html: Tweaked formatting a bit.
97
98 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
99
100         Reviewed by Darin.
101
102         - Add compare mode and a description of the benchmark to the hosted version.
103
104         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
105         * hosted/sunspider.html: Added intro text.
106         * make-hosted: Copy new file.
107         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
108         a function.
109         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
110         expects.
111         * sunspider-compare-results: Use sunspider-standalone-compare.js.
112
113 2007-11-02  Darin Adler  <darin@apple.com>
114
115         * sunspider: Changed "--base" to "--set-baseline".
116
117 2007-11-02  Darin Adler  <darin@apple.com>
118
119         Reviewed by Maciej.
120
121         * sunspider: Added a "--base" option that records the current test as a baseline.
122         Moved the code to find the newest .mshark file into a function (since I needed to use
123         it in the other script and I prefer to copy/paste a function).
124
125         * sunspider-compare-results: Added logic to compare the baseline with the last
126         run when you don't pass any parameters.
127
128 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
129
130         Reviewed by Eric.
131         
132         - Added list of ECMAScript3 features that SunSpider does not cover currently.
133
134         * UNCOVERED: Added.
135
136 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
137
138         Reviewed by Oliver.
139         
140         - Give better error messages when sunspider-compare-results fails
141
142         * sunspider-compare-results:
143
144 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
145
146         Reviewed by Darin.
147         
148         - fix Date tests to work in command-line SpiderMonkey
149
150         * tests/date-format-tofte.js:
151         * tests/date-format-xparb.js:
152
153 2007-10-29  Darin Adler  <darin@apple.com>
154
155         Reviewed by Maciej.
156
157         * sunspider: Added --shark20 option, to run Shark at its highest sample
158         resolution instead of its default.
159
160 2007-10-28  Adam Roben  <aroben@apple.com>
161
162         Fix SunSpider on Windows
163
164         Reviewed by Eric.
165
166         * sunspider: Don't use colons in filenames.
167
168 2007-10-28  Darin Adler  <darin@apple.com>
169
170         Reviewed by Adam.
171
172         * resources/sunspider-analyze-results.js: Tweak the output format a little.
173         Change so that when there's only one run we don't write out confidence
174         intervals at all rather than writing "NaN" over and over again.
175
176         * sunspider: Use the time and date as a suffix on the results file, that way
177         all the results are left behind in the tmp directory. This will make it easy
178         for us to add features that compare past results in the future. Also rename
179         the .mshark files using the same scheme.
180
181         * sunspider-compare-results: Relax the parsing rules so we can parse the
182         results file exactly as it's emitted from sunspider.
183
184 2007-10-25  Adam Roben  <aroben@apple.com>
185
186         Strip carriage returns from results
187
188         Carriage returns were screwing up sunspider-compare-results on
189         Windows.
190
191         Rubberstamped by Oliver.
192
193         * sunspider:
194
195 2007-10-24  Darin Adler  <darin@apple.com>
196
197         Reviewed by Eric.
198
199         * sunspider: Open the profile in shark after completing a run.
200         * sunspider-compare-results: Tiny perl tweaks.
201
202 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
203
204         Reviewed by Darin.
205
206         - Don't hardcode my path to testkjs
207         
208         * sunspider-compare-results:
209
210 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
211
212         Reviewed by Darin.
213         
214         - Add a compare script to compare results, and improve formatting of standard results a bit.
215
216         * resources/sunspider-analyze-results.js:
217         * resources/sunspider-compare-results.js: Added.
218         * sunspider-compare-results: Added.
219
220 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
221
222         Reviewed by Darin.
223
224         * resources/sunspider-analyze-results.js: Do the error range
225         properly, using the t-distribution instead of 1.96 (which was
226         based on the normal distribution).
227         
228         * sunspider: Print results in a way that will be friendlier to the
229         soon-to-come compare mode.
230
231 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
232
233         Reviewed by Review.
234         
235         - add more tests (probably the final set for now)
236         
237         Already balanced. Added date, regexp, control flow, and a few
238         more string and object/array access tests.
239         
240         * tests/LIST:
241         * tests/access-fannkuch.js: Added.
242         * tests/access-nbody.js: Added.
243         * tests/controlflow-recursive.js: Added.
244         * tests/date-format-tofte.js: Added.
245         * tests/date-format-xparb.js: Added.
246         * tests/regexp-dna.js: Added.
247         * tests/string-validate-input.js: Added.
248
249 2007-10-21  Eric Seidel  <eric@webkit.org>
250
251         Reviewed by mjs.
252
253         * sunspider: make --shark only sample from testkjs
254
255 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
256
257         Rubber stamped by Eric.
258         
259         - added crypto tests (already balanced)
260
261         * tests/LIST:
262         * tests/crypto-aes.js: Added.
263         * tests/crypto-md5.js: Added.
264         * tests/crypto-sha1.js: Added.
265
266 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
267
268         Reviewed by Adam.
269         
270         - rebalance new tests
271         
272         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
273         
274         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
275
276         This balances them with the existing tests.
277
278         * tests/access-binary-trees.js:
279         * tests/access-nsieve.js:
280         * tests/bitops-nsieve-bits.js:
281         * tests/math-partial-sums.js:
282         * tests/math-spectral-norm.js:
283         * tests/string-fasta.js:
284
285 2007-10-20  Eric Seidel  <eric@webkit.org>
286
287         Reviewed by darin.
288         
289         Add improved argument handling to sunspider, including
290         --runs=<number>, --tests=<pattern, --shark, and --help
291         Also re-factor code into subroutines
292
293         * sunspider: improved argument handling
294
295 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
296
297         Reviewed by Mark.
298
299         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
300         
301         * TODO:
302         * tests/LIST:
303         * tests/access-binary-trees.js: Added.
304         * tests/access-nsieve.js: Added.
305         * tests/bitops-nsieve-bits.js: Added.
306         * tests/math-partial-sums.js: Added.
307         * tests/math-spectral-norm.js: Added.
308         * tests/string-fasta.js: Added.
309
310 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
311
312         Reviewed by Darin.
313
314         * TODO: Updated to note areas that are now well-covered.
315         
316         - Change the tests and the driver to avoid leaving large object
317         graphs hanging around, since that throws off the subsequent tests.
318         
319         * resources/sunspider-standalone-driver.js:
320         * tests/3d-cube.js:
321         * tests/3d-morph.js:
322         * tests/3d-raytrace.js:
323         * tests/bitops-bitwise-and.js:
324         * tests/string-base64.js:
325         * tests/string-tagcloud.js:
326
327 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
328
329         Reviewed by Nikolas.
330         
331         - Rebalance test complexity.
332         
333         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
334         
335         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
336
337         However, for some tests, IE7 was a huge outlier compared to other
338         major browsers. For those tests, I used Firefox 2.0.0.8 on the
339         same OS and hardware as a normalization basis.
340         
341         * tests/3d-morph.js:
342         * tests/3d-raytrace.js:
343         * tests/bitops-3bit-bits-in-byte.js:
344         * tests/bitops-bits-in-byte.js:
345         * tests/bitops-bitwise-and.js:
346         * tests/math-cordic.js:
347         * tests/string-base64.js:
348         * tests/string-tagcloud.js:
349         * tests/string-unpack-code.js:
350
351 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
352
353         Reviewed by Oliver.
354         
355         - Fix the browser-hosted driver to be compatible with Safari 2
356
357         * hosted/sunspider-driver.html:
358
359 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
360
361         Rubber stamped by Hyatt.
362         
363         - Add BSD license notice to cordic test, the original author said to do so.
364
365         * tests/math-cordic.js:
366
367 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
368
369         Reviewed by Eric.
370         
371         - Support Windows and Windows browsers.
372
373         * hosted/sunspider-results.html: Display results correctly in IE.
374         * sunspider: tweaks to work on Windows, and with spaces in the path.
375
376 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
377
378         Rubber stamped by Oliver.
379
380         * tests/string-tagcloud.js: Remove stray print()
381
382 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
383
384         Reviewed by Oliver.
385         
386         Add some string test cases.
387
388         * tests/string-base64.js: Added. Does base64 encoding and decoding.
389         * tests/string-tagcloud.js: Added. Parses a JSON string and
390         generates tagcloud HTML markup based on the data.        
391         * tests/string-unpack-code.js: Added. Unpack compressed versions of
392         four of the most popular JavaScript libraries.
393         * tests/LIST: Added the new tests.
394
395 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
396
397         Reviewed by Anders.
398
399         * TODO: Updated with info on how to do the stats properly.
400
401         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
402         avoid overly slowing down the benchmark.
403
404         Added some 3D-related test cases.
405         
406         * tests/3d-cube.js: Added.
407         * tests/3d-morph.js: Added.
408         * tests/3d-raytrace.js: Added.
409         * tests/LIST:
410
411 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
412
413         Reviewed by Darin.
414
415         - New JavaScript benchmark
416         http://bugs.webkit.org/show_bug.cgi?id=15515
417         
418         A start on a new JavaScript benchmark, with standalone and
419         browser-hosted drivers.
420         
421         The test content is still incomplete and the drivers could use
422         some more features but there is enough here to make a good start.
423         
424         The drivers compute a 95% confidence interval on the mean for the
425         whole test, each category, and each individual test to make it
426         easier to tell whether differences are statistically
427         significant. The confidence interval can be narrowed by running
428         the test more times.
429         
430         * TODO: Added. Remaining things that need to be done.
431         * sunspider: Added. Perl script that acts as the standalone test driver.
432         * resources/sunspider-standalone-driver.js: Added. JavaScript part
433         of standalone test driver.
434         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
435         analysis code.
436         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
437         * make-hosted: Added. Script to generate browser-hosted tests.
438         * hosted/sunspider.html: Added. Start page for browser-hosted test.
439         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
440         * hosted/sunspider-results.html: Added. Results page for browser-hosted
441         tests.
442         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
443         tests.
444         * tests/LIST: Added. List of tests to use.
445         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
446         * tests/bitops-bits-in-byte.js: Added. Ditto.
447         * tests/bitops-bitwise-and.js: Added. Ditto.
448         * tests/math-cordic.js: Added. Ditto.
449