Reviewed by Oliver.
[WebKit-https.git] / SunSpider / ChangeLog
1 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
2
3         Reviewed by Oliver.
4         
5         - Give better error messages when sunspider-compare-results fails
6
7         * sunspider-compare-results:
8
9 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
10
11         Reviewed by Darin.
12         
13         - fix Date tests to work in command-line SpiderMonkey
14
15         * tests/date-format-tofte.js:
16         * tests/date-format-xparb.js:
17
18 2007-10-29  Darin Adler  <darin@apple.com>
19
20         Reviewed by Maciej.
21
22         * sunspider: Added --shark20 option, to run Shark at its highest sample
23         resolution instead of its default.
24
25 2007-10-28  Adam Roben  <aroben@apple.com>
26
27         Fix SunSpider on Windows
28
29         Reviewed by Eric.
30
31         * sunspider: Don't use colons in filenames.
32
33 2007-10-28  Darin Adler  <darin@apple.com>
34
35         Reviewed by Adam.
36
37         * resources/sunspider-analyze-results.js: Tweak the output format a little.
38         Change so that when there's only one run we don't write out confidence
39         intervals at all rather than writing "NaN" over and over again.
40
41         * sunspider: Use the time and date as a suffix on the results file, that way
42         all the results are left behind in the tmp directory. This will make it easy
43         for us to add features that compare past results in the future. Also rename
44         the .mshark files using the same scheme.
45
46         * sunspider-compare-results: Relax the parsing rules so we can parse the
47         results file exactly as it's emitted from sunspider.
48
49 2007-10-25  Adam Roben  <aroben@apple.com>
50
51         Strip carriage returns from results
52
53         Carriage returns were screwing up sunspider-compare-results on
54         Windows.
55
56         Rubberstamped by Oliver.
57
58         * sunspider:
59
60 2007-10-24  Darin Adler  <darin@apple.com>
61
62         Reviewed by Eric.
63
64         * sunspider: Open the profile in shark after completing a run.
65         * sunspider-compare-results: Tiny perl tweaks.
66
67 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
68
69         Reviewed by Darin.
70
71         - Don't hardcode my path to testkjs
72         
73         * sunspider-compare-results:
74
75 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
76
77         Reviewed by Darin.
78         
79         - Add a compare script to compare results, and improve formatting of standard results a bit.
80
81         * resources/sunspider-analyze-results.js:
82         * resources/sunspider-compare-results.js: Added.
83         * sunspider-compare-results: Added.
84
85 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
86
87         Reviewed by Darin.
88
89         * resources/sunspider-analyze-results.js: Do the error range
90         properly, using the t-distribution instead of 1.96 (which was
91         based on the normal distribution).
92         
93         * sunspider: Print results in a way that will be friendlier to the
94         soon-to-come compare mode.
95
96 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
97
98         Reviewed by Review.
99         
100         - add more tests (probably the final set for now)
101         
102         Already balanced. Added date, regexp, control flow, and a few
103         more string and object/array access tests.
104         
105         * tests/LIST:
106         * tests/access-fannkuch.js: Added.
107         * tests/access-nbody.js: Added.
108         * tests/controlflow-recursive.js: Added.
109         * tests/date-format-tofte.js: Added.
110         * tests/date-format-xparb.js: Added.
111         * tests/regexp-dna.js: Added.
112         * tests/string-validate-input.js: Added.
113
114 2007-10-21  Eric Seidel  <eric@webkit.org>
115
116         Reviewed by mjs.
117
118         * sunspider: make --shark only sample from testkjs
119
120 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
121
122         Rubber stamped by Eric.
123         
124         - added crypto tests (already balanced)
125
126         * tests/LIST:
127         * tests/crypto-aes.js: Added.
128         * tests/crypto-md5.js: Added.
129         * tests/crypto-sha1.js: Added.
130
131 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
132
133         Reviewed by Adam.
134         
135         - rebalance new tests
136         
137         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
138         
139         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
140
141         This balances them with the existing tests.
142
143         * tests/access-binary-trees.js:
144         * tests/access-nsieve.js:
145         * tests/bitops-nsieve-bits.js:
146         * tests/math-partial-sums.js:
147         * tests/math-spectral-norm.js:
148         * tests/string-fasta.js:
149
150 2007-10-20  Eric Seidel  <eric@webkit.org>
151
152         Reviewed by darin.
153         
154         Add improved argument handling to sunspider, including
155         --runs=<number>, --tests=<pattern, --shark, and --help
156         Also re-factor code into subroutines
157
158         * sunspider: improved argument handling
159
160 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
161
162         Reviewed by Mark.
163
164         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
165         
166         * TODO:
167         * tests/LIST:
168         * tests/access-binary-trees.js: Added.
169         * tests/access-nsieve.js: Added.
170         * tests/bitops-nsieve-bits.js: Added.
171         * tests/math-partial-sums.js: Added.
172         * tests/math-spectral-norm.js: Added.
173         * tests/string-fasta.js: Added.
174
175 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
176
177         Reviewed by Darin.
178
179         * TODO: Updated to note areas that are now well-covered.
180         
181         - Change the tests and the driver to avoid leaving large object
182         graphs hanging around, since that throws off the subsequent tests.
183         
184         * resources/sunspider-standalone-driver.js:
185         * tests/3d-cube.js:
186         * tests/3d-morph.js:
187         * tests/3d-raytrace.js:
188         * tests/bitops-bitwise-and.js:
189         * tests/string-base64.js:
190         * tests/string-tagcloud.js:
191
192 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
193
194         Reviewed by Nikolas.
195         
196         - Rebalance test complexity.
197         
198         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
199         
200         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
201
202         However, for some tests, IE7 was a huge outlier compared to other
203         major browsers. For those tests, I used Firefox 2.0.0.8 on the
204         same OS and hardware as a normalization basis.
205         
206         * tests/3d-morph.js:
207         * tests/3d-raytrace.js:
208         * tests/bitops-3bit-bits-in-byte.js:
209         * tests/bitops-bits-in-byte.js:
210         * tests/bitops-bitwise-and.js:
211         * tests/math-cordic.js:
212         * tests/string-base64.js:
213         * tests/string-tagcloud.js:
214         * tests/string-unpack-code.js:
215
216 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
217
218         Reviewed by Oliver.
219         
220         - Fix the browser-hosted driver to be compatible with Safari 2
221
222         * hosted/sunspider-driver.html:
223
224 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
225
226         Rubber stamped by Hyatt.
227         
228         - Add BSD license notice to cordic test, the original author said to do so.
229
230         * tests/math-cordic.js:
231
232 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
233
234         Reviewed by Eric.
235         
236         - Support Windows and Windows browsers.
237
238         * hosted/sunspider-results.html: Display results correctly in IE.
239         * sunspider: tweaks to work on Windows, and with spaces in the path.
240
241 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
242
243         Rubber stamped by Oliver.
244
245         * tests/string-tagcloud.js: Remove stray print()
246
247 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
248
249         Reviewed by Oliver.
250         
251         Add some string test cases.
252
253         * tests/string-base64.js: Added. Does base64 encoding and decoding.
254         * tests/string-tagcloud.js: Added. Parses a JSON string and
255         generates tagcloud HTML markup based on the data.        
256         * tests/string-unpack-code.js: Added. Unpack compressed versions of
257         four of the most popular JavaScript libraries.
258         * tests/LIST: Added the new tests.
259
260 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
261
262         Reviewed by Anders.
263
264         * TODO: Updated with info on how to do the stats properly.
265
266         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
267         avoid overly slowing down the benchmark.
268
269         Added some 3D-related test cases.
270         
271         * tests/3d-cube.js: Added.
272         * tests/3d-morph.js: Added.
273         * tests/3d-raytrace.js: Added.
274         * tests/LIST:
275
276 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
277
278         Reviewed by Darin.
279
280         - New JavaScript benchmark
281         http://bugs.webkit.org/show_bug.cgi?id=15515
282         
283         A start on a new JavaScript benchmark, with standalone and
284         browser-hosted drivers.
285         
286         The test content is still incomplete and the drivers could use
287         some more features but there is enough here to make a good start.
288         
289         The drivers compute a 95% confidence interval on the mean for the
290         whole test, each category, and each individual test to make it
291         easier to tell whether differences are statistically
292         significant. The confidence interval can be narrowed by running
293         the test more times.
294         
295         * TODO: Added. Remaining things that need to be done.
296         * sunspider: Added. Perl script that acts as the standalone test driver.
297         * resources/sunspider-standalone-driver.js: Added. JavaScript part
298         of standalone test driver.
299         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
300         analysis code.
301         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
302         * make-hosted: Added. Script to generate browser-hosted tests.
303         * hosted/sunspider.html: Added. Start page for browser-hosted test.
304         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
305         * hosted/sunspider-results.html: Added. Results page for browser-hosted
306         tests.
307         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
308         tests.
309         * tests/LIST: Added. List of tests to use.
310         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
311         * tests/bitops-bits-in-byte.js: Added. Ditto.
312         * tests/bitops-bitwise-and.js: Added. Ditto.
313         * tests/math-cordic.js: Added. Ditto.
314