Reviewed by Darin.
[WebKit-https.git] / SunSpider / ChangeLog
1 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
2
3         Reviewed by Darin.
4
5         - Add compare mode and a description of the benchmark to the hosted version.
6
7         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
8         * hosted/sunspider.html: Added intro text.
9         * make-hosted: Copy new file.
10         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
11         a function.
12         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
13         expects.
14         * sunspider-compare-results: Use sunspider-standalone-compare.js.
15
16 2007-11-02  Darin Adler  <darin@apple.com>
17
18         * sunspider: Changed "--base" to "--set-baseline".
19
20 2007-11-02  Darin Adler  <darin@apple.com>
21
22         Reviewed by Maciej.
23
24         * sunspider: Added a "--base" option that records the current test as a baseline.
25         Moved the code to find the newest .mshark file into a function (since I needed to use
26         it in the other script and I prefer to copy/paste a function).
27
28         * sunspider-compare-results: Added logic to compare the baseline with the last
29         run when you don't pass any parameters.
30
31 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
32
33         Reviewed by Eric.
34         
35         - Added list of ECMAScript3 features that SunSpider does not cover currently.
36
37         * UNCOVERED: Added.
38
39 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
40
41         Reviewed by Oliver.
42         
43         - Give better error messages when sunspider-compare-results fails
44
45         * sunspider-compare-results:
46
47 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
48
49         Reviewed by Darin.
50         
51         - fix Date tests to work in command-line SpiderMonkey
52
53         * tests/date-format-tofte.js:
54         * tests/date-format-xparb.js:
55
56 2007-10-29  Darin Adler  <darin@apple.com>
57
58         Reviewed by Maciej.
59
60         * sunspider: Added --shark20 option, to run Shark at its highest sample
61         resolution instead of its default.
62
63 2007-10-28  Adam Roben  <aroben@apple.com>
64
65         Fix SunSpider on Windows
66
67         Reviewed by Eric.
68
69         * sunspider: Don't use colons in filenames.
70
71 2007-10-28  Darin Adler  <darin@apple.com>
72
73         Reviewed by Adam.
74
75         * resources/sunspider-analyze-results.js: Tweak the output format a little.
76         Change so that when there's only one run we don't write out confidence
77         intervals at all rather than writing "NaN" over and over again.
78
79         * sunspider: Use the time and date as a suffix on the results file, that way
80         all the results are left behind in the tmp directory. This will make it easy
81         for us to add features that compare past results in the future. Also rename
82         the .mshark files using the same scheme.
83
84         * sunspider-compare-results: Relax the parsing rules so we can parse the
85         results file exactly as it's emitted from sunspider.
86
87 2007-10-25  Adam Roben  <aroben@apple.com>
88
89         Strip carriage returns from results
90
91         Carriage returns were screwing up sunspider-compare-results on
92         Windows.
93
94         Rubberstamped by Oliver.
95
96         * sunspider:
97
98 2007-10-24  Darin Adler  <darin@apple.com>
99
100         Reviewed by Eric.
101
102         * sunspider: Open the profile in shark after completing a run.
103         * sunspider-compare-results: Tiny perl tweaks.
104
105 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
106
107         Reviewed by Darin.
108
109         - Don't hardcode my path to testkjs
110         
111         * sunspider-compare-results:
112
113 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
114
115         Reviewed by Darin.
116         
117         - Add a compare script to compare results, and improve formatting of standard results a bit.
118
119         * resources/sunspider-analyze-results.js:
120         * resources/sunspider-compare-results.js: Added.
121         * sunspider-compare-results: Added.
122
123 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
124
125         Reviewed by Darin.
126
127         * resources/sunspider-analyze-results.js: Do the error range
128         properly, using the t-distribution instead of 1.96 (which was
129         based on the normal distribution).
130         
131         * sunspider: Print results in a way that will be friendlier to the
132         soon-to-come compare mode.
133
134 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
135
136         Reviewed by Review.
137         
138         - add more tests (probably the final set for now)
139         
140         Already balanced. Added date, regexp, control flow, and a few
141         more string and object/array access tests.
142         
143         * tests/LIST:
144         * tests/access-fannkuch.js: Added.
145         * tests/access-nbody.js: Added.
146         * tests/controlflow-recursive.js: Added.
147         * tests/date-format-tofte.js: Added.
148         * tests/date-format-xparb.js: Added.
149         * tests/regexp-dna.js: Added.
150         * tests/string-validate-input.js: Added.
151
152 2007-10-21  Eric Seidel  <eric@webkit.org>
153
154         Reviewed by mjs.
155
156         * sunspider: make --shark only sample from testkjs
157
158 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
159
160         Rubber stamped by Eric.
161         
162         - added crypto tests (already balanced)
163
164         * tests/LIST:
165         * tests/crypto-aes.js: Added.
166         * tests/crypto-md5.js: Added.
167         * tests/crypto-sha1.js: Added.
168
169 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
170
171         Reviewed by Adam.
172         
173         - rebalance new tests
174         
175         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
176         
177         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
178
179         This balances them with the existing tests.
180
181         * tests/access-binary-trees.js:
182         * tests/access-nsieve.js:
183         * tests/bitops-nsieve-bits.js:
184         * tests/math-partial-sums.js:
185         * tests/math-spectral-norm.js:
186         * tests/string-fasta.js:
187
188 2007-10-20  Eric Seidel  <eric@webkit.org>
189
190         Reviewed by darin.
191         
192         Add improved argument handling to sunspider, including
193         --runs=<number>, --tests=<pattern, --shark, and --help
194         Also re-factor code into subroutines
195
196         * sunspider: improved argument handling
197
198 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
199
200         Reviewed by Mark.
201
202         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
203         
204         * TODO:
205         * tests/LIST:
206         * tests/access-binary-trees.js: Added.
207         * tests/access-nsieve.js: Added.
208         * tests/bitops-nsieve-bits.js: Added.
209         * tests/math-partial-sums.js: Added.
210         * tests/math-spectral-norm.js: Added.
211         * tests/string-fasta.js: Added.
212
213 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
214
215         Reviewed by Darin.
216
217         * TODO: Updated to note areas that are now well-covered.
218         
219         - Change the tests and the driver to avoid leaving large object
220         graphs hanging around, since that throws off the subsequent tests.
221         
222         * resources/sunspider-standalone-driver.js:
223         * tests/3d-cube.js:
224         * tests/3d-morph.js:
225         * tests/3d-raytrace.js:
226         * tests/bitops-bitwise-and.js:
227         * tests/string-base64.js:
228         * tests/string-tagcloud.js:
229
230 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
231
232         Reviewed by Nikolas.
233         
234         - Rebalance test complexity.
235         
236         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
237         
238         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
239
240         However, for some tests, IE7 was a huge outlier compared to other
241         major browsers. For those tests, I used Firefox 2.0.0.8 on the
242         same OS and hardware as a normalization basis.
243         
244         * tests/3d-morph.js:
245         * tests/3d-raytrace.js:
246         * tests/bitops-3bit-bits-in-byte.js:
247         * tests/bitops-bits-in-byte.js:
248         * tests/bitops-bitwise-and.js:
249         * tests/math-cordic.js:
250         * tests/string-base64.js:
251         * tests/string-tagcloud.js:
252         * tests/string-unpack-code.js:
253
254 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
255
256         Reviewed by Oliver.
257         
258         - Fix the browser-hosted driver to be compatible with Safari 2
259
260         * hosted/sunspider-driver.html:
261
262 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
263
264         Rubber stamped by Hyatt.
265         
266         - Add BSD license notice to cordic test, the original author said to do so.
267
268         * tests/math-cordic.js:
269
270 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
271
272         Reviewed by Eric.
273         
274         - Support Windows and Windows browsers.
275
276         * hosted/sunspider-results.html: Display results correctly in IE.
277         * sunspider: tweaks to work on Windows, and with spaces in the path.
278
279 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
280
281         Rubber stamped by Oliver.
282
283         * tests/string-tagcloud.js: Remove stray print()
284
285 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
286
287         Reviewed by Oliver.
288         
289         Add some string test cases.
290
291         * tests/string-base64.js: Added. Does base64 encoding and decoding.
292         * tests/string-tagcloud.js: Added. Parses a JSON string and
293         generates tagcloud HTML markup based on the data.        
294         * tests/string-unpack-code.js: Added. Unpack compressed versions of
295         four of the most popular JavaScript libraries.
296         * tests/LIST: Added the new tests.
297
298 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
299
300         Reviewed by Anders.
301
302         * TODO: Updated with info on how to do the stats properly.
303
304         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
305         avoid overly slowing down the benchmark.
306
307         Added some 3D-related test cases.
308         
309         * tests/3d-cube.js: Added.
310         * tests/3d-morph.js: Added.
311         * tests/3d-raytrace.js: Added.
312         * tests/LIST:
313
314 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
315
316         Reviewed by Darin.
317
318         - New JavaScript benchmark
319         http://bugs.webkit.org/show_bug.cgi?id=15515
320         
321         A start on a new JavaScript benchmark, with standalone and
322         browser-hosted drivers.
323         
324         The test content is still incomplete and the drivers could use
325         some more features but there is enough here to make a good start.
326         
327         The drivers compute a 95% confidence interval on the mean for the
328         whole test, each category, and each individual test to make it
329         easier to tell whether differences are statistically
330         significant. The confidence interval can be narrowed by running
331         the test more times.
332         
333         * TODO: Added. Remaining things that need to be done.
334         * sunspider: Added. Perl script that acts as the standalone test driver.
335         * resources/sunspider-standalone-driver.js: Added. JavaScript part
336         of standalone test driver.
337         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
338         analysis code.
339         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
340         * make-hosted: Added. Script to generate browser-hosted tests.
341         * hosted/sunspider.html: Added. Start page for browser-hosted test.
342         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
343         * hosted/sunspider-results.html: Added. Results page for browser-hosted
344         tests.
345         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
346         tests.
347         * tests/LIST: Added. List of tests to use.
348         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
349         * tests/bitops-bits-in-byte.js: Added. Ditto.
350         * tests/bitops-bitwise-and.js: Added. Ditto.
351         * tests/math-cordic.js: Added. Ditto.
352