cfad82cbe28be7d292e844395bf1662d53bce2e7
[WebKit-https.git] / SunSpider / ChangeLog
1 2007-11-04  Darin Adler  <darin@apple.com>
2
3         * hosted/sunspider.html: Tweaked formatting a bit.
4
5 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
6
7         Reviewed by Darin.
8
9         - Add compare mode and a description of the benchmark to the hosted version.
10
11         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
12         * hosted/sunspider.html: Added intro text.
13         * make-hosted: Copy new file.
14         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
15         a function.
16         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
17         expects.
18         * sunspider-compare-results: Use sunspider-standalone-compare.js.
19
20 2007-11-02  Darin Adler  <darin@apple.com>
21
22         * sunspider: Changed "--base" to "--set-baseline".
23
24 2007-11-02  Darin Adler  <darin@apple.com>
25
26         Reviewed by Maciej.
27
28         * sunspider: Added a "--base" option that records the current test as a baseline.
29         Moved the code to find the newest .mshark file into a function (since I needed to use
30         it in the other script and I prefer to copy/paste a function).
31
32         * sunspider-compare-results: Added logic to compare the baseline with the last
33         run when you don't pass any parameters.
34
35 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
36
37         Reviewed by Eric.
38         
39         - Added list of ECMAScript3 features that SunSpider does not cover currently.
40
41         * UNCOVERED: Added.
42
43 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
44
45         Reviewed by Oliver.
46         
47         - Give better error messages when sunspider-compare-results fails
48
49         * sunspider-compare-results:
50
51 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
52
53         Reviewed by Darin.
54         
55         - fix Date tests to work in command-line SpiderMonkey
56
57         * tests/date-format-tofte.js:
58         * tests/date-format-xparb.js:
59
60 2007-10-29  Darin Adler  <darin@apple.com>
61
62         Reviewed by Maciej.
63
64         * sunspider: Added --shark20 option, to run Shark at its highest sample
65         resolution instead of its default.
66
67 2007-10-28  Adam Roben  <aroben@apple.com>
68
69         Fix SunSpider on Windows
70
71         Reviewed by Eric.
72
73         * sunspider: Don't use colons in filenames.
74
75 2007-10-28  Darin Adler  <darin@apple.com>
76
77         Reviewed by Adam.
78
79         * resources/sunspider-analyze-results.js: Tweak the output format a little.
80         Change so that when there's only one run we don't write out confidence
81         intervals at all rather than writing "NaN" over and over again.
82
83         * sunspider: Use the time and date as a suffix on the results file, that way
84         all the results are left behind in the tmp directory. This will make it easy
85         for us to add features that compare past results in the future. Also rename
86         the .mshark files using the same scheme.
87
88         * sunspider-compare-results: Relax the parsing rules so we can parse the
89         results file exactly as it's emitted from sunspider.
90
91 2007-10-25  Adam Roben  <aroben@apple.com>
92
93         Strip carriage returns from results
94
95         Carriage returns were screwing up sunspider-compare-results on
96         Windows.
97
98         Rubberstamped by Oliver.
99
100         * sunspider:
101
102 2007-10-24  Darin Adler  <darin@apple.com>
103
104         Reviewed by Eric.
105
106         * sunspider: Open the profile in shark after completing a run.
107         * sunspider-compare-results: Tiny perl tweaks.
108
109 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
110
111         Reviewed by Darin.
112
113         - Don't hardcode my path to testkjs
114         
115         * sunspider-compare-results:
116
117 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
118
119         Reviewed by Darin.
120         
121         - Add a compare script to compare results, and improve formatting of standard results a bit.
122
123         * resources/sunspider-analyze-results.js:
124         * resources/sunspider-compare-results.js: Added.
125         * sunspider-compare-results: Added.
126
127 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
128
129         Reviewed by Darin.
130
131         * resources/sunspider-analyze-results.js: Do the error range
132         properly, using the t-distribution instead of 1.96 (which was
133         based on the normal distribution).
134         
135         * sunspider: Print results in a way that will be friendlier to the
136         soon-to-come compare mode.
137
138 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
139
140         Reviewed by Review.
141         
142         - add more tests (probably the final set for now)
143         
144         Already balanced. Added date, regexp, control flow, and a few
145         more string and object/array access tests.
146         
147         * tests/LIST:
148         * tests/access-fannkuch.js: Added.
149         * tests/access-nbody.js: Added.
150         * tests/controlflow-recursive.js: Added.
151         * tests/date-format-tofte.js: Added.
152         * tests/date-format-xparb.js: Added.
153         * tests/regexp-dna.js: Added.
154         * tests/string-validate-input.js: Added.
155
156 2007-10-21  Eric Seidel  <eric@webkit.org>
157
158         Reviewed by mjs.
159
160         * sunspider: make --shark only sample from testkjs
161
162 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
163
164         Rubber stamped by Eric.
165         
166         - added crypto tests (already balanced)
167
168         * tests/LIST:
169         * tests/crypto-aes.js: Added.
170         * tests/crypto-md5.js: Added.
171         * tests/crypto-sha1.js: Added.
172
173 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
174
175         Reviewed by Adam.
176         
177         - rebalance new tests
178         
179         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
180         
181         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
182
183         This balances them with the existing tests.
184
185         * tests/access-binary-trees.js:
186         * tests/access-nsieve.js:
187         * tests/bitops-nsieve-bits.js:
188         * tests/math-partial-sums.js:
189         * tests/math-spectral-norm.js:
190         * tests/string-fasta.js:
191
192 2007-10-20  Eric Seidel  <eric@webkit.org>
193
194         Reviewed by darin.
195         
196         Add improved argument handling to sunspider, including
197         --runs=<number>, --tests=<pattern, --shark, and --help
198         Also re-factor code into subroutines
199
200         * sunspider: improved argument handling
201
202 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
203
204         Reviewed by Mark.
205
206         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
207         
208         * TODO:
209         * tests/LIST:
210         * tests/access-binary-trees.js: Added.
211         * tests/access-nsieve.js: Added.
212         * tests/bitops-nsieve-bits.js: Added.
213         * tests/math-partial-sums.js: Added.
214         * tests/math-spectral-norm.js: Added.
215         * tests/string-fasta.js: Added.
216
217 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
218
219         Reviewed by Darin.
220
221         * TODO: Updated to note areas that are now well-covered.
222         
223         - Change the tests and the driver to avoid leaving large object
224         graphs hanging around, since that throws off the subsequent tests.
225         
226         * resources/sunspider-standalone-driver.js:
227         * tests/3d-cube.js:
228         * tests/3d-morph.js:
229         * tests/3d-raytrace.js:
230         * tests/bitops-bitwise-and.js:
231         * tests/string-base64.js:
232         * tests/string-tagcloud.js:
233
234 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
235
236         Reviewed by Nikolas.
237         
238         - Rebalance test complexity.
239         
240         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
241         
242         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
243
244         However, for some tests, IE7 was a huge outlier compared to other
245         major browsers. For those tests, I used Firefox 2.0.0.8 on the
246         same OS and hardware as a normalization basis.
247         
248         * tests/3d-morph.js:
249         * tests/3d-raytrace.js:
250         * tests/bitops-3bit-bits-in-byte.js:
251         * tests/bitops-bits-in-byte.js:
252         * tests/bitops-bitwise-and.js:
253         * tests/math-cordic.js:
254         * tests/string-base64.js:
255         * tests/string-tagcloud.js:
256         * tests/string-unpack-code.js:
257
258 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
259
260         Reviewed by Oliver.
261         
262         - Fix the browser-hosted driver to be compatible with Safari 2
263
264         * hosted/sunspider-driver.html:
265
266 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
267
268         Rubber stamped by Hyatt.
269         
270         - Add BSD license notice to cordic test, the original author said to do so.
271
272         * tests/math-cordic.js:
273
274 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
275
276         Reviewed by Eric.
277         
278         - Support Windows and Windows browsers.
279
280         * hosted/sunspider-results.html: Display results correctly in IE.
281         * sunspider: tweaks to work on Windows, and with spaces in the path.
282
283 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
284
285         Rubber stamped by Oliver.
286
287         * tests/string-tagcloud.js: Remove stray print()
288
289 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
290
291         Reviewed by Oliver.
292         
293         Add some string test cases.
294
295         * tests/string-base64.js: Added. Does base64 encoding and decoding.
296         * tests/string-tagcloud.js: Added. Parses a JSON string and
297         generates tagcloud HTML markup based on the data.        
298         * tests/string-unpack-code.js: Added. Unpack compressed versions of
299         four of the most popular JavaScript libraries.
300         * tests/LIST: Added the new tests.
301
302 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
303
304         Reviewed by Anders.
305
306         * TODO: Updated with info on how to do the stats properly.
307
308         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
309         avoid overly slowing down the benchmark.
310
311         Added some 3D-related test cases.
312         
313         * tests/3d-cube.js: Added.
314         * tests/3d-morph.js: Added.
315         * tests/3d-raytrace.js: Added.
316         * tests/LIST:
317
318 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
319
320         Reviewed by Darin.
321
322         - New JavaScript benchmark
323         http://bugs.webkit.org/show_bug.cgi?id=15515
324         
325         A start on a new JavaScript benchmark, with standalone and
326         browser-hosted drivers.
327         
328         The test content is still incomplete and the drivers could use
329         some more features but there is enough here to make a good start.
330         
331         The drivers compute a 95% confidence interval on the mean for the
332         whole test, each category, and each individual test to make it
333         easier to tell whether differences are statistically
334         significant. The confidence interval can be narrowed by running
335         the test more times.
336         
337         * TODO: Added. Remaining things that need to be done.
338         * sunspider: Added. Perl script that acts as the standalone test driver.
339         * resources/sunspider-standalone-driver.js: Added. JavaScript part
340         of standalone test driver.
341         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
342         analysis code.
343         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
344         * make-hosted: Added. Script to generate browser-hosted tests.
345         * hosted/sunspider.html: Added. Start page for browser-hosted test.
346         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
347         * hosted/sunspider-results.html: Added. Results page for browser-hosted
348         tests.
349         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
350         tests.
351         * tests/LIST: Added. List of tests to use.
352         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
353         * tests/bitops-bits-in-byte.js: Added. Ditto.
354         * tests/bitops-bitwise-and.js: Added. Ditto.
355         * tests/math-cordic.js: Added. Ditto.
356