53af33715866084a512e147b43f0b3f1b6194b53
[WebKit-https.git] / SunSpider / ChangeLog
1 2007-11-02  Darin Adler  <darin@apple.com>
2
3         Reviewed by Maciej.
4
5         * sunspider: Added a "--base" option that records the current test as a baseline.
6         Moved the code to find the newest .mshark file into a function (since I needed to use
7         it in the other script and I prefer to copy/paste a function).
8
9         * sunspider-compare-results: Added logic to compare the baseline with the last
10         run when you don't pass any parameters.
11
12 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
13
14         Reviewed by Eric.
15         
16         - Added list of ECMAScript3 features that SunSpider does not cover currently.
17
18         * UNCOVERED: Added.
19
20 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
21
22         Reviewed by Oliver.
23         
24         - Give better error messages when sunspider-compare-results fails
25
26         * sunspider-compare-results:
27
28 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
29
30         Reviewed by Darin.
31         
32         - fix Date tests to work in command-line SpiderMonkey
33
34         * tests/date-format-tofte.js:
35         * tests/date-format-xparb.js:
36
37 2007-10-29  Darin Adler  <darin@apple.com>
38
39         Reviewed by Maciej.
40
41         * sunspider: Added --shark20 option, to run Shark at its highest sample
42         resolution instead of its default.
43
44 2007-10-28  Adam Roben  <aroben@apple.com>
45
46         Fix SunSpider on Windows
47
48         Reviewed by Eric.
49
50         * sunspider: Don't use colons in filenames.
51
52 2007-10-28  Darin Adler  <darin@apple.com>
53
54         Reviewed by Adam.
55
56         * resources/sunspider-analyze-results.js: Tweak the output format a little.
57         Change so that when there's only one run we don't write out confidence
58         intervals at all rather than writing "NaN" over and over again.
59
60         * sunspider: Use the time and date as a suffix on the results file, that way
61         all the results are left behind in the tmp directory. This will make it easy
62         for us to add features that compare past results in the future. Also rename
63         the .mshark files using the same scheme.
64
65         * sunspider-compare-results: Relax the parsing rules so we can parse the
66         results file exactly as it's emitted from sunspider.
67
68 2007-10-25  Adam Roben  <aroben@apple.com>
69
70         Strip carriage returns from results
71
72         Carriage returns were screwing up sunspider-compare-results on
73         Windows.
74
75         Rubberstamped by Oliver.
76
77         * sunspider:
78
79 2007-10-24  Darin Adler  <darin@apple.com>
80
81         Reviewed by Eric.
82
83         * sunspider: Open the profile in shark after completing a run.
84         * sunspider-compare-results: Tiny perl tweaks.
85
86 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
87
88         Reviewed by Darin.
89
90         - Don't hardcode my path to testkjs
91         
92         * sunspider-compare-results:
93
94 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
95
96         Reviewed by Darin.
97         
98         - Add a compare script to compare results, and improve formatting of standard results a bit.
99
100         * resources/sunspider-analyze-results.js:
101         * resources/sunspider-compare-results.js: Added.
102         * sunspider-compare-results: Added.
103
104 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
105
106         Reviewed by Darin.
107
108         * resources/sunspider-analyze-results.js: Do the error range
109         properly, using the t-distribution instead of 1.96 (which was
110         based on the normal distribution).
111         
112         * sunspider: Print results in a way that will be friendlier to the
113         soon-to-come compare mode.
114
115 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
116
117         Reviewed by Review.
118         
119         - add more tests (probably the final set for now)
120         
121         Already balanced. Added date, regexp, control flow, and a few
122         more string and object/array access tests.
123         
124         * tests/LIST:
125         * tests/access-fannkuch.js: Added.
126         * tests/access-nbody.js: Added.
127         * tests/controlflow-recursive.js: Added.
128         * tests/date-format-tofte.js: Added.
129         * tests/date-format-xparb.js: Added.
130         * tests/regexp-dna.js: Added.
131         * tests/string-validate-input.js: Added.
132
133 2007-10-21  Eric Seidel  <eric@webkit.org>
134
135         Reviewed by mjs.
136
137         * sunspider: make --shark only sample from testkjs
138
139 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
140
141         Rubber stamped by Eric.
142         
143         - added crypto tests (already balanced)
144
145         * tests/LIST:
146         * tests/crypto-aes.js: Added.
147         * tests/crypto-md5.js: Added.
148         * tests/crypto-sha1.js: Added.
149
150 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
151
152         Reviewed by Adam.
153         
154         - rebalance new tests
155         
156         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
157         
158         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
159
160         This balances them with the existing tests.
161
162         * tests/access-binary-trees.js:
163         * tests/access-nsieve.js:
164         * tests/bitops-nsieve-bits.js:
165         * tests/math-partial-sums.js:
166         * tests/math-spectral-norm.js:
167         * tests/string-fasta.js:
168
169 2007-10-20  Eric Seidel  <eric@webkit.org>
170
171         Reviewed by darin.
172         
173         Add improved argument handling to sunspider, including
174         --runs=<number>, --tests=<pattern, --shark, and --help
175         Also re-factor code into subroutines
176
177         * sunspider: improved argument handling
178
179 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
180
181         Reviewed by Mark.
182
183         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
184         
185         * TODO:
186         * tests/LIST:
187         * tests/access-binary-trees.js: Added.
188         * tests/access-nsieve.js: Added.
189         * tests/bitops-nsieve-bits.js: Added.
190         * tests/math-partial-sums.js: Added.
191         * tests/math-spectral-norm.js: Added.
192         * tests/string-fasta.js: Added.
193
194 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
195
196         Reviewed by Darin.
197
198         * TODO: Updated to note areas that are now well-covered.
199         
200         - Change the tests and the driver to avoid leaving large object
201         graphs hanging around, since that throws off the subsequent tests.
202         
203         * resources/sunspider-standalone-driver.js:
204         * tests/3d-cube.js:
205         * tests/3d-morph.js:
206         * tests/3d-raytrace.js:
207         * tests/bitops-bitwise-and.js:
208         * tests/string-base64.js:
209         * tests/string-tagcloud.js:
210
211 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
212
213         Reviewed by Nikolas.
214         
215         - Rebalance test complexity.
216         
217         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
218         
219         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
220
221         However, for some tests, IE7 was a huge outlier compared to other
222         major browsers. For those tests, I used Firefox 2.0.0.8 on the
223         same OS and hardware as a normalization basis.
224         
225         * tests/3d-morph.js:
226         * tests/3d-raytrace.js:
227         * tests/bitops-3bit-bits-in-byte.js:
228         * tests/bitops-bits-in-byte.js:
229         * tests/bitops-bitwise-and.js:
230         * tests/math-cordic.js:
231         * tests/string-base64.js:
232         * tests/string-tagcloud.js:
233         * tests/string-unpack-code.js:
234
235 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
236
237         Reviewed by Oliver.
238         
239         - Fix the browser-hosted driver to be compatible with Safari 2
240
241         * hosted/sunspider-driver.html:
242
243 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
244
245         Rubber stamped by Hyatt.
246         
247         - Add BSD license notice to cordic test, the original author said to do so.
248
249         * tests/math-cordic.js:
250
251 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
252
253         Reviewed by Eric.
254         
255         - Support Windows and Windows browsers.
256
257         * hosted/sunspider-results.html: Display results correctly in IE.
258         * sunspider: tweaks to work on Windows, and with spaces in the path.
259
260 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
261
262         Rubber stamped by Oliver.
263
264         * tests/string-tagcloud.js: Remove stray print()
265
266 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
267
268         Reviewed by Oliver.
269         
270         Add some string test cases.
271
272         * tests/string-base64.js: Added. Does base64 encoding and decoding.
273         * tests/string-tagcloud.js: Added. Parses a JSON string and
274         generates tagcloud HTML markup based on the data.        
275         * tests/string-unpack-code.js: Added. Unpack compressed versions of
276         four of the most popular JavaScript libraries.
277         * tests/LIST: Added the new tests.
278
279 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
280
281         Reviewed by Anders.
282
283         * TODO: Updated with info on how to do the stats properly.
284
285         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
286         avoid overly slowing down the benchmark.
287
288         Added some 3D-related test cases.
289         
290         * tests/3d-cube.js: Added.
291         * tests/3d-morph.js: Added.
292         * tests/3d-raytrace.js: Added.
293         * tests/LIST:
294
295 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
296
297         Reviewed by Darin.
298
299         - New JavaScript benchmark
300         http://bugs.webkit.org/show_bug.cgi?id=15515
301         
302         A start on a new JavaScript benchmark, with standalone and
303         browser-hosted drivers.
304         
305         The test content is still incomplete and the drivers could use
306         some more features but there is enough here to make a good start.
307         
308         The drivers compute a 95% confidence interval on the mean for the
309         whole test, each category, and each individual test to make it
310         easier to tell whether differences are statistically
311         significant. The confidence interval can be narrowed by running
312         the test more times.
313         
314         * TODO: Added. Remaining things that need to be done.
315         * sunspider: Added. Perl script that acts as the standalone test driver.
316         * resources/sunspider-standalone-driver.js: Added. JavaScript part
317         of standalone test driver.
318         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
319         analysis code.
320         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
321         * make-hosted: Added. Script to generate browser-hosted tests.
322         * hosted/sunspider.html: Added. Start page for browser-hosted test.
323         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
324         * hosted/sunspider-results.html: Added. Results page for browser-hosted
325         tests.
326         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
327         tests.
328         * tests/LIST: Added. List of tests to use.
329         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
330         * tests/bitops-bits-in-byte.js: Added. Ditto.
331         * tests/bitops-bitwise-and.js: Added. Ditto.
332         * tests/math-cordic.js: Added. Ditto.
333