Reviewed by Darin.
[WebKit-https.git] / SunSpider / ChangeLog
1 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
2
3         Reviewed by Darin.
4         
5         - fix Date tests to work in command-line SpiderMonkey
6
7         * tests/date-format-tofte.js:
8         * tests/date-format-xparb.js:
9
10 2007-10-29  Darin Adler  <darin@apple.com>
11
12         Reviewed by Maciej.
13
14         * sunspider: Added --shark20 option, to run Shark at its highest sample
15         resolution instead of its default.
16
17 2007-10-28  Adam Roben  <aroben@apple.com>
18
19         Fix SunSpider on Windows
20
21         Reviewed by Eric.
22
23         * sunspider: Don't use colons in filenames.
24
25 2007-10-28  Darin Adler  <darin@apple.com>
26
27         Reviewed by Adam.
28
29         * resources/sunspider-analyze-results.js: Tweak the output format a little.
30         Change so that when there's only one run we don't write out confidence
31         intervals at all rather than writing "NaN" over and over again.
32
33         * sunspider: Use the time and date as a suffix on the results file, that way
34         all the results are left behind in the tmp directory. This will make it easy
35         for us to add features that compare past results in the future. Also rename
36         the .mshark files using the same scheme.
37
38         * sunspider-compare-results: Relax the parsing rules so we can parse the
39         results file exactly as it's emitted from sunspider.
40
41 2007-10-25  Adam Roben  <aroben@apple.com>
42
43         Strip carriage returns from results
44
45         Carriage returns were screwing up sunspider-compare-results on
46         Windows.
47
48         Rubberstamped by Oliver.
49
50         * sunspider:
51
52 2007-10-24  Darin Adler  <darin@apple.com>
53
54         Reviewed by Eric.
55
56         * sunspider: Open the profile in shark after completing a run.
57         * sunspider-compare-results: Tiny perl tweaks.
58
59 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
60
61         Reviewed by Darin.
62
63         - Don't hardcode my path to testkjs
64         
65         * sunspider-compare-results:
66
67 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
68
69         Reviewed by Darin.
70         
71         - Add a compare script to compare results, and improve formatting of standard results a bit.
72
73         * resources/sunspider-analyze-results.js:
74         * resources/sunspider-compare-results.js: Added.
75         * sunspider-compare-results: Added.
76
77 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
78
79         Reviewed by Darin.
80
81         * resources/sunspider-analyze-results.js: Do the error range
82         properly, using the t-distribution instead of 1.96 (which was
83         based on the normal distribution).
84         
85         * sunspider: Print results in a way that will be friendlier to the
86         soon-to-come compare mode.
87
88 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
89
90         Reviewed by Review.
91         
92         - add more tests (probably the final set for now)
93         
94         Already balanced. Added date, regexp, control flow, and a few
95         more string and object/array access tests.
96         
97         * tests/LIST:
98         * tests/access-fannkuch.js: Added.
99         * tests/access-nbody.js: Added.
100         * tests/controlflow-recursive.js: Added.
101         * tests/date-format-tofte.js: Added.
102         * tests/date-format-xparb.js: Added.
103         * tests/regexp-dna.js: Added.
104         * tests/string-validate-input.js: Added.
105
106 2007-10-21  Eric Seidel  <eric@webkit.org>
107
108         Reviewed by mjs.
109
110         * sunspider: make --shark only sample from testkjs
111
112 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
113
114         Rubber stamped by Eric.
115         
116         - added crypto tests (already balanced)
117
118         * tests/LIST:
119         * tests/crypto-aes.js: Added.
120         * tests/crypto-md5.js: Added.
121         * tests/crypto-sha1.js: Added.
122
123 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
124
125         Reviewed by Adam.
126         
127         - rebalance new tests
128         
129         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
130         
131         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
132
133         This balances them with the existing tests.
134
135         * tests/access-binary-trees.js:
136         * tests/access-nsieve.js:
137         * tests/bitops-nsieve-bits.js:
138         * tests/math-partial-sums.js:
139         * tests/math-spectral-norm.js:
140         * tests/string-fasta.js:
141
142 2007-10-20  Eric Seidel  <eric@webkit.org>
143
144         Reviewed by darin.
145         
146         Add improved argument handling to sunspider, including
147         --runs=<number>, --tests=<pattern, --shark, and --help
148         Also re-factor code into subroutines
149
150         * sunspider: improved argument handling
151
152 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
153
154         Reviewed by Mark.
155
156         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
157         
158         * TODO:
159         * tests/LIST:
160         * tests/access-binary-trees.js: Added.
161         * tests/access-nsieve.js: Added.
162         * tests/bitops-nsieve-bits.js: Added.
163         * tests/math-partial-sums.js: Added.
164         * tests/math-spectral-norm.js: Added.
165         * tests/string-fasta.js: Added.
166
167 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
168
169         Reviewed by Darin.
170
171         * TODO: Updated to note areas that are now well-covered.
172         
173         - Change the tests and the driver to avoid leaving large object
174         graphs hanging around, since that throws off the subsequent tests.
175         
176         * resources/sunspider-standalone-driver.js:
177         * tests/3d-cube.js:
178         * tests/3d-morph.js:
179         * tests/3d-raytrace.js:
180         * tests/bitops-bitwise-and.js:
181         * tests/string-base64.js:
182         * tests/string-tagcloud.js:
183
184 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
185
186         Reviewed by Nikolas.
187         
188         - Rebalance test complexity.
189         
190         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
191         
192         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
193
194         However, for some tests, IE7 was a huge outlier compared to other
195         major browsers. For those tests, I used Firefox 2.0.0.8 on the
196         same OS and hardware as a normalization basis.
197         
198         * tests/3d-morph.js:
199         * tests/3d-raytrace.js:
200         * tests/bitops-3bit-bits-in-byte.js:
201         * tests/bitops-bits-in-byte.js:
202         * tests/bitops-bitwise-and.js:
203         * tests/math-cordic.js:
204         * tests/string-base64.js:
205         * tests/string-tagcloud.js:
206         * tests/string-unpack-code.js:
207
208 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
209
210         Reviewed by Oliver.
211         
212         - Fix the browser-hosted driver to be compatible with Safari 2
213
214         * hosted/sunspider-driver.html:
215
216 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
217
218         Rubber stamped by Hyatt.
219         
220         - Add BSD license notice to cordic test, the original author said to do so.
221
222         * tests/math-cordic.js:
223
224 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
225
226         Reviewed by Eric.
227         
228         - Support Windows and Windows browsers.
229
230         * hosted/sunspider-results.html: Display results correctly in IE.
231         * sunspider: tweaks to work on Windows, and with spaces in the path.
232
233 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
234
235         Rubber stamped by Oliver.
236
237         * tests/string-tagcloud.js: Remove stray print()
238
239 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
240
241         Reviewed by Oliver.
242         
243         Add some string test cases.
244
245         * tests/string-base64.js: Added. Does base64 encoding and decoding.
246         * tests/string-tagcloud.js: Added. Parses a JSON string and
247         generates tagcloud HTML markup based on the data.        
248         * tests/string-unpack-code.js: Added. Unpack compressed versions of
249         four of the most popular JavaScript libraries.
250         * tests/LIST: Added the new tests.
251
252 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
253
254         Reviewed by Anders.
255
256         * TODO: Updated with info on how to do the stats properly.
257
258         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
259         avoid overly slowing down the benchmark.
260
261         Added some 3D-related test cases.
262         
263         * tests/3d-cube.js: Added.
264         * tests/3d-morph.js: Added.
265         * tests/3d-raytrace.js: Added.
266         * tests/LIST:
267
268 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
269
270         Reviewed by Darin.
271
272         - New JavaScript benchmark
273         http://bugs.webkit.org/show_bug.cgi?id=15515
274         
275         A start on a new JavaScript benchmark, with standalone and
276         browser-hosted drivers.
277         
278         The test content is still incomplete and the drivers could use
279         some more features but there is enough here to make a good start.
280         
281         The drivers compute a 95% confidence interval on the mean for the
282         whole test, each category, and each individual test to make it
283         easier to tell whether differences are statistically
284         significant. The confidence interval can be narrowed by running
285         the test more times.
286         
287         * TODO: Added. Remaining things that need to be done.
288         * sunspider: Added. Perl script that acts as the standalone test driver.
289         * resources/sunspider-standalone-driver.js: Added. JavaScript part
290         of standalone test driver.
291         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
292         analysis code.
293         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
294         * make-hosted: Added. Script to generate browser-hosted tests.
295         * hosted/sunspider.html: Added. Start page for browser-hosted test.
296         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
297         * hosted/sunspider-results.html: Added. Results page for browser-hosted
298         tests.
299         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
300         tests.
301         * tests/LIST: Added. List of tests to use.
302         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
303         * tests/bitops-bits-in-byte.js: Added. Ditto.
304         * tests/bitops-bitwise-and.js: Added. Ditto.
305         * tests/math-cordic.js: Added. Ditto.
306