Reviewed by Adam.
[WebKit-https.git] / SunSpider / ChangeLog
1 2007-10-28  Darin Adler  <darin@apple.com>
2
3         Reviewed by Adam.
4
5         * resources/sunspider-analyze-results.js: Tweak the output format a little.
6         Change so that when there's only one run we don't write out confidence
7         intervals at all rather than writing "NaN" over and over again.
8
9         * sunspider: Use the time and date as a suffix on the results file, that way
10         all the results are left behind in the tmp directory. This will make it easy
11         for us to add features that compare past results in the future. Also rename
12         the .mshark files using the same scheme.
13
14         * sunspider-compare-results: Relax the parsing rules so we can parse the
15         results file exactly as it's emitted from sunspider.
16
17 2007-10-25  Adam Roben  <aroben@apple.com>
18
19         Strip carriage returns from results
20
21         Carriage returns were screwing up sunspider-compare-results on
22         Windows.
23
24         Rubberstamped by Oliver.
25
26         * sunspider:
27
28 2007-10-24  Darin Adler  <darin@apple.com>
29
30         Reviewed by Eric.
31
32         * sunspider: Open the profile in shark after completing a run.
33         * sunspider-compare-results: Tiny perl tweaks.
34
35 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
36
37         Reviewed by Darin.
38
39         - Don't hardcode my path to testkjs
40         
41         * sunspider-compare-results:
42
43 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
44
45         Reviewed by Darin.
46         
47         - Add a compare script to compare results, and improve formatting of standard results a bit.
48
49         * resources/sunspider-analyze-results.js:
50         * resources/sunspider-compare-results.js: Added.
51         * sunspider-compare-results: Added.
52
53 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
54
55         Reviewed by Darin.
56
57         * resources/sunspider-analyze-results.js: Do the error range
58         properly, using the t-distribution instead of 1.96 (which was
59         based on the normal distribution).
60         
61         * sunspider: Print results in a way that will be friendlier to the
62         soon-to-come compare mode.
63
64 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
65
66         Reviewed by Review.
67         
68         - add more tests (probably the final set for now)
69         
70         Already balanced. Added date, regexp, control flow, and a few
71         more string and object/array access tests.
72         
73         * tests/LIST:
74         * tests/access-fannkuch.js: Added.
75         * tests/access-nbody.js: Added.
76         * tests/controlflow-recursive.js: Added.
77         * tests/date-format-tofte.js: Added.
78         * tests/date-format-xparb.js: Added.
79         * tests/regexp-dna.js: Added.
80         * tests/string-validate-input.js: Added.
81
82 2007-10-21  Eric Seidel  <eric@webkit.org>
83
84         Reviewed by mjs.
85
86         * sunspider: make --shark only sample from testkjs
87
88 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
89
90         Rubber stamped by Eric.
91         
92         - added crypto tests (already balanced)
93
94         * tests/LIST:
95         * tests/crypto-aes.js: Added.
96         * tests/crypto-md5.js: Added.
97         * tests/crypto-sha1.js: Added.
98
99 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
100
101         Reviewed by Adam.
102         
103         - rebalance new tests
104         
105         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
106         
107         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
108
109         This balances them with the existing tests.
110
111         * tests/access-binary-trees.js:
112         * tests/access-nsieve.js:
113         * tests/bitops-nsieve-bits.js:
114         * tests/math-partial-sums.js:
115         * tests/math-spectral-norm.js:
116         * tests/string-fasta.js:
117
118 2007-10-20  Eric Seidel  <eric@webkit.org>
119
120         Reviewed by darin.
121         
122         Add improved argument handling to sunspider, including
123         --runs=<number>, --tests=<pattern, --shark, and --help
124         Also re-factor code into subroutines
125
126         * sunspider: improved argument handling
127
128 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
129
130         Reviewed by Mark.
131
132         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
133         
134         * TODO:
135         * tests/LIST:
136         * tests/access-binary-trees.js: Added.
137         * tests/access-nsieve.js: Added.
138         * tests/bitops-nsieve-bits.js: Added.
139         * tests/math-partial-sums.js: Added.
140         * tests/math-spectral-norm.js: Added.
141         * tests/string-fasta.js: Added.
142
143 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
144
145         Reviewed by Darin.
146
147         * TODO: Updated to note areas that are now well-covered.
148         
149         - Change the tests and the driver to avoid leaving large object
150         graphs hanging around, since that throws off the subsequent tests.
151         
152         * resources/sunspider-standalone-driver.js:
153         * tests/3d-cube.js:
154         * tests/3d-morph.js:
155         * tests/3d-raytrace.js:
156         * tests/bitops-bitwise-and.js:
157         * tests/string-base64.js:
158         * tests/string-tagcloud.js:
159
160 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
161
162         Reviewed by Nikolas.
163         
164         - Rebalance test complexity.
165         
166         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
167         
168         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
169
170         However, for some tests, IE7 was a huge outlier compared to other
171         major browsers. For those tests, I used Firefox 2.0.0.8 on the
172         same OS and hardware as a normalization basis.
173         
174         * tests/3d-morph.js:
175         * tests/3d-raytrace.js:
176         * tests/bitops-3bit-bits-in-byte.js:
177         * tests/bitops-bits-in-byte.js:
178         * tests/bitops-bitwise-and.js:
179         * tests/math-cordic.js:
180         * tests/string-base64.js:
181         * tests/string-tagcloud.js:
182         * tests/string-unpack-code.js:
183
184 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
185
186         Reviewed by Oliver.
187         
188         - Fix the browser-hosted driver to be compatible with Safari 2
189
190         * hosted/sunspider-driver.html:
191
192 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
193
194         Rubber stamped by Hyatt.
195         
196         - Add BSD license notice to cordic test, the original author said to do so.
197
198         * tests/math-cordic.js:
199
200 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
201
202         Reviewed by Eric.
203         
204         - Support Windows and Windows browsers.
205
206         * hosted/sunspider-results.html: Display results correctly in IE.
207         * sunspider: tweaks to work on Windows, and with spaces in the path.
208
209 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
210
211         Rubber stamped by Oliver.
212
213         * tests/string-tagcloud.js: Remove stray print()
214
215 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
216
217         Reviewed by Oliver.
218         
219         Add some string test cases.
220
221         * tests/string-base64.js: Added. Does base64 encoding and decoding.
222         * tests/string-tagcloud.js: Added. Parses a JSON string and
223         generates tagcloud HTML markup based on the data.        
224         * tests/string-unpack-code.js: Added. Unpack compressed versions of
225         four of the most popular JavaScript libraries.
226         * tests/LIST: Added the new tests.
227
228 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
229
230         Reviewed by Anders.
231
232         * TODO: Updated with info on how to do the stats properly.
233
234         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
235         avoid overly slowing down the benchmark.
236
237         Added some 3D-related test cases.
238         
239         * tests/3d-cube.js: Added.
240         * tests/3d-morph.js: Added.
241         * tests/3d-raytrace.js: Added.
242         * tests/LIST:
243
244 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
245
246         Reviewed by Darin.
247
248         - New JavaScript benchmark
249         http://bugs.webkit.org/show_bug.cgi?id=15515
250         
251         A start on a new JavaScript benchmark, with standalone and
252         browser-hosted drivers.
253         
254         The test content is still incomplete and the drivers could use
255         some more features but there is enough here to make a good start.
256         
257         The drivers compute a 95% confidence interval on the mean for the
258         whole test, each category, and each individual test to make it
259         easier to tell whether differences are statistically
260         significant. The confidence interval can be narrowed by running
261         the test more times.
262         
263         * TODO: Added. Remaining things that need to be done.
264         * sunspider: Added. Perl script that acts as the standalone test driver.
265         * resources/sunspider-standalone-driver.js: Added. JavaScript part
266         of standalone test driver.
267         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
268         analysis code.
269         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
270         * make-hosted: Added. Script to generate browser-hosted tests.
271         * hosted/sunspider.html: Added. Start page for browser-hosted test.
272         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
273         * hosted/sunspider-results.html: Added. Results page for browser-hosted
274         tests.
275         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
276         tests.
277         * tests/LIST: Added. List of tests to use.
278         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
279         * tests/bitops-bits-in-byte.js: Added. Ditto.
280         * tests/bitops-bitwise-and.js: Added. Ditto.
281         * tests/math-cordic.js: Added. Ditto.
282