Fix <rdar://5133816> keepWebHistory is not implemented
[WebKit-https.git] / SunSpider / ChangeLog
1 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
2
3         Reviewed by Darin.
4
5         http://bugs.webkit.org/show_bug.cgi?id=15925
6         SunSpider should check for Shark being installed
7
8         * sunspider: Explicitly test that shark command line tool is installed.
9
10 2007-11-04  Darin Adler  <darin@apple.com>
11
12         * hosted/sunspider.html: Tweaked formatting a bit.
13
14 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
15
16         Reviewed by Darin.
17
18         - Add compare mode and a description of the benchmark to the hosted version.
19
20         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
21         * hosted/sunspider.html: Added intro text.
22         * make-hosted: Copy new file.
23         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
24         a function.
25         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
26         expects.
27         * sunspider-compare-results: Use sunspider-standalone-compare.js.
28
29 2007-11-02  Darin Adler  <darin@apple.com>
30
31         * sunspider: Changed "--base" to "--set-baseline".
32
33 2007-11-02  Darin Adler  <darin@apple.com>
34
35         Reviewed by Maciej.
36
37         * sunspider: Added a "--base" option that records the current test as a baseline.
38         Moved the code to find the newest .mshark file into a function (since I needed to use
39         it in the other script and I prefer to copy/paste a function).
40
41         * sunspider-compare-results: Added logic to compare the baseline with the last
42         run when you don't pass any parameters.
43
44 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
45
46         Reviewed by Eric.
47         
48         - Added list of ECMAScript3 features that SunSpider does not cover currently.
49
50         * UNCOVERED: Added.
51
52 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
53
54         Reviewed by Oliver.
55         
56         - Give better error messages when sunspider-compare-results fails
57
58         * sunspider-compare-results:
59
60 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
61
62         Reviewed by Darin.
63         
64         - fix Date tests to work in command-line SpiderMonkey
65
66         * tests/date-format-tofte.js:
67         * tests/date-format-xparb.js:
68
69 2007-10-29  Darin Adler  <darin@apple.com>
70
71         Reviewed by Maciej.
72
73         * sunspider: Added --shark20 option, to run Shark at its highest sample
74         resolution instead of its default.
75
76 2007-10-28  Adam Roben  <aroben@apple.com>
77
78         Fix SunSpider on Windows
79
80         Reviewed by Eric.
81
82         * sunspider: Don't use colons in filenames.
83
84 2007-10-28  Darin Adler  <darin@apple.com>
85
86         Reviewed by Adam.
87
88         * resources/sunspider-analyze-results.js: Tweak the output format a little.
89         Change so that when there's only one run we don't write out confidence
90         intervals at all rather than writing "NaN" over and over again.
91
92         * sunspider: Use the time and date as a suffix on the results file, that way
93         all the results are left behind in the tmp directory. This will make it easy
94         for us to add features that compare past results in the future. Also rename
95         the .mshark files using the same scheme.
96
97         * sunspider-compare-results: Relax the parsing rules so we can parse the
98         results file exactly as it's emitted from sunspider.
99
100 2007-10-25  Adam Roben  <aroben@apple.com>
101
102         Strip carriage returns from results
103
104         Carriage returns were screwing up sunspider-compare-results on
105         Windows.
106
107         Rubberstamped by Oliver.
108
109         * sunspider:
110
111 2007-10-24  Darin Adler  <darin@apple.com>
112
113         Reviewed by Eric.
114
115         * sunspider: Open the profile in shark after completing a run.
116         * sunspider-compare-results: Tiny perl tweaks.
117
118 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
119
120         Reviewed by Darin.
121
122         - Don't hardcode my path to testkjs
123         
124         * sunspider-compare-results:
125
126 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
127
128         Reviewed by Darin.
129         
130         - Add a compare script to compare results, and improve formatting of standard results a bit.
131
132         * resources/sunspider-analyze-results.js:
133         * resources/sunspider-compare-results.js: Added.
134         * sunspider-compare-results: Added.
135
136 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
137
138         Reviewed by Darin.
139
140         * resources/sunspider-analyze-results.js: Do the error range
141         properly, using the t-distribution instead of 1.96 (which was
142         based on the normal distribution).
143         
144         * sunspider: Print results in a way that will be friendlier to the
145         soon-to-come compare mode.
146
147 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
148
149         Reviewed by Review.
150         
151         - add more tests (probably the final set for now)
152         
153         Already balanced. Added date, regexp, control flow, and a few
154         more string and object/array access tests.
155         
156         * tests/LIST:
157         * tests/access-fannkuch.js: Added.
158         * tests/access-nbody.js: Added.
159         * tests/controlflow-recursive.js: Added.
160         * tests/date-format-tofte.js: Added.
161         * tests/date-format-xparb.js: Added.
162         * tests/regexp-dna.js: Added.
163         * tests/string-validate-input.js: Added.
164
165 2007-10-21  Eric Seidel  <eric@webkit.org>
166
167         Reviewed by mjs.
168
169         * sunspider: make --shark only sample from testkjs
170
171 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
172
173         Rubber stamped by Eric.
174         
175         - added crypto tests (already balanced)
176
177         * tests/LIST:
178         * tests/crypto-aes.js: Added.
179         * tests/crypto-md5.js: Added.
180         * tests/crypto-sha1.js: Added.
181
182 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
183
184         Reviewed by Adam.
185         
186         - rebalance new tests
187         
188         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
189         
190         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
191
192         This balances them with the existing tests.
193
194         * tests/access-binary-trees.js:
195         * tests/access-nsieve.js:
196         * tests/bitops-nsieve-bits.js:
197         * tests/math-partial-sums.js:
198         * tests/math-spectral-norm.js:
199         * tests/string-fasta.js:
200
201 2007-10-20  Eric Seidel  <eric@webkit.org>
202
203         Reviewed by darin.
204         
205         Add improved argument handling to sunspider, including
206         --runs=<number>, --tests=<pattern, --shark, and --help
207         Also re-factor code into subroutines
208
209         * sunspider: improved argument handling
210
211 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
212
213         Reviewed by Mark.
214
215         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
216         
217         * TODO:
218         * tests/LIST:
219         * tests/access-binary-trees.js: Added.
220         * tests/access-nsieve.js: Added.
221         * tests/bitops-nsieve-bits.js: Added.
222         * tests/math-partial-sums.js: Added.
223         * tests/math-spectral-norm.js: Added.
224         * tests/string-fasta.js: Added.
225
226 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
227
228         Reviewed by Darin.
229
230         * TODO: Updated to note areas that are now well-covered.
231         
232         - Change the tests and the driver to avoid leaving large object
233         graphs hanging around, since that throws off the subsequent tests.
234         
235         * resources/sunspider-standalone-driver.js:
236         * tests/3d-cube.js:
237         * tests/3d-morph.js:
238         * tests/3d-raytrace.js:
239         * tests/bitops-bitwise-and.js:
240         * tests/string-base64.js:
241         * tests/string-tagcloud.js:
242
243 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
244
245         Reviewed by Nikolas.
246         
247         - Rebalance test complexity.
248         
249         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
250         
251         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
252
253         However, for some tests, IE7 was a huge outlier compared to other
254         major browsers. For those tests, I used Firefox 2.0.0.8 on the
255         same OS and hardware as a normalization basis.
256         
257         * tests/3d-morph.js:
258         * tests/3d-raytrace.js:
259         * tests/bitops-3bit-bits-in-byte.js:
260         * tests/bitops-bits-in-byte.js:
261         * tests/bitops-bitwise-and.js:
262         * tests/math-cordic.js:
263         * tests/string-base64.js:
264         * tests/string-tagcloud.js:
265         * tests/string-unpack-code.js:
266
267 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
268
269         Reviewed by Oliver.
270         
271         - Fix the browser-hosted driver to be compatible with Safari 2
272
273         * hosted/sunspider-driver.html:
274
275 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
276
277         Rubber stamped by Hyatt.
278         
279         - Add BSD license notice to cordic test, the original author said to do so.
280
281         * tests/math-cordic.js:
282
283 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
284
285         Reviewed by Eric.
286         
287         - Support Windows and Windows browsers.
288
289         * hosted/sunspider-results.html: Display results correctly in IE.
290         * sunspider: tweaks to work on Windows, and with spaces in the path.
291
292 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
293
294         Rubber stamped by Oliver.
295
296         * tests/string-tagcloud.js: Remove stray print()
297
298 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
299
300         Reviewed by Oliver.
301         
302         Add some string test cases.
303
304         * tests/string-base64.js: Added. Does base64 encoding and decoding.
305         * tests/string-tagcloud.js: Added. Parses a JSON string and
306         generates tagcloud HTML markup based on the data.        
307         * tests/string-unpack-code.js: Added. Unpack compressed versions of
308         four of the most popular JavaScript libraries.
309         * tests/LIST: Added the new tests.
310
311 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
312
313         Reviewed by Anders.
314
315         * TODO: Updated with info on how to do the stats properly.
316
317         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
318         avoid overly slowing down the benchmark.
319
320         Added some 3D-related test cases.
321         
322         * tests/3d-cube.js: Added.
323         * tests/3d-morph.js: Added.
324         * tests/3d-raytrace.js: Added.
325         * tests/LIST:
326
327 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
328
329         Reviewed by Darin.
330
331         - New JavaScript benchmark
332         http://bugs.webkit.org/show_bug.cgi?id=15515
333         
334         A start on a new JavaScript benchmark, with standalone and
335         browser-hosted drivers.
336         
337         The test content is still incomplete and the drivers could use
338         some more features but there is enough here to make a good start.
339         
340         The drivers compute a 95% confidence interval on the mean for the
341         whole test, each category, and each individual test to make it
342         easier to tell whether differences are statistically
343         significant. The confidence interval can be narrowed by running
344         the test more times.
345         
346         * TODO: Added. Remaining things that need to be done.
347         * sunspider: Added. Perl script that acts as the standalone test driver.
348         * resources/sunspider-standalone-driver.js: Added. JavaScript part
349         of standalone test driver.
350         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
351         analysis code.
352         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
353         * make-hosted: Added. Script to generate browser-hosted tests.
354         * hosted/sunspider.html: Added. Start page for browser-hosted test.
355         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
356         * hosted/sunspider-results.html: Added. Results page for browser-hosted
357         tests.
358         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
359         tests.
360         * tests/LIST: Added. List of tests to use.
361         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
362         * tests/bitops-bits-in-byte.js: Added. Ditto.
363         * tests/bitops-bitwise-and.js: Added. Ditto.
364         * tests/math-cordic.js: Added. Ditto.
365