Fix Windows debug build for opensource developers.
[WebKit-https.git] / SunSpider / ChangeLog
1 2007-12-29  Alp Toker  <alp@atoker.com>
2
3         Fix typo.
4
5         * hosted/sunspider.html:
6
7 2007-12-29  Alp Toker  <alp@atoker.com>
8
9         Reviewed by Maciej.
10
11         Provide the current URL in the SunSpider results page. Useful for
12         copying benchmark results out of browsers without a location bar or
13         where the location bar can't handle long URLs.
14
15         * hosted/sunspider-results.html:
16
17 2007-12-20  Eric Seidel  <eric@webkit.org>
18
19         Reviewed by mjs.
20
21         * sunspider: increase default number of runs for more accurate numbers
22
23 2007-12-18  Mark Rowe  <mrowe@apple.com>
24
25         Fix typo.
26
27         * hosted/sunspider.html:
28
29 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
30
31         Reviewed by Eric.
32         
33         - give an extra digit of precision for small differences.
34
35         * resources/sunspider-compare-results.js:
36
37 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
38
39         Reviewed by Eric.
40         
41         - work around IE layout bugs
42
43         * hosted/sunspider.css:
44
45 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
46
47         Reviewed by Mark.
48         
49         - fix compare feature
50
51         * hosted/sunspider-results.html:
52
53 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
54
55         Reviewed by Mark.
56         
57         - Change the format of the output URL to be shorter, to meet IE constraints
58
59         * hosted/sunspider-driver.html:
60         * hosted/sunspider-results.html:
61
62 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
63
64         Reviewed by Oliver.
65         
66         - Add some explanatory test and a dumb logo to SunSpider web hosted version
67
68         * hosted/sunspider-driver.html:
69         * hosted/sunspider-results.html:
70         * hosted/sunspider.css:
71         * hosted/sunspider.html:
72
73 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
74
75         Reviewed by Oliver.
76         
77         - Make the web-hosted version of SunSpider look a little bit less drab.
78
79         * hosted/sunspider-driver.html:
80         * hosted/sunspider-results.html:
81         * hosted/sunspider.css: Added.
82         * hosted/sunspider.html:
83         * resources/TEMPLATE.html:
84
85 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
86
87         Reviewed by Sam.
88         
89         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
90         
91         A sample of what new-style results would look like:
92         
93         TEST                   COMPARISON            FROM                 TO             DETAILS
94         ...
95         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
96         ...
97             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
98         
99         * resources/sunspider-compare-results.js:
100
101 2007-11-14  Eric Seidel  <eric@webkit.org>
102
103         Reviewed by Sam.
104
105         * sunspider: add --shark-cache for L2 Cache Miss profiling
106
107 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
108
109         Reviewed by Darin.
110
111         http://bugs.webkit.org/show_bug.cgi?id=15925
112         SunSpider should check for Shark being installed
113
114         * sunspider: Explicitly test that shark command line tool is installed.
115
116 2007-11-04  Darin Adler  <darin@apple.com>
117
118         * hosted/sunspider.html: Tweaked formatting a bit.
119
120 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
121
122         Reviewed by Darin.
123
124         - Add compare mode and a description of the benchmark to the hosted version.
125
126         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
127         * hosted/sunspider.html: Added intro text.
128         * make-hosted: Copy new file.
129         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
130         a function.
131         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
132         expects.
133         * sunspider-compare-results: Use sunspider-standalone-compare.js.
134
135 2007-11-02  Darin Adler  <darin@apple.com>
136
137         * sunspider: Changed "--base" to "--set-baseline".
138
139 2007-11-02  Darin Adler  <darin@apple.com>
140
141         Reviewed by Maciej.
142
143         * sunspider: Added a "--base" option that records the current test as a baseline.
144         Moved the code to find the newest .mshark file into a function (since I needed to use
145         it in the other script and I prefer to copy/paste a function).
146
147         * sunspider-compare-results: Added logic to compare the baseline with the last
148         run when you don't pass any parameters.
149
150 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
151
152         Reviewed by Eric.
153         
154         - Added list of ECMAScript3 features that SunSpider does not cover currently.
155
156         * UNCOVERED: Added.
157
158 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
159
160         Reviewed by Oliver.
161         
162         - Give better error messages when sunspider-compare-results fails
163
164         * sunspider-compare-results:
165
166 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
167
168         Reviewed by Darin.
169         
170         - fix Date tests to work in command-line SpiderMonkey
171
172         * tests/date-format-tofte.js:
173         * tests/date-format-xparb.js:
174
175 2007-10-29  Darin Adler  <darin@apple.com>
176
177         Reviewed by Maciej.
178
179         * sunspider: Added --shark20 option, to run Shark at its highest sample
180         resolution instead of its default.
181
182 2007-10-28  Adam Roben  <aroben@apple.com>
183
184         Fix SunSpider on Windows
185
186         Reviewed by Eric.
187
188         * sunspider: Don't use colons in filenames.
189
190 2007-10-28  Darin Adler  <darin@apple.com>
191
192         Reviewed by Adam.
193
194         * resources/sunspider-analyze-results.js: Tweak the output format a little.
195         Change so that when there's only one run we don't write out confidence
196         intervals at all rather than writing "NaN" over and over again.
197
198         * sunspider: Use the time and date as a suffix on the results file, that way
199         all the results are left behind in the tmp directory. This will make it easy
200         for us to add features that compare past results in the future. Also rename
201         the .mshark files using the same scheme.
202
203         * sunspider-compare-results: Relax the parsing rules so we can parse the
204         results file exactly as it's emitted from sunspider.
205
206 2007-10-25  Adam Roben  <aroben@apple.com>
207
208         Strip carriage returns from results
209
210         Carriage returns were screwing up sunspider-compare-results on
211         Windows.
212
213         Rubberstamped by Oliver.
214
215         * sunspider:
216
217 2007-10-24  Darin Adler  <darin@apple.com>
218
219         Reviewed by Eric.
220
221         * sunspider: Open the profile in shark after completing a run.
222         * sunspider-compare-results: Tiny perl tweaks.
223
224 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
225
226         Reviewed by Darin.
227
228         - Don't hardcode my path to testkjs
229         
230         * sunspider-compare-results:
231
232 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
233
234         Reviewed by Darin.
235         
236         - Add a compare script to compare results, and improve formatting of standard results a bit.
237
238         * resources/sunspider-analyze-results.js:
239         * resources/sunspider-compare-results.js: Added.
240         * sunspider-compare-results: Added.
241
242 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
243
244         Reviewed by Darin.
245
246         * resources/sunspider-analyze-results.js: Do the error range
247         properly, using the t-distribution instead of 1.96 (which was
248         based on the normal distribution).
249         
250         * sunspider: Print results in a way that will be friendlier to the
251         soon-to-come compare mode.
252
253 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
254
255         Reviewed by Review.
256         
257         - add more tests (probably the final set for now)
258         
259         Already balanced. Added date, regexp, control flow, and a few
260         more string and object/array access tests.
261         
262         * tests/LIST:
263         * tests/access-fannkuch.js: Added.
264         * tests/access-nbody.js: Added.
265         * tests/controlflow-recursive.js: Added.
266         * tests/date-format-tofte.js: Added.
267         * tests/date-format-xparb.js: Added.
268         * tests/regexp-dna.js: Added.
269         * tests/string-validate-input.js: Added.
270
271 2007-10-21  Eric Seidel  <eric@webkit.org>
272
273         Reviewed by mjs.
274
275         * sunspider: make --shark only sample from testkjs
276
277 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
278
279         Rubber stamped by Eric.
280         
281         - added crypto tests (already balanced)
282
283         * tests/LIST:
284         * tests/crypto-aes.js: Added.
285         * tests/crypto-md5.js: Added.
286         * tests/crypto-sha1.js: Added.
287
288 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
289
290         Reviewed by Adam.
291         
292         - rebalance new tests
293         
294         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
295         
296         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
297
298         This balances them with the existing tests.
299
300         * tests/access-binary-trees.js:
301         * tests/access-nsieve.js:
302         * tests/bitops-nsieve-bits.js:
303         * tests/math-partial-sums.js:
304         * tests/math-spectral-norm.js:
305         * tests/string-fasta.js:
306
307 2007-10-20  Eric Seidel  <eric@webkit.org>
308
309         Reviewed by darin.
310         
311         Add improved argument handling to sunspider, including
312         --runs=<number>, --tests=<pattern, --shark, and --help
313         Also re-factor code into subroutines
314
315         * sunspider: improved argument handling
316
317 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
318
319         Reviewed by Mark.
320
321         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
322         
323         * TODO:
324         * tests/LIST:
325         * tests/access-binary-trees.js: Added.
326         * tests/access-nsieve.js: Added.
327         * tests/bitops-nsieve-bits.js: Added.
328         * tests/math-partial-sums.js: Added.
329         * tests/math-spectral-norm.js: Added.
330         * tests/string-fasta.js: Added.
331
332 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
333
334         Reviewed by Darin.
335
336         * TODO: Updated to note areas that are now well-covered.
337         
338         - Change the tests and the driver to avoid leaving large object
339         graphs hanging around, since that throws off the subsequent tests.
340         
341         * resources/sunspider-standalone-driver.js:
342         * tests/3d-cube.js:
343         * tests/3d-morph.js:
344         * tests/3d-raytrace.js:
345         * tests/bitops-bitwise-and.js:
346         * tests/string-base64.js:
347         * tests/string-tagcloud.js:
348
349 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
350
351         Reviewed by Nikolas.
352         
353         - Rebalance test complexity.
354         
355         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
356         
357         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
358
359         However, for some tests, IE7 was a huge outlier compared to other
360         major browsers. For those tests, I used Firefox 2.0.0.8 on the
361         same OS and hardware as a normalization basis.
362         
363         * tests/3d-morph.js:
364         * tests/3d-raytrace.js:
365         * tests/bitops-3bit-bits-in-byte.js:
366         * tests/bitops-bits-in-byte.js:
367         * tests/bitops-bitwise-and.js:
368         * tests/math-cordic.js:
369         * tests/string-base64.js:
370         * tests/string-tagcloud.js:
371         * tests/string-unpack-code.js:
372
373 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
374
375         Reviewed by Oliver.
376         
377         - Fix the browser-hosted driver to be compatible with Safari 2
378
379         * hosted/sunspider-driver.html:
380
381 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
382
383         Rubber stamped by Hyatt.
384         
385         - Add BSD license notice to cordic test, the original author said to do so.
386
387         * tests/math-cordic.js:
388
389 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
390
391         Reviewed by Eric.
392         
393         - Support Windows and Windows browsers.
394
395         * hosted/sunspider-results.html: Display results correctly in IE.
396         * sunspider: tweaks to work on Windows, and with spaces in the path.
397
398 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
399
400         Rubber stamped by Oliver.
401
402         * tests/string-tagcloud.js: Remove stray print()
403
404 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
405
406         Reviewed by Oliver.
407         
408         Add some string test cases.
409
410         * tests/string-base64.js: Added. Does base64 encoding and decoding.
411         * tests/string-tagcloud.js: Added. Parses a JSON string and
412         generates tagcloud HTML markup based on the data.        
413         * tests/string-unpack-code.js: Added. Unpack compressed versions of
414         four of the most popular JavaScript libraries.
415         * tests/LIST: Added the new tests.
416
417 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
418
419         Reviewed by Anders.
420
421         * TODO: Updated with info on how to do the stats properly.
422
423         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
424         avoid overly slowing down the benchmark.
425
426         Added some 3D-related test cases.
427         
428         * tests/3d-cube.js: Added.
429         * tests/3d-morph.js: Added.
430         * tests/3d-raytrace.js: Added.
431         * tests/LIST:
432
433 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
434
435         Reviewed by Darin.
436
437         - New JavaScript benchmark
438         http://bugs.webkit.org/show_bug.cgi?id=15515
439         
440         A start on a new JavaScript benchmark, with standalone and
441         browser-hosted drivers.
442         
443         The test content is still incomplete and the drivers could use
444         some more features but there is enough here to make a good start.
445         
446         The drivers compute a 95% confidence interval on the mean for the
447         whole test, each category, and each individual test to make it
448         easier to tell whether differences are statistically
449         significant. The confidence interval can be narrowed by running
450         the test more times.
451         
452         * TODO: Added. Remaining things that need to be done.
453         * sunspider: Added. Perl script that acts as the standalone test driver.
454         * resources/sunspider-standalone-driver.js: Added. JavaScript part
455         of standalone test driver.
456         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
457         analysis code.
458         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
459         * make-hosted: Added. Script to generate browser-hosted tests.
460         * hosted/sunspider.html: Added. Start page for browser-hosted test.
461         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
462         * hosted/sunspider-results.html: Added. Results page for browser-hosted
463         tests.
464         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
465         tests.
466         * tests/LIST: Added. List of tests to use.
467         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
468         * tests/bitops-bits-in-byte.js: Added. Ditto.
469         * tests/bitops-bitwise-and.js: Added. Ditto.
470         * tests/math-cordic.js: Added. Ditto.
471