2008-09-02 Cameron Zwarich <cwzwarich@uwaterloo.ca>
[WebKit-https.git] / SunSpider / ChangeLog
1 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
2
3         Not reviewed.
4
5         Remove a local change that was accidentally committed in r36034.
6
7         * resources/sunspider-standalone-driver.js:
8
9 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
10
11         Rubber-stamped by Oliver Hunt.
12
13         Make SunSpider work with JavaScript shells that don't take filenames
14         after a '-f' argument.
15
16         * sunspider:
17
18 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
19
20         Reviewed by Timothy.
21
22         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
23
24         * sunspider:
25         * tests/LIST-SQUIRRELFISH: Removed.
26
27 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
28
29         Rubber-stamped by Mark Rowe.
30
31         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
32
33         * sunspider:
34
35 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
36
37         Rubber stamped by Oliver.
38         
39         - add newly working tests to squirrelfish list
40
41         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
42
43 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
44
45         Rubber stamped by Oliver.
46         
47         - move ubench tests properly.
48
49         * tests/ubench: Added.
50         * tests/ubench/function-closure.js: Added.
51         * tests/ubench/function-empty.js: Added.
52         * tests/ubench/function-missing-args.js: Added.
53         * tests/ubench/function-sum.js: Added.
54         * tests/ubench/loop-empty-resolve.js: Added.
55         * tests/ubench/loop-empty.js: Added.
56         * tests/ubench/loop-sum.js: Added.
57
58 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
59
60         Rubber stamped by Oliver.
61
62         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
63
64 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
65
66         Reviewed by Oliver.
67
68         - make --squirrelfish mode use the real harness, but a reduced set of tests
69
70         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
71
72         * sunspider:
73         * tests/LIST-SQUIRRELFISH:
74         * tests/LIST-UBENCH: Added.
75         * tests/squirrelfish: Moved to tests/ubench
76
77 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
78
79         Reviewed by Maciej Stachowiak.
80         
81         Removed an unused test.
82
83         * tests/squirrelfish/loop-resolve.js: Removed.
84
85 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
86
87         Reviewed by Oliver.
88         
89         Add newly runnable tests to --squirrelfish mode.
90
91         * tests/LIST-SQUIRRELFISH:
92
93 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
94
95         Reviewed by Maciej Stachowiak.
96         
97         A little more love for --squirrelfish mode:
98         
99         Fixed a misplaced sort. Results properly sort now.
100         
101         Added a test list just for squirrelfish, and updated it to include
102         all currently passing SunSpider tests.
103         
104         Fixed the pruning regexp to match 3d-morph.js.
105
106         * sunspider:
107
108 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
109
110         Reviewed by Oliver Hunt.
111         
112         --squirrelfish mode: pared down tests for squirrelfish to chew on.
113
114         * sunspider:
115         * tests/LIST:
116         * tests/squirrelfish: Added.
117         * tests/squirrelfish/function-closure.js: Added.
118         * tests/squirrelfish/function-empty.js: Added.
119         * tests/squirrelfish/function-missing-args.js: Added.
120         * tests/squirrelfish/function-sum.js: Added.
121         * tests/squirrelfish/loop-empty-resolve.js: Added.
122         * tests/squirrelfish/loop-empty.js: Added.
123         * tests/squirrelfish/loop-resolve.js: Added.
124         * tests/squirrelfish/loop-sum.js: Added.
125
126 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
127
128         Reviewed by Geoff.
129
130         * sunspider: Pause update daemon on Darwin to make results reliable
131         again (workaround for <rdar://problem/5811127>).
132
133 2008-02-07  Eric Seidel  <eric@webkit.org>
134
135         Reviewed by Sam.
136
137         * sunspider: add an error message when ./tests/LIST is not found
138
139 2007-12-29  Alp Toker  <alp@atoker.com>
140
141         Fix typo.
142
143         * hosted/sunspider.html:
144
145 2007-12-29  Alp Toker  <alp@atoker.com>
146
147         Reviewed by Maciej.
148
149         Provide the current URL in the SunSpider results page. Useful for
150         copying benchmark results out of browsers without a location bar or
151         where the location bar can't handle long URLs.
152
153         * hosted/sunspider-results.html:
154
155 2007-12-20  Eric Seidel  <eric@webkit.org>
156
157         Reviewed by mjs.
158
159         * sunspider: increase default number of runs for more accurate numbers
160
161 2007-12-18  Mark Rowe  <mrowe@apple.com>
162
163         Fix typo.
164
165         * hosted/sunspider.html:
166
167 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
168
169         Reviewed by Eric.
170         
171         - give an extra digit of precision for small differences.
172
173         * resources/sunspider-compare-results.js:
174
175 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
176
177         Reviewed by Eric.
178         
179         - work around IE layout bugs
180
181         * hosted/sunspider.css:
182
183 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
184
185         Reviewed by Mark.
186         
187         - fix compare feature
188
189         * hosted/sunspider-results.html:
190
191 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
192
193         Reviewed by Mark.
194         
195         - Change the format of the output URL to be shorter, to meet IE constraints
196
197         * hosted/sunspider-driver.html:
198         * hosted/sunspider-results.html:
199
200 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
201
202         Reviewed by Oliver.
203         
204         - Add some explanatory test and a dumb logo to SunSpider web hosted version
205
206         * hosted/sunspider-driver.html:
207         * hosted/sunspider-results.html:
208         * hosted/sunspider.css:
209         * hosted/sunspider.html:
210
211 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
212
213         Reviewed by Oliver.
214         
215         - Make the web-hosted version of SunSpider look a little bit less drab.
216
217         * hosted/sunspider-driver.html:
218         * hosted/sunspider-results.html:
219         * hosted/sunspider.css: Added.
220         * hosted/sunspider.html:
221         * resources/TEMPLATE.html:
222
223 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
224
225         Reviewed by Sam.
226         
227         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
228         
229         A sample of what new-style results would look like:
230         
231         TEST                   COMPARISON            FROM                 TO             DETAILS
232         ...
233         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
234         ...
235             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
236         
237         * resources/sunspider-compare-results.js:
238
239 2007-11-14  Eric Seidel  <eric@webkit.org>
240
241         Reviewed by Sam.
242
243         * sunspider: add --shark-cache for L2 Cache Miss profiling
244
245 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
246
247         Reviewed by Darin.
248
249         http://bugs.webkit.org/show_bug.cgi?id=15925
250         SunSpider should check for Shark being installed
251
252         * sunspider: Explicitly test that shark command line tool is installed.
253
254 2007-11-04  Darin Adler  <darin@apple.com>
255
256         * hosted/sunspider.html: Tweaked formatting a bit.
257
258 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
259
260         Reviewed by Darin.
261
262         - Add compare mode and a description of the benchmark to the hosted version.
263
264         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
265         * hosted/sunspider.html: Added intro text.
266         * make-hosted: Copy new file.
267         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
268         a function.
269         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
270         expects.
271         * sunspider-compare-results: Use sunspider-standalone-compare.js.
272
273 2007-11-02  Darin Adler  <darin@apple.com>
274
275         * sunspider: Changed "--base" to "--set-baseline".
276
277 2007-11-02  Darin Adler  <darin@apple.com>
278
279         Reviewed by Maciej.
280
281         * sunspider: Added a "--base" option that records the current test as a baseline.
282         Moved the code to find the newest .mshark file into a function (since I needed to use
283         it in the other script and I prefer to copy/paste a function).
284
285         * sunspider-compare-results: Added logic to compare the baseline with the last
286         run when you don't pass any parameters.
287
288 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
289
290         Reviewed by Eric.
291         
292         - Added list of ECMAScript3 features that SunSpider does not cover currently.
293
294         * UNCOVERED: Added.
295
296 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
297
298         Reviewed by Oliver.
299         
300         - Give better error messages when sunspider-compare-results fails
301
302         * sunspider-compare-results:
303
304 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
305
306         Reviewed by Darin.
307         
308         - fix Date tests to work in command-line SpiderMonkey
309
310         * tests/date-format-tofte.js:
311         * tests/date-format-xparb.js:
312
313 2007-10-29  Darin Adler  <darin@apple.com>
314
315         Reviewed by Maciej.
316
317         * sunspider: Added --shark20 option, to run Shark at its highest sample
318         resolution instead of its default.
319
320 2007-10-28  Adam Roben  <aroben@apple.com>
321
322         Fix SunSpider on Windows
323
324         Reviewed by Eric.
325
326         * sunspider: Don't use colons in filenames.
327
328 2007-10-28  Darin Adler  <darin@apple.com>
329
330         Reviewed by Adam.
331
332         * resources/sunspider-analyze-results.js: Tweak the output format a little.
333         Change so that when there's only one run we don't write out confidence
334         intervals at all rather than writing "NaN" over and over again.
335
336         * sunspider: Use the time and date as a suffix on the results file, that way
337         all the results are left behind in the tmp directory. This will make it easy
338         for us to add features that compare past results in the future. Also rename
339         the .mshark files using the same scheme.
340
341         * sunspider-compare-results: Relax the parsing rules so we can parse the
342         results file exactly as it's emitted from sunspider.
343
344 2007-10-25  Adam Roben  <aroben@apple.com>
345
346         Strip carriage returns from results
347
348         Carriage returns were screwing up sunspider-compare-results on
349         Windows.
350
351         Rubberstamped by Oliver.
352
353         * sunspider:
354
355 2007-10-24  Darin Adler  <darin@apple.com>
356
357         Reviewed by Eric.
358
359         * sunspider: Open the profile in shark after completing a run.
360         * sunspider-compare-results: Tiny perl tweaks.
361
362 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
363
364         Reviewed by Darin.
365
366         - Don't hardcode my path to testkjs
367         
368         * sunspider-compare-results:
369
370 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
371
372         Reviewed by Darin.
373         
374         - Add a compare script to compare results, and improve formatting of standard results a bit.
375
376         * resources/sunspider-analyze-results.js:
377         * resources/sunspider-compare-results.js: Added.
378         * sunspider-compare-results: Added.
379
380 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
381
382         Reviewed by Darin.
383
384         * resources/sunspider-analyze-results.js: Do the error range
385         properly, using the t-distribution instead of 1.96 (which was
386         based on the normal distribution).
387         
388         * sunspider: Print results in a way that will be friendlier to the
389         soon-to-come compare mode.
390
391 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
392
393         Reviewed by Review.
394         
395         - add more tests (probably the final set for now)
396         
397         Already balanced. Added date, regexp, control flow, and a few
398         more string and object/array access tests.
399         
400         * tests/LIST:
401         * tests/access-fannkuch.js: Added.
402         * tests/access-nbody.js: Added.
403         * tests/controlflow-recursive.js: Added.
404         * tests/date-format-tofte.js: Added.
405         * tests/date-format-xparb.js: Added.
406         * tests/regexp-dna.js: Added.
407         * tests/string-validate-input.js: Added.
408
409 2007-10-21  Eric Seidel  <eric@webkit.org>
410
411         Reviewed by mjs.
412
413         * sunspider: make --shark only sample from testkjs
414
415 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
416
417         Rubber stamped by Eric.
418         
419         - added crypto tests (already balanced)
420
421         * tests/LIST:
422         * tests/crypto-aes.js: Added.
423         * tests/crypto-md5.js: Added.
424         * tests/crypto-sha1.js: Added.
425
426 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
427
428         Reviewed by Adam.
429         
430         - rebalance new tests
431         
432         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
433         
434         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
435
436         This balances them with the existing tests.
437
438         * tests/access-binary-trees.js:
439         * tests/access-nsieve.js:
440         * tests/bitops-nsieve-bits.js:
441         * tests/math-partial-sums.js:
442         * tests/math-spectral-norm.js:
443         * tests/string-fasta.js:
444
445 2007-10-20  Eric Seidel  <eric@webkit.org>
446
447         Reviewed by darin.
448         
449         Add improved argument handling to sunspider, including
450         --runs=<number>, --tests=<pattern, --shark, and --help
451         Also re-factor code into subroutines
452
453         * sunspider: improved argument handling
454
455 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
456
457         Reviewed by Mark.
458
459         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
460         
461         * TODO:
462         * tests/LIST:
463         * tests/access-binary-trees.js: Added.
464         * tests/access-nsieve.js: Added.
465         * tests/bitops-nsieve-bits.js: Added.
466         * tests/math-partial-sums.js: Added.
467         * tests/math-spectral-norm.js: Added.
468         * tests/string-fasta.js: Added.
469
470 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
471
472         Reviewed by Darin.
473
474         * TODO: Updated to note areas that are now well-covered.
475         
476         - Change the tests and the driver to avoid leaving large object
477         graphs hanging around, since that throws off the subsequent tests.
478         
479         * resources/sunspider-standalone-driver.js:
480         * tests/3d-cube.js:
481         * tests/3d-morph.js:
482         * tests/3d-raytrace.js:
483         * tests/bitops-bitwise-and.js:
484         * tests/string-base64.js:
485         * tests/string-tagcloud.js:
486
487 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
488
489         Reviewed by Nikolas.
490         
491         - Rebalance test complexity.
492         
493         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
494         
495         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
496
497         However, for some tests, IE7 was a huge outlier compared to other
498         major browsers. For those tests, I used Firefox 2.0.0.8 on the
499         same OS and hardware as a normalization basis.
500         
501         * tests/3d-morph.js:
502         * tests/3d-raytrace.js:
503         * tests/bitops-3bit-bits-in-byte.js:
504         * tests/bitops-bits-in-byte.js:
505         * tests/bitops-bitwise-and.js:
506         * tests/math-cordic.js:
507         * tests/string-base64.js:
508         * tests/string-tagcloud.js:
509         * tests/string-unpack-code.js:
510
511 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
512
513         Reviewed by Oliver.
514         
515         - Fix the browser-hosted driver to be compatible with Safari 2
516
517         * hosted/sunspider-driver.html:
518
519 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
520
521         Rubber stamped by Hyatt.
522         
523         - Add BSD license notice to cordic test, the original author said to do so.
524
525         * tests/math-cordic.js:
526
527 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
528
529         Reviewed by Eric.
530         
531         - Support Windows and Windows browsers.
532
533         * hosted/sunspider-results.html: Display results correctly in IE.
534         * sunspider: tweaks to work on Windows, and with spaces in the path.
535
536 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
537
538         Rubber stamped by Oliver.
539
540         * tests/string-tagcloud.js: Remove stray print()
541
542 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
543
544         Reviewed by Oliver.
545         
546         Add some string test cases.
547
548         * tests/string-base64.js: Added. Does base64 encoding and decoding.
549         * tests/string-tagcloud.js: Added. Parses a JSON string and
550         generates tagcloud HTML markup based on the data.        
551         * tests/string-unpack-code.js: Added. Unpack compressed versions of
552         four of the most popular JavaScript libraries.
553         * tests/LIST: Added the new tests.
554
555 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
556
557         Reviewed by Anders.
558
559         * TODO: Updated with info on how to do the stats properly.
560
561         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
562         avoid overly slowing down the benchmark.
563
564         Added some 3D-related test cases.
565         
566         * tests/3d-cube.js: Added.
567         * tests/3d-morph.js: Added.
568         * tests/3d-raytrace.js: Added.
569         * tests/LIST:
570
571 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
572
573         Reviewed by Darin.
574
575         - New JavaScript benchmark
576         http://bugs.webkit.org/show_bug.cgi?id=15515
577         
578         A start on a new JavaScript benchmark, with standalone and
579         browser-hosted drivers.
580         
581         The test content is still incomplete and the drivers could use
582         some more features but there is enough here to make a good start.
583         
584         The drivers compute a 95% confidence interval on the mean for the
585         whole test, each category, and each individual test to make it
586         easier to tell whether differences are statistically
587         significant. The confidence interval can be narrowed by running
588         the test more times.
589         
590         * TODO: Added. Remaining things that need to be done.
591         * sunspider: Added. Perl script that acts as the standalone test driver.
592         * resources/sunspider-standalone-driver.js: Added. JavaScript part
593         of standalone test driver.
594         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
595         analysis code.
596         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
597         * make-hosted: Added. Script to generate browser-hosted tests.
598         * hosted/sunspider.html: Added. Start page for browser-hosted test.
599         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
600         * hosted/sunspider-results.html: Added. Results page for browser-hosted
601         tests.
602         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
603         tests.
604         * tests/LIST: Added. List of tests to use.
605         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
606         * tests/bitops-bits-in-byte.js: Added. Ditto.
607         * tests/bitops-bitwise-and.js: Added. Ditto.
608         * tests/math-cordic.js: Added. Ditto.
609