From: fpizlo@apple.com Date: Mon, 29 Feb 2016 19:35:17 +0000 (+0000) Subject: We've been running Octane/regexp all wrong in run-jsc-benchmarks X-Git-Url: http://git.webkit.org/?p=WebKit-https.git;a=commitdiff_plain;h=edd8c5b57a50a7fddf1a77b9b7c15ffa2d5d786f We've been running Octane/regexp all wrong in run-jsc-benchmarks https://bugs.webkit.org/show_bug.cgi?id=154827 Reviewed by Andreas Kling. Octane v.2 and JetStream v.1.1 run this benchmark with warmup. This script was running it without warmup. This patches fixes this by making this script run it with warmup. This fix shows that my last patch, which added FTL support for regexp, was actually a 3% speed-up on Octane/regexp, not a slow-down as the ChangeLog claimed. It discovered this bug because for each Octane test that I want to debug, I usually make a standalone .js file that contains the whole test along with a miniharness - usually a plain loop - that runs it almost like it would for real but with whatever hacks I'm using for debugging. When I wrote such a thing for regexp, I used a ~20 iteration warmup to match the one second of warmup that this benchmark gets in Octane. To my surprise, this quite faithful regexp runner did not see the regression that run-jsc-benchmarks saw. That's when I found out that run-jsc-benchmarks was running it wrong. The reason for the no-warmup slow-down is that the FTL is actually fairly expensive to run on some of these very large functions in the regexp benchmark. I don't think we can do anything about that, and I'd argue that the speed-up we see after the compilation is done suggests that it was worth it. * Scripts/run-jsc-benchmarks: git-svn-id: https://svn.webkit.org/repository/webkit/trunk@197361 268f45cc-cd09-0410-ab3c-d52691b4dbfc --- diff --git a/Tools/ChangeLog b/Tools/ChangeLog index cf56f42..f9d9a36 100644 --- a/Tools/ChangeLog +++ b/Tools/ChangeLog @@ -1,3 +1,31 @@ +2016-02-29 Filip Pizlo + + We've been running Octane/regexp all wrong in run-jsc-benchmarks + https://bugs.webkit.org/show_bug.cgi?id=154827 + + Reviewed by Andreas Kling. + + Octane v.2 and JetStream v.1.1 run this benchmark with warmup. This script was running + it without warmup. This patches fixes this by making this script run it with warmup. + + This fix shows that my last patch, which added FTL support for regexp, was actually a 3% + speed-up on Octane/regexp, not a slow-down as the ChangeLog claimed. + + It discovered this bug because for each Octane test that I want to debug, I usually make + a standalone .js file that contains the whole test along with a miniharness - usually + a plain loop - that runs it almost like it would for real but with whatever hacks I'm + using for debugging. When I wrote such a thing for regexp, I used a ~20 iteration warmup + to match the one second of warmup that this benchmark gets in Octane. To my surprise, + this quite faithful regexp runner did not see the regression that run-jsc-benchmarks + saw. That's when I found out that run-jsc-benchmarks was running it wrong. + + The reason for the no-warmup slow-down is that the FTL is actually fairly expensive to + run on some of these very large functions in the regexp benchmark. I don't think we can + do anything about that, and I'd argue that the speed-up we see after the compilation is + done suggests that it was worth it. + + * Scripts/run-jsc-benchmarks: + 2016-02-29 Gavin Barraclough RefCounter::Event -> RefCounterEvent diff --git a/Tools/Scripts/run-jsc-benchmarks b/Tools/Scripts/run-jsc-benchmarks index d07ea2a..a8aaf5b 100755 --- a/Tools/Scripts/run-jsc-benchmarks +++ b/Tools/Scripts/run-jsc-benchmarks @@ -2930,7 +2930,7 @@ begin [["raytrace"], "raytrace", 2, true, false, 32], [["richards"], "richards", 2, true, false, 32], [["splay"], "splay", 2, true, false, 32], - [["regexp"], "regexp", 2, false, false, 16], + [["regexp"], "regexp", 2, true, false, 16], [["pdfjs"], "pdfjs", 2, false, false, 4], [["mandreel"], "mandreel", 2, false, false, 4], [["gbemu-part1", "gbemu-part2"], "gbemu", 2, false, false, 4],