buildbot syncing scripts sometimes schedule more than one requests per builder

https://bugs.webkit.org/show_bug.cgi?id=153047

Reviewed by Chris Dumez.

The bug was caused by the check for singularity of scheduledRequests being conducted per configuration
instead of per builder. So if there were multiple test configurations (e.g. Speedometer and Octane) that
both used the same builder, then we may end up scheduling both at once.

Fixed the bug by sharing a single set to keep track of the scheduled requests for all configurations per
builder.

* tools/sync-with-buildbot.py:
(load_config): Share a set amongst test configurations for each builder.
(find_request_updates): Instead of creating a new set for each configuration, reuse the existing sets to
share a single set agmonst test configurations for each builder.

git-svn-id: http://svn.webkit.org/repository/webkit/trunk@194935 268f45cc-cd09-0410-ab3c-d52691b4dbfc

diff --git a/Websites/perf.webkit.org/ChangeLog b/Websites/perf.webkit.org/ChangeLog
index 0b570be..26c3cc7 100644 (file)
--- a/Websites/perf.webkit.org/ChangeLog
+++ b/Websites/perf.webkit.org/ChangeLog
@@ -1,5 +1,24 @@
 2016-01-12  Ryosuke Niwa  <rniwa@webkit.org>
 
+        buildbot syncing scripts sometimes schedule more than one requests per builder
+        https://bugs.webkit.org/show_bug.cgi?id=153047
+
+        Reviewed by Chris Dumez.
+
+        The bug was caused by the check for singularity of scheduledRequests being conducted per configuration
+        instead of per builder. So if there were multiple test configurations (e.g. Speedometer and Octane) that
+        both used the same builder, then we may end up scheduling both at once.
+
+        Fixed the bug by sharing a single set to keep track of the scheduled requests for all configurations per
+        builder.
+
+        * tools/sync-with-buildbot.py:
+        (load_config): Share a set amongst test configurations for each builder.
+        (find_request_updates): Instead of creating a new set for each configuration, reuse the existing sets to
+        share a single set agmonst test configurations for each builder.
+
+2016-01-12  Ryosuke Niwa  <rniwa@webkit.org>
+
         Analysis results viewer sometimes doesn't show the correct relative difference
         https://bugs.webkit.org/show_bug.cgi?id=152930
 

diff --git a/Websites/perf.webkit.org/tools/sync-with-buildbot.py b/Websites/perf.webkit.org/tools/sync-with-buildbot.py
index 76d14d4..f44125e 100755 (executable)
--- a/Websites/perf.webkit.org/tools/sync-with-buildbot.py
+++ b/Websites/perf.webkit.org/tools/sync-with-buildbot.py
@@ -79,6 +79,8 @@ def load_config(config_json_path, buildbot_url):
             else:
                 config[key] = value
 
+    scheduled_requests_by_builder = {}
+
     configurations = options['configurations']
     for config in configurations:
 
@@ -89,7 +91,9 @@ def load_config(config_json_path, buildbot_url):
         escaped_builder_name = urllib.quote(config['builder'])
         config['url'] = '%s/builders/%s/' % (buildbot_url, escaped_builder_name)
         config['jsonURL'] = '%s/json/builders/%s/' % (buildbot_url, escaped_builder_name)
-        config['scheduledRequests'] = set()
+
+        scheduled_requests_by_builder.setdefault(config['builder'], set())
+        config['scheduledRequests'] = scheduled_requests_by_builder[config['builder']]
 
     return configurations
 
@@ -98,12 +102,15 @@ def find_request_updates(configurations, lookback_count):
     request_updates = {}
 
     for config in configurations:
+        config['scheduledRequests'].clear()
+
+    for config in configurations:
         try:
             pending_builds = fetch_json(config['jsonURL'] + 'pendingBuilds')
             scheduled_requests = filter(None, [request_id_from_build(config, build) for build in pending_builds])
             for request_id in scheduled_requests:
                 request_updates[request_id] = {'status': 'scheduled', 'url': config['url']}
-            config['scheduledRequests'] = set(scheduled_requests)
+                config['scheduledRequests'].add(request_id)
         except (IOError, ValueError) as error:
             print >> sys.stderr, "Failed to fetch pending builds for %s: %s" % (config['builder'], str(error))