update-test-expectations-from-bugzilla tool not working with new EWS
authorclopez@igalia.com <clopez@igalia.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 20 May 2020 13:29:09 +0000 (13:29 +0000)
committerclopez@igalia.com <clopez@igalia.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 20 May 2020 13:29:09 +0000 (13:29 +0000)
https://bugs.webkit.org/show_bug.cgi?id=210975

Reviewed by Youenn Fablet.

With the new EWS, the layout test archive results are not longer
posted to bugzilla as attachment. Now we need to inspect the EWS
server to get the status of the runs for the patch id, and then
query the EWS builbot server in order to get the details of each
run to finally obtain the URL with the zip file for the results.

The tool now automatically applies platform-specific and generic
results automatically (its not longer needed to specify whether
the result its generic or not). It uses mac-wk2 results as generic.

Also now it updates the test results for tests where the result
is MISSING.

* Scripts/webkitpy/common/net/bugzilla/test_expectation_updater.py:
(configure_logging):
(argument_parser):
(TestExpectationUpdater.__init__):
(TestExpectationUpdater._platform_name):
(TestExpectationUpdater):
(TestExpectationUpdater._get_layout_tests_run):
(TestExpectationUpdater._lookup_ews_results):
(TestExpectationUpdater._tests_to_update):
(TestExpectationUpdater._update_for_generic_bot):
(TestExpectationUpdater._update_for_platform_specific_bot):
(TestExpectationUpdater.do_update):
(main):
* Scripts/webkitpy/common/net/bugzilla/test_expectation_updater_unittest.py:
(MockAttachment.__init__):
(MockAttachment.is_patch):
(MockAttachment):
(MockAttachment.is_obsolete):
(MockBugzilla):
(MockBugzilla.attachments):
(MockRequests):
(MockRequests.__init__):
(MockRequests.get):
(MockRequests.content):
(MockRequests.text):
(MockZip.__init__):
(TestExpectationUpdaterTest.test_update_test_expectations):

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@261914 268f45cc-cd09-0410-ab3c-d52691b4dbfc

Tools/ChangeLog
Tools/Scripts/webkitpy/common/net/bugzilla/test_expectation_updater.py
Tools/Scripts/webkitpy/common/net/bugzilla/test_expectation_updater_unittest.py

index c8a19ae..b819d28 100644 (file)
@@ -1,3 +1,51 @@
+2020-05-20  Carlos Alberto Lopez Perez  <clopez@igalia.com>
+
+        update-test-expectations-from-bugzilla tool not working with new EWS
+        https://bugs.webkit.org/show_bug.cgi?id=210975
+
+        Reviewed by Youenn Fablet.
+
+        With the new EWS, the layout test archive results are not longer
+        posted to bugzilla as attachment. Now we need to inspect the EWS
+        server to get the status of the runs for the patch id, and then
+        query the EWS builbot server in order to get the details of each
+        run to finally obtain the URL with the zip file for the results.
+
+        The tool now automatically applies platform-specific and generic
+        results automatically (its not longer needed to specify whether
+        the result its generic or not). It uses mac-wk2 results as generic.
+
+        Also now it updates the test results for tests where the result
+        is MISSING.
+
+        * Scripts/webkitpy/common/net/bugzilla/test_expectation_updater.py:
+        (configure_logging):
+        (argument_parser):
+        (TestExpectationUpdater.__init__):
+        (TestExpectationUpdater._platform_name):
+        (TestExpectationUpdater):
+        (TestExpectationUpdater._get_layout_tests_run):
+        (TestExpectationUpdater._lookup_ews_results):
+        (TestExpectationUpdater._tests_to_update):
+        (TestExpectationUpdater._update_for_generic_bot):
+        (TestExpectationUpdater._update_for_platform_specific_bot):
+        (TestExpectationUpdater.do_update):
+        (main):
+        * Scripts/webkitpy/common/net/bugzilla/test_expectation_updater_unittest.py:
+        (MockAttachment.__init__):
+        (MockAttachment.is_patch):
+        (MockAttachment):
+        (MockAttachment.is_obsolete):
+        (MockBugzilla):
+        (MockBugzilla.attachments):
+        (MockRequests):
+        (MockRequests.__init__):
+        (MockRequests.get):
+        (MockRequests.content):
+        (MockRequests.text):
+        (MockZip.__init__):
+        (TestExpectationUpdaterTest.test_update_test_expectations):
+
 2020-05-20  Carlos Garcia Campos  <cgarcia@igalia.com>
 
         Unreviewed. Fix GTK4 build with GTK 3.98.4
index a39a157..a31d458 100644 (file)
@@ -1,6 +1,7 @@
 #!/usr/bin/env python
 
 # Copyright (C) 2017 Apple Inc. All rights reserved.
+# Copyright (C) 2020 Igalia S.L.
 #
 # Redistribution and use in source and binary forms, with or without
 # modification, are permitted provided that the following conditions
@@ -32,17 +33,25 @@ import io
 import logging
 import zipfile
 
+from webkitpy.common.config.urls import ewsserver_default_host
 from webkitpy.common.host import Host
 from webkitpy.common.net.bugzilla import bugzilla
 from webkitpy.common.net.layouttestresults import LayoutTestResults
 from webkitpy.common.webkit_finder import WebKitFinder
 from webkitpy.layout_tests.controllers.test_result_writer import TestResultWriter
 from webkitpy.layout_tests.models import test_expectations
+from webkitpy.thirdparty.autoinstalled import requests
+import json
+import re
+
+# Buildbot status codes referenced from https://github.com/buildbot/buildbot/blob/master/master/buildbot/process/results.py
+EWS_STATECODES = {'SUCCESS': 0, 'WARNINGS': 1, 'FAILURE': 2, 'SKIPPED': 3, 'EXCEPTION': 4, 'RETRY': 5, 'CANCELLED': 6}
+
 
 _log = logging.getLogger(__name__)
 
 
-def configure_logging():
+def configure_logging(debug=False):
     class LogHandler(logging.StreamHandler):
 
         def format(self, record):
@@ -50,10 +59,11 @@ def configure_logging():
                 return "%s: %s" % (record.levelname, record.getMessage())
             return record.getMessage()
 
-    logger = logging.getLogger()
-    logger.setLevel(logging.INFO)
+    log_level = logging.DEBUG if debug else logging.INFO
+    logger = logging.getLogger(__name__)
+    logger.setLevel(log_level)
     handler = LogHandler()
-    handler.setLevel(logging.INFO)
+    handler.setLevel(log_level)
     logger.addHandler(handler)
     return handler
 
@@ -61,35 +71,35 @@ def configure_logging():
 def argument_parser():
     description = """Update expected.txt files from patches submitted by EWS bots on bugzilla. Given id refers to a bug id by default."""
     parser = argparse.ArgumentParser(prog='importupdate-test-expectations-from-bugzilla bugzilla_id', description=description, formatter_class=argparse.RawDescriptionHelpFormatter)
-
     parser.add_argument('-a', '--is-attachment-id', dest='is_bug_id', action='store_false', default=True, help='Whether the given id is a bugzilla attachment and not a bug id')
-    parser.add_argument('-s', '--is-platform-specific', dest='is_attachment_platform_specific', action='store_true', default=False, help='Whether generic expected files should be updated or not')
+    parser.add_argument('-b', '--bot-filter', dest='bot_filter_name', action='store', default=None, help='Only process EWS results for bots where BOT_FILTER_NAME its part of the name')
+    parser.add_argument('-d', '--debug', dest='debug', action='store_true', default=False, help='Log debug messages')
     return parser
 
 
 class TestExpectationUpdater(object):
 
-    def __init__(self, host, bugzilla_id, is_bug_id=True, is_attachment_platform_specific=False, attachment_fetcher=bugzilla.Bugzilla(), unzip=None):
+    def __init__(self, host, bugzilla_id, is_bug_id=True, bot_filter_name=None, attachment_fetcher=bugzilla.Bugzilla(), unzip=None):
         self.host = host
         self.filesystem = self.host.filesystem
+        self.bot_filter_name = bot_filter_name
         self.unzip = unzip if unzip else lambda content: zipfile.ZipFile(io.BytesIO(content))
+        self.layout_test_repository = WebKitFinder(self.filesystem).path_from_webkit_base("LayoutTests")
         if is_bug_id:
-            self.platform_specific_attachments = {}
-            for attachment in attachment_fetcher.fetch_bug(bugzilla_id).attachments():
-                bot_type = self._bot_type(attachment)
-                if bot_type:
-                    self.platform_specific_attachments[bot_type] = attachment
-            self.generic_attachment = self.platform_specific_attachments.pop("mac-wk2") if "mac-wk2" in self.platform_specific_attachments else None
+            bug_info = attachment_fetcher.fetch_bug(bugzilla_id)
+            attachments = [attachments for attachments in bug_info.attachments(include_obsolete=False) if attachments.is_patch()]
+            if len(attachments) > 1:
+                raise RuntimeError("Found more than one non-obsolete patch in bug {}. Please specify which one to process.".format(bugzilla_id))
+            if len(attachments) < 1:
+                raise RuntimeError("Couldn't find any non-obsolete patch in bug {}. Please specify which one to process.".format(bugzilla_id))
+            self.patch = attachments[0]
         else:
-            attachment = attachment_fetcher.fetch_attachment(bugzilla_id)
-            self.platform_specific_attachments = {self._bot_type(attachment): attachment} if is_attachment_platform_specific else {}
-            self.generic_attachment = attachment if not is_attachment_platform_specific else None
-
-        webkit_finder = WebKitFinder(self.filesystem)
-        self.layout_test_repository = webkit_finder.path_from_webkit_base("LayoutTests")
+            self.patch = attachment_fetcher.fetch_attachment(bugzilla_id)
+        if not self.patch.is_patch():
+            raise RuntimeError("Attachment {} its not a patch. Can't continue.".format(bugzilla_id))
 
-    def _bot_type(self, attachment):
-        name = attachment.name()
+    def _platform_name(self, bot_name):
+        name = bot_name
         if "mac" in name and name.endswith("-wk2"):
             return "mac-wk2"
         if "mac" in name and not name.endswith("-wk2"):
@@ -98,48 +108,112 @@ class TestExpectationUpdater(object):
             return "ios-wk2"
         if "win-future" in name:
             return "win"
+        if "gtk" in name:
+            return "gtk"
+        if "wpe" in name:
+            return "wpe"
+        if "-debug" in name:
+            name = name.replace("-debug", "")
+        return name
+
+    def _get_layout_tests_run(self, bot_name, ews_build_url):
+        if not re.match("http.*webkit.org/#/builders/[0-9]+/builds/[0-9]+$", ews_build_url):
+            raise ValueError("The URL {} from EWS has an unexpected format".format(ews_build_url))
+        ews_buildbot_server, ews_buildbot_path = ews_build_url.split('#')
+        ews_buildbot_steps_url = "{}api/v2{}/steps".format(ews_buildbot_server, ews_buildbot_path)
+        ews_buildbot_steps_request = requests.get(ews_buildbot_steps_url)
+        ews_buildbot_steps = json.loads(ews_buildbot_steps_request.text)
+        _log.debug("Trying to finding failed layout test run for bot {} at url {}".format(bot_name, ews_buildbot_steps_url))
+        for step in ews_buildbot_steps['steps']:
+            # pick the first run as the one for updating the layout tests.
+            if step['name'] != 'layout-tests':
+                continue
+            if not step['complete']:
+                _log.info("Skipping to process unfinished layout-tests run for bot {}. Please retry later".format(bot_name))
+                continue
+            for url_entry in step['urls']:
+                url = url_entry['url']
+                if url.endswith('.zip'):
+                    if url.startswith('/'):
+                        url = "{}{}".format(ews_buildbot_server, url.lstrip('/'))
+                    return {"layout-tests-results-string": step['state_string'], "layout-tests-archive-url": url}
         return None
 
-    def _tests_to_update(self, attachment, bot_type=None):
-        _log.info("Processing attachment " + str(attachment.id()))
-        zip_file = self.unzip(attachment.contents())
+    def _lookup_ews_results(self):
+        _log.info("Looking for EWS results in patch attachment %s from bug %s" % (self.patch.id(), self.patch.bug().id()))
+        ews_bubbles_url = "https://{}/status/{}/".format(ewsserver_default_host, self.patch.id())
+        _log.debug("Querying bubble status at {}".format(ews_bubbles_url))
+        ews_bubbles_status_request = requests.get(ews_bubbles_url)
+        ews_bubbles_status = json.loads(ews_bubbles_status_request.text)
+        self.ews_results = {}
+        for bot in ews_bubbles_status:
+            if ews_bubbles_status[bot]['state'] is None:
+                _log.info("EWS bot {} has still not finished. Ignoring".format(bot))
+                continue
+            _log.debug("Found EWS run at {} for bot {} with state {}".format(ews_bubbles_status[bot]['url'], bot,
+                       [STATE for STATE in EWS_STATECODES.keys()][ews_bubbles_status[bot]['state']]))
+            if self.bot_filter_name:
+                if self.bot_filter_name in bot:
+                    self.ews_results[bot] = self._get_layout_tests_run(bot, ews_bubbles_status[bot]['url'])
+            elif ews_bubbles_status[bot]['state'] in [EWS_STATECODES['FAILURE'], EWS_STATECODES['WARNINGS']]:
+                self.ews_results[bot] = self._get_layout_tests_run(bot, ews_bubbles_status[bot]['url'])
+
+        # Delete entries with null value (if any)
+        for bot in self.ews_results.keys():
+            if not self.ews_results[bot]:
+                del self.ews_results[bot]
+
+        if len(self.ews_results) == 0:
+            if self.bot_filter_name:
+                raise RuntimeError("Couldn't find any failed EWS result in attachment/patch {} that matches filter name.".format(self.patch.id(), self.bot_filter_name))
+            raise RuntimeError("Couldn't find any failed EWS result for attachment/patch {}. Try to specify a bot filter name manually.".format(self.patch.id()))
+
+    def _tests_to_update(self, bot_name):
+        _log.info("{} archive: {}".format(bot_name, self.ews_results[bot_name]['layout-tests-archive-url']))
+        _log.info("{} status: {}".format(bot_name, self.ews_results[bot_name]['layout-tests-results-string']))
+        layout_tests_archive_request = requests.get(self.ews_results[bot_name]['layout-tests-archive-url'])
+        layout_tests_archive_content = layout_tests_archive_request.content
+        zip_file = self.unzip(layout_tests_archive_content)
         results = LayoutTestResults.results_from_string(zip_file.read("full_results.json"))
-        results_to_update = [result.test_name for result in results.failing_test_results() if result.type == test_expectations.TEXT]
+        results_to_update = [result.test_name for result in results.failing_test_results() if result.type in [test_expectations.TEXT, test_expectations.MISSING]]
         return {result: zip_file.read(TestResultWriter.actual_filename(result, self.filesystem)) for result in results_to_update}
 
     def _file_content_if_exists(self, filename):
         return self.filesystem.read_text_file(filename) if self.filesystem.exists(filename) else ""
 
-    # FIXME: Investigate the possibility to align what is done there with what single_test_runner is doing.
-    # In particular the fact of not overwriting the file if content is the same.
-    def _update_from_generic_attachment(self):
-        for test_name, expected_content in self._tests_to_update(self.generic_attachment).items():
+    def _update_for_generic_bot(self, bot_name):
+        for test_name, expected_content in self._tests_to_update(bot_name).items():
             expected_filename = self.filesystem.join(self.layout_test_repository, TestResultWriter.expected_filename(test_name, self.filesystem))
             if expected_content != self._file_content_if_exists(expected_filename):
-                _log.info("Updating " + test_name + " (" + expected_filename + ")")
+                _log.info("Updating " + test_name + " for " + bot_name + " (" + expected_filename + ")")
+                self.filesystem.maybe_make_directory(self.filesystem.dirname(expected_filename))
                 self.filesystem.write_text_file(expected_filename, expected_content)
 
-    # FIXME: Investigate the possibility to align what is done there with what single_test_runner is doing.
-    # In particular the ability to remove a new specific expectation if it is the same as the generic one.
-    def _update_from_platform_specific_attachment(self, attachment, bot_type):
-        for test_name, expected_content in self._tests_to_update(attachment, bot_type).items():
-            expected_filename = self.filesystem.join(self.layout_test_repository, TestResultWriter.expected_filename(test_name, self.filesystem, bot_type))
+    def _update_for_platform_specific_bot(self, bot_name):
+        platform_name = self._platform_name(bot_name)
+        for test_name, expected_content in self._tests_to_update(bot_name).items():
+            expected_filename = self.filesystem.join(self.layout_test_repository, TestResultWriter.expected_filename(test_name, self.filesystem, platform_name))
             generic_expected_filename = self.filesystem.join(self.layout_test_repository, TestResultWriter.expected_filename(test_name, self.filesystem))
             if expected_content != self._file_content_if_exists(generic_expected_filename):
-                _log.info("Updating " + test_name + " for " + bot_type + " (" + expected_filename + ")")
-                self.filesystem.maybe_make_directory(self.filesystem.dirname(expected_filename))
-                self.filesystem.write_text_file(expected_filename, expected_content)
+                if expected_content != self._file_content_if_exists(expected_filename):
+                    _log.info("Updating " + test_name + " for " + bot_name + " (" + expected_filename + ")")
+                    self.filesystem.maybe_make_directory(self.filesystem.dirname(expected_filename))
+                    self.filesystem.write_text_file(expected_filename, expected_content)
             elif self.filesystem.exists(expected_filename):
+                _log.info("Updating " + test_name + " for " + bot_name + " ( REMOVED: " + expected_filename + ")")
                 self.filesystem.remove(expected_filename)
 
     def do_update(self):
-        if not self.generic_attachment and not self.platform_specific_attachments:
-            _log.info("No attachment to process")
-            return
-        if self.generic_attachment:
-            self._update_from_generic_attachment()
-        for bot_type, attachment in self.platform_specific_attachments.items():
-            self._update_from_platform_specific_attachment(attachment, bot_type)
+        self._lookup_ews_results()
+        generic_bots = [bot_name for bot_name in self.ews_results.keys() if self._platform_name(bot_name) == "mac-wk2"]
+        platform_bots = [bot_name for bot_name in self.ews_results.keys() if self._platform_name(bot_name) != "mac-wk2"]
+        # First update the generic results, so updates for platform bots can use this new generic result as base.
+        for bot_name in generic_bots:
+            _log.info("Updating results from bot {} (generic)".format(bot_name))
+            self._update_for_generic_bot(bot_name)
+        for bot_name in platform_bots:
+            _log.info("Updating results from bot {} (platform)".format(bot_name))
+            self._update_for_platform_specific_bot(bot_name)
 
 
 def main(_argv, _stdout, _stderr):
@@ -149,8 +223,8 @@ def main(_argv, _stdout, _stderr):
     if not args:
         raise Exception("Please provide a bug id or use -a option")
 
-    configure_logging()
+    configure_logging(options.debug)
 
     bugzilla_id = args[0]
-    updater = TestExpectationUpdater(Host(), bugzilla_id, options.is_bug_id, options.is_attachment_platform_specific)
+    updater = TestExpectationUpdater(Host(), bugzilla_id, options.is_bug_id, options.bot_filter_name)
     updater.do_update()
index 99df6a3..5bd64b9 100644 (file)
@@ -34,6 +34,7 @@ from webkitpy.common.system.filesystem_mock import MockFileSystem
 from webkitpy.common.system.executive_mock import MockExecutive2
 from webkitpy.common.net.bugzilla.test_expectation_updater import TestExpectationUpdater
 from webkitpy.common.net.bugzilla.attachment import Attachment
+from webkitpy.thirdparty import mock
 
 FAKE_FILES = {
     '/tests/csswg/css-fake-1/empty_dir/README.txt': '',
@@ -42,34 +43,64 @@ FAKE_FILES = {
 
 
 class MockAttachment(Attachment):
-    def __init__(self, attachment_dictionary, contents):
+    def __init__(self, attachment_dictionary, contents, is_patch, is_obsolete):
         Attachment.__init__(self, attachment_dictionary, self)
         self._contents = contents
+        self._is_patch = is_patch
+        self._is_obsolete = is_obsolete
 
     def contents(self):
         return self._contents
 
+    def is_patch(self):
+        return self._is_patch
+
+    def is_obsolete(self):
+        return self._is_obsolete
 
-class MockBugzilla():
-    def __init__(self, include_wk2=True, include_win_future=False):
-        self._include_wk2 = include_wk2
-        self._include_win_future = include_win_future
 
+class MockBugzilla():
     def fetch_bug(self, id):
         return self
 
-    def attachments(self):
+    def attachments(self, include_obsolete=False):
         attachments = []
-        if self._include_wk2:
-            attachments.append(MockAttachment({"id": 1, "name": "Archive of layout-test-results from ews103 for mac-elcapitan-wk2"}, "mac-wk2"))
-        attachments.append(MockAttachment({"id": 2, "name": "Archive of layout-test-results from ews103 for mac-elcapitan"}, "mac-wk1a"))
-        attachments.append(MockAttachment({"id": 3, "name": "Archive of layout-test-results from ews104 for mac-elcapitan"}, "mac-wk1b"))
-        attachments.append(MockAttachment({"id": 4, "name": "Archive of layout-test-results from ews122 for ios-simulator-wk2"}, "ios-sim"))
-        if self._include_win_future:
-            attachments.append(MockAttachment({"id": 5, "name": "Archive of layout-test-results from ews124 for win-future"}, "win-future"))
+        attachments.append(MockAttachment({"id": 1, "name": "Test case"}, "How to reproduce the issue", False, False))
+        if include_obsolete:
+            attachments.append(MockAttachment({"id": 2, "name": "Patch"}, "Patch v1", True, True))
+            attachments.append(MockAttachment({"id": 3, "name": "Patch"}, "Patch v2", True, True))
+        attachments.append(MockAttachment({"id": 4, "name": "Patch"}, "Patch v3", True, False))
         return attachments
 
 
+class MockRequestsGet():
+    def __init__(self, url):
+        self._urls_data = {
+            'https://ews.webkit.org/status/4/': '{"mac-wk1": {"url": "https://ews-build.webkit.org/#/builders/1/builds/99", "state": 2}, "mac-wk2": {"url": "https://ews-build.webkit.org/#/builders/2/builds/99", "state": 2}, "mac-debug-wk1": {"url": "https://ews-build.webkit.org/#/builders/3/builds/99", "state": 2}, "ios-wk2": {"url": "https://ews-build.webkit.org/#/builders/4/builds/99", "state": 2}, "win": {"url": "https://ews-build.webkit.org/#/builders/5/builds/99", "state": 2}}',
+            'https://ews-build.webkit.org/api/v2/builders/1/builds/99/steps': '{ "steps": [ { "complete": true, "name": "layout-tests", "state_string": "Ran layout tests",  "urls": [ { "name": "download layout test results", "url": "/results/mac-wk1/r12345.zip" } ] } ] }',
+            'https://ews-build.webkit.org/api/v2/builders/2/builds/99/steps': '{ "steps": [ { "complete": true, "name": "layout-tests", "state_string": "Ran layout tests",  "urls": [ { "name": "download layout test results", "url": "/results/mac-wk2/r12345.zip" } ] } ] }',
+            'https://ews-build.webkit.org/api/v2/builders/3/builds/99/steps': '{ "steps": [ { "complete": true, "name": "layout-tests", "state_string": "Ran layout tests",  "urls": [ { "name": "download layout test results", "url": "/results/mac-debug-wk1/r12345.zip" } ] } ] }',
+            'https://ews-build.webkit.org/api/v2/builders/4/builds/99/steps': '{ "steps": [ { "complete": true, "name": "layout-tests", "state_string": "Ran layout tests",  "urls": [ { "name": "download layout test results", "url": "/results/ios-wk2/r12345.zip" }  ] } ] }',
+            'https://ews-build.webkit.org/api/v2/builders/5/builds/99/steps': '{ "steps": [ { "complete": true, "name": "layout-tests", "state_string": "Ran layout tests",  "urls": [ { "name": "download layout test results", "url": "/results/win/r12345.zip" }  ] } ] }',
+            "https://ews-build.webkit.org/results/mac-wk1/r12345.zip": "mac-wk1a",
+            "https://ews-build.webkit.org/results/mac-debug-wk1/r12345.zip": "mac-wk1b",
+            "https://ews-build.webkit.org/results/mac-wk2/r12345.zip": "mac-wk2",
+            "https://ews-build.webkit.org/results/ios-wk2/r12345.zip": "ios-wk2",
+            "https://ews-build.webkit.org/results/win/r12345.zip": "win"
+        }
+        self._url = url
+        if url not in self._urls_data:
+            raise ValueError("url {} not mocked".format(url))
+
+    @property
+    def content(self):
+        return self._urls_data[self._url]
+
+    @property
+    def text(self):
+        return str(self.content)
+
+
 class MockZip():
     def __init__(self):
         self.content = None
@@ -95,17 +126,17 @@ class MockZip():
             "imported/w3c/web-platform-tests/fetch/api/redirect/redirect-location-worker-actual.txt": "d",
             "imported/w3c/web-platform-tests/fetch/api/redirect/redirect-location-actual.txt": "e",
             "imported/w3c/web-platform-tests/html/browsers/windows/browsing-context-actual.txt": "f-wk1b"}
-        ios_sim_files = {"full_results.json": 'ADD_RESULTS({"tests":{"imported":{"w3c":{"web-platform-tests":{"url":{"interfaces.html":{"report":"REGRESSION","expected":"PASS","actual":"TEXT"}},"html":{"browsers":{"windows":{"browsing-context.html":{"report":"REGRESSION","expected":"PASS","actual":"TEXT"}},"the-window-object":{"apis-for-creating-and-navigating-browsing-contexts-by-name":{"open-features-tokenization-001.html":{"report":"REGRESSION","expected":"PASS","actual":"TEXT"}}}}},"dom":{"events":{"EventTarget-dispatchEvent.html":{"report":"REGRESSION","expected":"PASS","actual":"TEXT"}}}}}},"animations":{"trigger-container-scroll-empty.html":{"report":"FLAKY","expected":"PASS","actual":"TEXT PASS"}}},"skipped":9881,"num_regressions":4,"other_crashes":{},"interrupted":false,"num_missing":0,"layout_tests_dir":"/Volumes/Data/EWS/WebKit/LayoutTests","version":4,"num_passes":38225,"pixel_tests_enabled":false,"date":"07:33PM on April 08, 2017","has_pretty_patch":true,"fixable":48110,"num_flaky":1,"uses_expectations_file":true});',
+        ios_files = {"full_results.json": 'ADD_RESULTS({"tests":{"imported":{"w3c":{"web-platform-tests":{"url":{"interfaces.html":{"report":"REGRESSION","expected":"PASS","actual":"TEXT"}},"html":{"browsers":{"windows":{"browsing-context.html":{"report":"REGRESSION","expected":"PASS","actual":"TEXT"}},"the-window-object":{"apis-for-creating-and-navigating-browsing-contexts-by-name":{"open-features-tokenization-001.html":{"report":"REGRESSION","expected":"PASS","actual":"TEXT"}}}}},"dom":{"events":{"EventTarget-dispatchEvent.html":{"report":"REGRESSION","expected":"PASS","actual":"TEXT"}}}}}},"animations":{"trigger-container-scroll-empty.html":{"report":"FLAKY","expected":"PASS","actual":"TEXT PASS"}}},"skipped":9881,"num_regressions":4,"other_crashes":{},"interrupted":false,"num_missing":0,"layout_tests_dir":"/Volumes/Data/EWS/WebKit/LayoutTests","version":4,"num_passes":38225,"pixel_tests_enabled":false,"date":"07:33PM on April 08, 2017","has_pretty_patch":true,"fixable":48110,"num_flaky":1,"uses_expectations_file":true});',
             "imported/w3c/web-platform-tests/dom/events/EventTarget-dispatchEvent-actual.txt": "g",
             "imported/w3c/web-platform-tests/html/browsers/the-window-object/apis-for-creating-and-navigating-browsing-contexts-by-name/open-features-tokenization-001-actual.txt": "h",
             "imported/w3c/web-platform-tests/html/browsers/windows/browsing-context-actual.txt": "i",
             "imported/w3c/web-platform-tests/url/interfaces-actual.txt": "j"}
-        win_future_files = {"full_results.json": 'ADD_RESULTS({"tests":{"imported":{"w3c":{"web-platform-tests":{"url":{"interfaces.html":{"report":"REGRESSION","expected":"PASS","actual":"TEXT"}},"html":{"browsers":{"windows":{"browsing-context.html":{"report":"REGRESSION","expected":"PASS","actual":"TEXT"}},"the-window-object":{"apis-for-creating-and-navigating-browsing-contexts-by-name":{"open-features-tokenization-001.html":{"report":"REGRESSION","expected":"PASS","actual":"TEXT"}}}}},"dom":{"events":{"EventTarget-dispatchEvent.html":{"report":"REGRESSION","expected":"PASS","actual":"TEXT"}}}}}},"animations":{"trigger-container-scroll-empty.html":{"report":"FLAKY","expected":"PASS","actual":"TEXT PASS"}}},"skipped":9881,"num_regressions":4,"other_crashes":{},"interrupted":false,"num_missing":0,"layout_tests_dir":"/Volumes/Data/EWS/WebKit/LayoutTests","version":4,"num_passes":38225,"pixel_tests_enabled":false,"date":"07:33PM on April 08, 2017","has_pretty_patch":true,"fixable":48110,"num_flaky":1,"uses_expectations_file":true});',
+        win_files = {"full_results.json": 'ADD_RESULTS({"tests":{"imported":{"w3c":{"web-platform-tests":{"url":{"interfaces.html":{"report":"REGRESSION","expected":"PASS","actual":"TEXT"}},"html":{"browsers":{"windows":{"browsing-context.html":{"report":"REGRESSION","expected":"PASS","actual":"TEXT"}},"the-window-object":{"apis-for-creating-and-navigating-browsing-contexts-by-name":{"open-features-tokenization-001.html":{"report":"REGRESSION","expected":"PASS","actual":"TEXT"}}}}},"dom":{"events":{"EventTarget-dispatchEvent.html":{"report":"REGRESSION","expected":"PASS","actual":"TEXT"}}}}}},"animations":{"trigger-container-scroll-empty.html":{"report":"FLAKY","expected":"PASS","actual":"TEXT PASS"}}},"skipped":9881,"num_regressions":4,"other_crashes":{},"interrupted":false,"num_missing":0,"layout_tests_dir":"/Volumes/Data/EWS/WebKit/LayoutTests","version":4,"num_passes":38225,"pixel_tests_enabled":false,"date":"07:33PM on April 08, 2017","has_pretty_patch":true,"fixable":48110,"num_flaky":1,"uses_expectations_file":true});',
             "imported/w3c/web-platform-tests/dom/events/EventTarget-dispatchEvent-actual.txt": "g",
             "imported/w3c/web-platform-tests/html/browsers/the-window-object/apis-for-creating-and-navigating-browsing-contexts-by-name/open-features-tokenization-001-actual.txt": "h",
             "imported/w3c/web-platform-tests/html/browsers/windows/browsing-context-actual.txt": "i",
             "imported/w3c/web-platform-tests/url/interfaces-actual.txt": "j"}
-        self.files = {"mac-wk2": mac_wk2_files, "mac-wk1a": mac_wk1a_files, "mac-wk1b": mac_wk1b_files, "ios-sim": ios_sim_files, "win-future": win_future_files}
+        self.files = {"mac-wk2": mac_wk2_files, "mac-wk1a": mac_wk1a_files, "mac-wk1b": mac_wk1b_files, "ios-wk2": ios_files, "win": win_files}
 
     def unzip(self, content):
         self.content = content
@@ -133,29 +164,19 @@ class TestExpectationUpdaterTest(unittest.TestCase):
             '/mock-checkout/LayoutTests/imported/w3c/web-platform-tests/url/interfaces-expected.txt': "j-mac-wk2"})
 
         mock_zip = MockZip()
-        updater = TestExpectationUpdater(host, "123456", True, False, MockBugzilla(), lambda content: mock_zip.unzip(content))
-        updater.do_update()
-        # mac-wk2 expectation
-        self.assertTrue(self._is_matching(host, "imported/w3c/web-platform-tests/fetch/api/redirect/redirect-count-cross-origin-expected.txt", "a"))
-        # no need to add mac-wk1 specific expectation
-        self.assertFalse(self._exists(host, "platform/mac-wk1/imported/w3c/web-platform-tests/fetch/api/redirect/redirect-count-cross-origin-expected.txt"))
-        # mac-wk1/ios-simulator-wk2 specific expectation
-        self.assertTrue(self._is_matching(host, "platform/mac-wk1/imported/w3c/web-platform-tests/html/browsers/windows/browsing-context-expected.txt", "f-wk1b"))
-        self.assertTrue(self._is_matching(host, "platform/ios-wk2/imported/w3c/web-platform-tests/url/interfaces-expected.txt", "j"))
-        # removal of mac-wk1 expectation since no longer different
-        self.assertFalse(self._exists(host, "platform/mac-wk1/imported/w3c/web-platform-tests/fetch/api/redirect/redirect-location-expected.txt"))
-
-    def test_update_win_future_test_expectations(self):
-        host = MockHost()
-        host.executive = MockExecutive2(exception=OSError())
-        host.filesystem = MockFileSystem(files={
-            '/mock-checkout/LayoutTests/platform/mac-wk1/imported/w3c/web-platform-tests/fetch/api/redirect/redirect-location-expected.txt': 'e-wk1',
-            '/mock-checkout/LayoutTests/imported/w3c/web-platform-tests/dom/events/EventTarget-dispatchEvent-expected.txt': "g",
-            '/mock-checkout/LayoutTests/imported/w3c/web-platform-tests/html/browsers/the-window-object/apis-for-creating-and-navigating-browsing-contexts-by-name/open-features-tokenization-001-expected.txt': "h",
-            '/mock-checkout/LayoutTests/imported/w3c/web-platform-tests/html/browsers/windows/browsing-context-expected.txt': "i",
-            '/mock-checkout/LayoutTests/imported/w3c/web-platform-tests/url/interfaces-expected.txt': "j-mac-wk2"})
+        with mock.patch('webkitpy.common.net.bugzilla.test_expectation_updater.requests.get', MockRequestsGet):
+            updater = TestExpectationUpdater(host, "123456", True, False, MockBugzilla(), lambda content: mock_zip.unzip(content))
+            updater.do_update()
+            # mac-wk2 expectation
+            self.assertTrue(self._is_matching(host, "imported/w3c/web-platform-tests/fetch/api/redirect/redirect-count-cross-origin-expected.txt", "a"))
+            # no need to add mac-wk1 specific expectation
+            self.assertFalse(self._exists(host, "platform/mac-wk1/imported/w3c/web-platform-tests/fetch/api/redirect/redirect-count-cross-origin-expected.txt"))
+            # mac-wk1/ios-simulator-wk2 specific expectation
+            self.assertTrue(self._is_matching(host, "platform/mac-wk1/imported/w3c/web-platform-tests/html/browsers/windows/browsing-context-expected.txt", "f-wk1b"))
+            self.assertTrue(self._is_matching(host, "platform/ios-wk2/imported/w3c/web-platform-tests/url/interfaces-expected.txt", "j"))
+            # removal of mac-wk1 expectation since no longer different
+            self.assertFalse(self._exists(host, "platform/mac-wk1/imported/w3c/web-platform-tests/fetch/api/redirect/redirect-location-expected.txt"))
+            # windows specific expectation
+            self.assertTrue(self._is_matching(host, "platform/win/imported/w3c/web-platform-tests/url/interfaces-expected.txt", "j"))
+
 
-        mock_zip = MockZip()
-        updater = TestExpectationUpdater(host, "123456", True, False, MockBugzilla(include_wk2=False, include_win_future=True), lambda content: mock_zip.unzip(content))
-        updater.do_update()
-        self.assertTrue(self._is_matching(host, "platform/win/imported/w3c/web-platform-tests/url/interfaces-expected.txt", "j"))