Percent-encode non-ASCII code points in hosts of URLs with unrecognized schemes
authorachristensen@apple.com <achristensen@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 1 Nov 2016 21:31:40 +0000 (21:31 +0000)
committerachristensen@apple.com <achristensen@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 1 Nov 2016 21:31:40 +0000 (21:31 +0000)
https://bugs.webkit.org/show_bug.cgi?id=164290

Reviewed by Tim Horton.

LayoutTests/imported/w3c:

* web-platform-tests/url/a-element-expected.txt:
* web-platform-tests/url/a-element-xhtml-expected.txt:
* web-platform-tests/url/url-constructor-expected.txt:

Source/WebCore:

NSURL fails to parse these URLs, WebKit used to punycode encode them, but now we match Chrome and Firefox,
as well as what will likely become the standard once https://github.com/whatwg/url/issues/148 is resolved.
We continue to parse IPv6 address hosts because otherwise we wouldn't be able to tell where a port
starts in a URL with colons in the IPv6 address and before the port like "a://[b::c]:4"

Covered by new API tests and updated LayoutTests.

* platform/URLParser.cpp:
(WebCore::URLParser::domainToASCII):
(WebCore::URLParser::parseHostAndPort):

Tools:

* TestWebKitAPI/Tests/WebCore/URLParser.cpp:
(TestWebKitAPI::checkRelativeURL):
(TestWebKitAPI::checkURLDifferences):
(TestWebKitAPI::checkRelativeURLDifferences):
Move helper functions to the top so I can use them from any tests.
(TestWebKitAPI::shouldFail):
(TestWebKitAPI::checkURL):
(TestWebKitAPI::TEST_F):

LayoutTests:

* fast/url/host-lowercase-per-scheme-expected.txt:
* fast/url/safari-extension-expected.txt:
* fetch/fetch-url-serialization-expected.txt:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@208239 268f45cc-cd09-0410-ab3c-d52691b4dbfc

12 files changed:
LayoutTests/ChangeLog
LayoutTests/fast/url/host-lowercase-per-scheme-expected.txt
LayoutTests/fast/url/safari-extension-expected.txt
LayoutTests/fetch/fetch-url-serialization-expected.txt
LayoutTests/imported/w3c/ChangeLog
LayoutTests/imported/w3c/web-platform-tests/url/a-element-expected.txt
LayoutTests/imported/w3c/web-platform-tests/url/a-element-xhtml-expected.txt
LayoutTests/imported/w3c/web-platform-tests/url/url-constructor-expected.txt
Source/WebCore/ChangeLog
Source/WebCore/platform/URLParser.cpp
Tools/ChangeLog
Tools/TestWebKitAPI/Tests/WebCore/URLParser.cpp

index c6d2c24..0aede17 100644 (file)
@@ -1,3 +1,14 @@
+2016-11-01  Alex Christensen  <achristensen@webkit.org>
+
+        Percent-encode non-ASCII code points in hosts of URLs with unrecognized schemes
+        https://bugs.webkit.org/show_bug.cgi?id=164290
+
+        Reviewed by Tim Horton.
+
+        * fast/url/host-lowercase-per-scheme-expected.txt:
+        * fast/url/safari-extension-expected.txt:
+        * fetch/fetch-url-serialization-expected.txt:
+
 2016-11-01  Ryan Haddad  <ryanhaddad@apple.com>
 
         Marking inspector/css/pseudo-element-matches.html as flaky on mac-wk2.
index 2420ccf..708210d 100644 (file)
@@ -26,7 +26,7 @@ PASS src is expected
 PASS src is expected
 PASS src is expected
 PASS src is expected
-FAIL src should be ghost://UnicodeF%C3%AAte/. Was ghost://xn--unicodefte-t7a/.
+PASS src is expected
 PASS successfullyParsed is true
 
 TEST COMPLETE
index ecaf9f7..32845c5 100644 (file)
@@ -6,7 +6,7 @@ On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE
 PASS src is expected
 PASS src is expected
 PASS src is expected
-FAIL src should be safari-extension://com.f%C3%AAte/. Was safari-extension://com.xn--fte-fma/.
+PASS src is expected
 PASS successfullyParsed is true
 
 TEST COMPLETE
index 0df8fe0..7a45cdc 100644 (file)
@@ -309,7 +309,7 @@ PASS Testing Request url 'about:/../' with base 'about:blank'
 PASS Testing Request url 'data:/../' with base 'about:blank' 
 PASS Testing Request url 'javascript:/../' with base 'about:blank' 
 PASS Testing Request url 'mailto:/../' with base 'about:blank' 
-PASS Testing Request url 'sc://ñ.test/' with base 'about:blank' 
+FAIL Testing Request url 'sc://ñ.test/' with base 'about:blank' assert_equals: expected "sc://xn--ida.test/" but got "sc://%C3%B1.test/"
 PASS Testing Request url 'sc:\../' with base 'about:blank' 
 PASS Testing Request url 'sc::a@example.net' with base 'about:blank' 
 PASS Testing Request url 'http://127.0.0.1:10100/relative_import.html' with base 'about:blank' 
index 3dd3859..5d4edf4 100644 (file)
@@ -1,3 +1,14 @@
+2016-11-01  Alex Christensen  <achristensen@webkit.org>
+
+        Percent-encode non-ASCII code points in hosts of URLs with unrecognized schemes
+        https://bugs.webkit.org/show_bug.cgi?id=164290
+
+        Reviewed by Tim Horton.
+
+        * web-platform-tests/url/a-element-expected.txt:
+        * web-platform-tests/url/a-element-xhtml-expected.txt:
+        * web-platform-tests/url/url-constructor-expected.txt:
+
 2016-10-31  Brady Eidson  <beidson@apple.com>
 
         IndexedDB 2.0: Support IDBObjectStore getAll/getAllKeys.
index d098d7d..fb3f4d3 100644 (file)
@@ -313,7 +313,7 @@ PASS Parsing: <about:/../> against <about:blank>
 PASS Parsing: <data:/../> against <about:blank> 
 PASS Parsing: <javascript:/../> against <about:blank> 
 FAIL Parsing: <mailto:/../> against <about:blank> assert_equals: origin expected "null" but got "mailto://"
-FAIL Parsing: <sc://ñ.test/> against <about:blank> assert_equals: origin expected "null" but got "sc://xn--ida.test"
+FAIL Parsing: <sc://ñ.test/> against <about:blank> assert_equals: href expected "sc://xn--ida.test/" but got "sc://%C3%B1.test/"
 FAIL Parsing: <sc:\../> against <about:blank> assert_equals: origin expected "null" but got "sc://"
 FAIL Parsing: <sc::a@example.net> against <about:blank> assert_equals: origin expected "null" but got "sc://"
 PASS Parsing: <http://127.0.0.1:10100/relative_import.html> against <about:blank> 
index d098d7d..fb3f4d3 100644 (file)
@@ -313,7 +313,7 @@ PASS Parsing: <about:/../> against <about:blank>
 PASS Parsing: <data:/../> against <about:blank> 
 PASS Parsing: <javascript:/../> against <about:blank> 
 FAIL Parsing: <mailto:/../> against <about:blank> assert_equals: origin expected "null" but got "mailto://"
-FAIL Parsing: <sc://ñ.test/> against <about:blank> assert_equals: origin expected "null" but got "sc://xn--ida.test"
+FAIL Parsing: <sc://ñ.test/> against <about:blank> assert_equals: href expected "sc://xn--ida.test/" but got "sc://%C3%B1.test/"
 FAIL Parsing: <sc:\../> against <about:blank> assert_equals: origin expected "null" but got "sc://"
 FAIL Parsing: <sc::a@example.net> against <about:blank> assert_equals: origin expected "null" but got "sc://"
 PASS Parsing: <http://127.0.0.1:10100/relative_import.html> against <about:blank> 
index 5b1b3e1..0d06115 100644 (file)
@@ -318,7 +318,7 @@ PASS Parsing: <about:/../> against <about:blank>
 PASS Parsing: <data:/../> against <about:blank> 
 PASS Parsing: <javascript:/../> against <about:blank> 
 FAIL Parsing: <mailto:/../> against <about:blank> assert_equals: origin expected "null" but got "mailto://"
-FAIL Parsing: <sc://ñ.test/> against <about:blank> assert_equals: origin expected "null" but got "sc://xn--ida.test"
+FAIL Parsing: <sc://ñ.test/> against <about:blank> assert_equals: href expected "sc://xn--ida.test/" but got "sc://%C3%B1.test/"
 FAIL Parsing: <sc:\../> against <about:blank> assert_equals: origin expected "null" but got "sc://"
 FAIL Parsing: <sc::a@example.net> against <about:blank> assert_equals: origin expected "null" but got "sc://"
 PASS Parsing: <http://127.0.0.1:10100/relative_import.html> against <about:blank> 
index 230f870..fbe24b4 100644 (file)
@@ -1,3 +1,21 @@
+2016-11-01  Alex Christensen  <achristensen@webkit.org>
+
+        Percent-encode non-ASCII code points in hosts of URLs with unrecognized schemes
+        https://bugs.webkit.org/show_bug.cgi?id=164290
+
+        Reviewed by Tim Horton.
+
+        NSURL fails to parse these URLs, WebKit used to punycode encode them, but now we match Chrome and Firefox,
+        as well as what will likely become the standard once https://github.com/whatwg/url/issues/148 is resolved.
+        We continue to parse IPv6 address hosts because otherwise we wouldn't be able to tell where a port
+        starts in a URL with colons in the IPv6 address and before the port like "a://[b::c]:4"
+
+        Covered by new API tests and updated LayoutTests.
+
+        * platform/URLParser.cpp:
+        (WebCore::URLParser::domainToASCII):
+        (WebCore::URLParser::parseHostAndPort):
+
 2016-11-01  Gavin Barraclough  <barraclough@apple.com>
 
         Add IsAudible, IsLoading to ActivityState
index 34ca6b3..4bdd978 100644 (file)
@@ -2469,28 +2469,18 @@ Optional<Vector<LChar, URLParser::defaultInlineBufferSize>> URLParser::domainToA
         if (domain.is8Bit()) {
             const LChar* characters = domain.characters8();
             ascii.reserveInitialCapacity(length);
-            if (m_urlIsSpecial) {
-                for (size_t i = 0; i < length; ++i) {
-                    if (UNLIKELY(isASCIIUpper(characters[i])))
-                        syntaxViolation(iteratorForSyntaxViolationPosition);
-                    ascii.uncheckedAppend(toASCIILower(characters[i]));
-                }
-            } else {
-                for (size_t i = 0; i < length; ++i)
-                    ascii.uncheckedAppend(characters[i]);
+            for (size_t i = 0; i < length; ++i) {
+                if (UNLIKELY(isASCIIUpper(characters[i])))
+                    syntaxViolation(iteratorForSyntaxViolationPosition);
+                ascii.uncheckedAppend(toASCIILower(characters[i]));
             }
         } else {
             const UChar* characters = domain.characters16();
             ascii.reserveInitialCapacity(length);
-            if (m_urlIsSpecial) {
-                for (size_t i = 0; i < length; ++i) {
-                    if (UNLIKELY(isASCIIUpper(characters[i])))
-                        syntaxViolation(iteratorForSyntaxViolationPosition);
-                    ascii.uncheckedAppend(toASCIILower(characters[i]));
-                }
-            } else {
-                for (size_t i = 0; i < length; ++i)
-                    ascii.uncheckedAppend(characters[i]);
+            for (size_t i = 0; i < length; ++i) {
+                if (UNLIKELY(isASCIIUpper(characters[i])))
+                    syntaxViolation(iteratorForSyntaxViolationPosition);
+                ascii.uncheckedAppend(toASCIILower(characters[i]));
             }
         }
         return ascii;
@@ -2610,8 +2600,27 @@ bool URLParser::parseHostAndPort(CodePointIterator<CharacterType> iterator)
             m_url.m_hostEnd = currentPosition(ipv6End);
             return true;
         }
+        return false;
     }
 
+    if (!m_urlIsSpecial) {
+        for (; !iterator.atEnd(); ++iterator) {
+            if (UNLIKELY(isTabOrNewline(*iterator))) {
+                syntaxViolation(iterator);
+                continue;
+            }
+            if (*iterator == ':')
+                break;
+            utf8PercentEncode<isInSimpleEncodeSet>(iterator);
+        }
+        m_url.m_hostEnd = currentPosition(iterator);
+        if (iterator.atEnd()) {
+            m_url.m_portEnd = currentPosition(iterator);
+            return true;
+        }
+        return parsePort(iterator);
+    }
+    
     if (LIKELY(!m_hostHasPercentOrNonASCII)) {
         auto hostIterator = iterator;
         for (; !iterator.atEnd(); ++iterator) {
@@ -2622,28 +2631,23 @@ bool URLParser::parseHostAndPort(CodePointIterator<CharacterType> iterator)
             if (isInvalidDomainCharacter(*iterator))
                 return false;
         }
-        if (m_urlIsSpecial) {
-            if (auto address = parseIPv4Host(CodePointIterator<CharacterType>(hostIterator, iterator))) {
-                serializeIPv4(address.value());
-                m_url.m_hostEnd = currentPosition(iterator);
-                if (iterator.atEnd()) {
-                    m_url.m_portEnd = currentPosition(iterator);
-                    return true;
-                }
-                return parsePort(iterator);
+        if (auto address = parseIPv4Host(CodePointIterator<CharacterType>(hostIterator, iterator))) {
+            serializeIPv4(address.value());
+            m_url.m_hostEnd = currentPosition(iterator);
+            if (iterator.atEnd()) {
+                m_url.m_portEnd = currentPosition(iterator);
+                return true;
             }
+            return parsePort(iterator);
         }
         for (; hostIterator != iterator; ++hostIterator) {
             if (UNLIKELY(isTabOrNewline(*hostIterator))) {
                 syntaxViolation(hostIterator);
                 continue;
             }
-            if (m_urlIsSpecial) {
-                if (UNLIKELY(isASCIIUpper(*hostIterator)))
-                    syntaxViolation(hostIterator);
-                appendToASCIIBuffer(toASCIILower(*hostIterator));
-            } else
-                appendToASCIIBuffer(*hostIterator);
+            if (UNLIKELY(isASCIIUpper(*hostIterator)))
+                syntaxViolation(hostIterator);
+            appendToASCIIBuffer(toASCIILower(*hostIterator));
         }
         m_url.m_hostEnd = currentPosition(iterator);
         if (!hostIterator.atEnd())
index 27cb700..5132ef0 100644 (file)
@@ -1,3 +1,19 @@
+2016-11-01  Alex Christensen  <achristensen@webkit.org>
+
+        Percent-encode non-ASCII code points in hosts of URLs with unrecognized schemes
+        https://bugs.webkit.org/show_bug.cgi?id=164290
+
+        Reviewed by Tim Horton.
+
+        * TestWebKitAPI/Tests/WebCore/URLParser.cpp:
+        (TestWebKitAPI::checkRelativeURL):
+        (TestWebKitAPI::checkURLDifferences):
+        (TestWebKitAPI::checkRelativeURLDifferences):
+        Move helper functions to the top so I can use them from any tests.
+        (TestWebKitAPI::shouldFail):
+        (TestWebKitAPI::checkURL):
+        (TestWebKitAPI::TEST_F):
+
 2016-11-01  Hyowon Kim  <hw1008.kim@samsung.com>
 
         [GTK] Failed to generate GeoClue2Interface files.
index 47cd072..eea9b91 100644 (file)
@@ -129,6 +129,202 @@ static void checkURL(const String& urlString, const ExpectedParts& parts, TestTa
     }
 }
 
+static void checkRelativeURL(const String& urlString, const String& baseURLString, const ExpectedParts& parts, TestTabs testTabs = TestTabs::Yes)
+{
+    bool wasEnabled = URLParser::enabled();
+    URLParser::setEnabled(true);
+    auto url = URL(URL(URL(), baseURLString), urlString);
+    URLParser::setEnabled(false);
+    auto oldURL = URL(URL(URL(), baseURLString), urlString);
+    URLParser::setEnabled(wasEnabled);
+    
+    EXPECT_TRUE(eq(parts.protocol, url.protocol().toString()));
+    EXPECT_TRUE(eq(parts.user, url.user()));
+    EXPECT_TRUE(eq(parts.password, url.pass()));
+    EXPECT_TRUE(eq(parts.host, url.host()));
+    EXPECT_EQ(parts.port, url.port().valueOr(0));
+    EXPECT_TRUE(eq(parts.path, url.path()));
+    EXPECT_TRUE(eq(parts.query, url.query()));
+    EXPECT_TRUE(eq(parts.fragment, url.fragmentIdentifier()));
+    EXPECT_TRUE(eq(parts.string, url.string()));
+    
+    EXPECT_TRUE(eq(parts.protocol, oldURL.protocol().toString()));
+    EXPECT_TRUE(eq(parts.user, oldURL.user()));
+    EXPECT_TRUE(eq(parts.password, oldURL.pass()));
+    EXPECT_TRUE(eq(parts.host, oldURL.host()));
+    EXPECT_EQ(parts.port, oldURL.port().valueOr(0));
+    EXPECT_TRUE(eq(parts.path, oldURL.path()));
+    EXPECT_TRUE(eq(parts.query, oldURL.query()));
+    EXPECT_TRUE(eq(parts.fragment, oldURL.fragmentIdentifier()));
+    EXPECT_TRUE(eq(parts.string, oldURL.string()));
+    
+    EXPECT_TRUE(URLParser::allValuesEqual(url, oldURL));
+    EXPECT_TRUE(URLParser::internalValuesConsistent(url));
+    EXPECT_TRUE(URLParser::internalValuesConsistent(oldURL));
+    
+    if (testTabs == TestTabs::No)
+        return;
+    
+    for (size_t i = 0; i < urlString.length(); ++i) {
+        String urlStringWithTab = insertTabAtLocation(urlString, i);
+        checkRelativeURL(urlStringWithTab,
+            baseURLString,
+            parts.isInvalid() ? invalidParts(urlStringWithTab) : parts,
+            TestTabs::No);
+    }
+}
+
+static void checkURLDifferences(const String& urlString, const ExpectedParts& partsNew, const ExpectedParts& partsOld, TestTabs testTabs = TestTabs::Yes)
+{
+    bool wasEnabled = URLParser::enabled();
+    URLParser::setEnabled(true);
+    auto url = URL(URL(), urlString);
+    URLParser::setEnabled(false);
+    auto oldURL = URL(URL(), urlString);
+    URLParser::setEnabled(wasEnabled);
+    
+    EXPECT_TRUE(eq(partsNew.protocol, url.protocol().toString()));
+    EXPECT_TRUE(eq(partsNew.user, url.user()));
+    EXPECT_TRUE(eq(partsNew.password, url.pass()));
+    EXPECT_TRUE(eq(partsNew.host, url.host()));
+    EXPECT_EQ(partsNew.port, url.port().valueOr(0));
+    EXPECT_TRUE(eq(partsNew.path, url.path()));
+    EXPECT_TRUE(eq(partsNew.query, url.query()));
+    EXPECT_TRUE(eq(partsNew.fragment, url.fragmentIdentifier()));
+    EXPECT_TRUE(eq(partsNew.string, url.string()));
+    
+    EXPECT_TRUE(eq(partsOld.protocol, oldURL.protocol().toString()));
+    EXPECT_TRUE(eq(partsOld.user, oldURL.user()));
+    EXPECT_TRUE(eq(partsOld.password, oldURL.pass()));
+    EXPECT_TRUE(eq(partsOld.host, oldURL.host()));
+    EXPECT_EQ(partsOld.port, oldURL.port().valueOr(0));
+    EXPECT_TRUE(eq(partsOld.path, oldURL.path()));
+    EXPECT_TRUE(eq(partsOld.query, oldURL.query()));
+    EXPECT_TRUE(eq(partsOld.fragment, oldURL.fragmentIdentifier()));
+    EXPECT_TRUE(eq(partsOld.string, oldURL.string()));
+    
+    EXPECT_FALSE(URLParser::allValuesEqual(url, oldURL));
+    EXPECT_TRUE(URLParser::internalValuesConsistent(url));
+    EXPECT_TRUE(URLParser::internalValuesConsistent(oldURL));
+    
+    if (testTabs == TestTabs::No)
+        return;
+    
+    for (size_t i = 0; i < urlString.length(); ++i) {
+        String urlStringWithTab = insertTabAtLocation(urlString, i);
+        checkURLDifferences(urlStringWithTab,
+            partsNew.isInvalid() ? invalidParts(urlStringWithTab) : partsNew,
+            partsOld.isInvalid() ? invalidParts(urlStringWithTab) : partsOld,
+            TestTabs::No);
+    }
+}
+
+static void checkRelativeURLDifferences(const String& urlString, const String& baseURLString, const ExpectedParts& partsNew, const ExpectedParts& partsOld, TestTabs testTabs = TestTabs::Yes)
+{
+    bool wasEnabled = URLParser::enabled();
+    URLParser::setEnabled(true);
+    auto url = URL(URL(URL(), baseURLString), urlString);
+    URLParser::setEnabled(false);
+    auto oldURL = URL(URL(URL(), baseURLString), urlString);
+    URLParser::setEnabled(wasEnabled);
+    
+    EXPECT_TRUE(eq(partsNew.protocol, url.protocol().toString()));
+    EXPECT_TRUE(eq(partsNew.user, url.user()));
+    EXPECT_TRUE(eq(partsNew.password, url.pass()));
+    EXPECT_TRUE(eq(partsNew.host, url.host()));
+    EXPECT_EQ(partsNew.port, url.port().valueOr(0));
+    EXPECT_TRUE(eq(partsNew.path, url.path()));
+    EXPECT_TRUE(eq(partsNew.query, url.query()));
+    EXPECT_TRUE(eq(partsNew.fragment, url.fragmentIdentifier()));
+    EXPECT_TRUE(eq(partsNew.string, url.string()));
+    
+    EXPECT_TRUE(eq(partsOld.protocol, oldURL.protocol().toString()));
+    EXPECT_TRUE(eq(partsOld.user, oldURL.user()));
+    EXPECT_TRUE(eq(partsOld.password, oldURL.pass()));
+    EXPECT_TRUE(eq(partsOld.host, oldURL.host()));
+    EXPECT_EQ(partsOld.port, oldURL.port().valueOr(0));
+    EXPECT_TRUE(eq(partsOld.path, oldURL.path()));
+    EXPECT_TRUE(eq(partsOld.query, oldURL.query()));
+    EXPECT_TRUE(eq(partsOld.fragment, oldURL.fragmentIdentifier()));
+    EXPECT_TRUE(eq(partsOld.string, oldURL.string()));
+    
+    EXPECT_FALSE(URLParser::allValuesEqual(url, oldURL));
+    EXPECT_TRUE(URLParser::internalValuesConsistent(url));
+    EXPECT_TRUE(URLParser::internalValuesConsistent(oldURL));
+    
+    if (testTabs == TestTabs::No)
+        return;
+    
+    for (size_t i = 0; i < urlString.length(); ++i) {
+        String urlStringWithTab = insertTabAtLocation(urlString, i);
+        checkRelativeURLDifferences(urlStringWithTab, baseURLString,
+            partsNew.isInvalid() ? invalidParts(urlStringWithTab) : partsNew,
+            partsOld.isInvalid() ? invalidParts(urlStringWithTab) : partsOld,
+            TestTabs::No);
+    }
+}
+
+static void shouldFail(const String& urlString)
+{
+    checkURL(urlString, {"", "", "", "", 0, "", "", "", urlString});
+}
+
+static void shouldFail(const String& urlString, const String& baseString)
+{
+    checkRelativeURL(urlString, baseString, {"", "", "", "", 0, "", "", "", urlString});
+}
+
+static void checkURL(const String& urlString, const TextEncoding& encoding, const ExpectedParts& parts, TestTabs testTabs = TestTabs::Yes)
+{
+    URLParser parser(urlString, { }, encoding);
+    auto url = parser.result();
+    EXPECT_TRUE(eq(parts.protocol, url.protocol().toString()));
+    EXPECT_TRUE(eq(parts.user, url.user()));
+    EXPECT_TRUE(eq(parts.password, url.pass()));
+    EXPECT_TRUE(eq(parts.host, url.host()));
+    EXPECT_EQ(parts.port, url.port().valueOr(0));
+    EXPECT_TRUE(eq(parts.path, url.path()));
+    EXPECT_TRUE(eq(parts.query, url.query()));
+    EXPECT_TRUE(eq(parts.fragment, url.fragmentIdentifier()));
+    EXPECT_TRUE(eq(parts.string, url.string()));
+    
+    if (testTabs == TestTabs::No)
+        return;
+    
+    for (size_t i = 0; i < urlString.length(); ++i) {
+        String urlStringWithTab = insertTabAtLocation(urlString, i);
+        checkURL(urlStringWithTab, encoding,
+            parts.isInvalid() ? invalidParts(urlStringWithTab) : parts,
+            TestTabs::No);
+    }
+}
+
+static void checkURL(const String& urlString, const String& baseURLString, const TextEncoding& encoding, const ExpectedParts& parts, TestTabs testTabs = TestTabs::Yes)
+{
+    URLParser baseParser(baseURLString, { }, encoding);
+    URLParser parser(urlString, baseParser.result(), encoding);
+    auto url = parser.result();
+    EXPECT_TRUE(eq(parts.protocol, url.protocol().toString()));
+    EXPECT_TRUE(eq(parts.user, url.user()));
+    EXPECT_TRUE(eq(parts.password, url.pass()));
+    EXPECT_TRUE(eq(parts.host, url.host()));
+    EXPECT_EQ(parts.port, url.port().valueOr(0));
+    EXPECT_TRUE(eq(parts.path, url.path()));
+    EXPECT_TRUE(eq(parts.query, url.query()));
+    EXPECT_TRUE(eq(parts.fragment, url.fragmentIdentifier()));
+    EXPECT_TRUE(eq(parts.string, url.string()));
+    
+    if (testTabs == TestTabs::No)
+        return;
+    
+    for (size_t i = 0; i < urlString.length(); ++i) {
+        String urlStringWithTab = insertTabAtLocation(urlString, i);
+        checkURL(urlStringWithTab, baseURLString, encoding,
+            parts.isInvalid() ? invalidParts(urlStringWithTab) : parts,
+            TestTabs::No);
+    }
+}
+
 template<size_t length>
 static String utf16String(const char16_t (&url)[length])
 {
@@ -341,51 +537,6 @@ TEST_F(URLParserTest, Basic)
     checkURL("http://:@host", {"http", "", "", "host", 0, "/", "", "", "http://host/"});
 }
 
-static void checkRelativeURL(const String& urlString, const String& baseURLString, const ExpectedParts& parts, TestTabs testTabs = TestTabs::Yes)
-{
-    bool wasEnabled = URLParser::enabled();
-    URLParser::setEnabled(true);
-    auto url = URL(URL(URL(), baseURLString), urlString);
-    URLParser::setEnabled(false);
-    auto oldURL = URL(URL(URL(), baseURLString), urlString);
-    URLParser::setEnabled(wasEnabled);
-
-    EXPECT_TRUE(eq(parts.protocol, url.protocol().toString()));
-    EXPECT_TRUE(eq(parts.user, url.user()));
-    EXPECT_TRUE(eq(parts.password, url.pass()));
-    EXPECT_TRUE(eq(parts.host, url.host()));
-    EXPECT_EQ(parts.port, url.port().valueOr(0));
-    EXPECT_TRUE(eq(parts.path, url.path()));
-    EXPECT_TRUE(eq(parts.query, url.query()));
-    EXPECT_TRUE(eq(parts.fragment, url.fragmentIdentifier()));
-    EXPECT_TRUE(eq(parts.string, url.string()));
-
-    EXPECT_TRUE(eq(parts.protocol, oldURL.protocol().toString()));
-    EXPECT_TRUE(eq(parts.user, oldURL.user()));
-    EXPECT_TRUE(eq(parts.password, oldURL.pass()));
-    EXPECT_TRUE(eq(parts.host, oldURL.host()));
-    EXPECT_EQ(parts.port, oldURL.port().valueOr(0));
-    EXPECT_TRUE(eq(parts.path, oldURL.path()));
-    EXPECT_TRUE(eq(parts.query, oldURL.query()));
-    EXPECT_TRUE(eq(parts.fragment, oldURL.fragmentIdentifier()));
-    EXPECT_TRUE(eq(parts.string, oldURL.string()));
-
-    EXPECT_TRUE(URLParser::allValuesEqual(url, oldURL));
-    EXPECT_TRUE(URLParser::internalValuesConsistent(url));
-    EXPECT_TRUE(URLParser::internalValuesConsistent(oldURL));
-    
-    if (testTabs == TestTabs::No)
-        return;
-
-    for (size_t i = 0; i < urlString.length(); ++i) {
-        String urlStringWithTab = insertTabAtLocation(urlString, i);
-        checkRelativeURL(urlStringWithTab,
-            baseURLString,
-            parts.isInvalid() ? invalidParts(urlStringWithTab) : parts,
-            TestTabs::No);
-    }
-}
-
 TEST_F(URLParserTest, ParseRelative)
 {
     checkRelativeURL("/index.html", "http://webkit.org/path1/path2/", {"http", "", "", "webkit.org", 0, "/index.html", "", "", "http://webkit.org/index.html"});
@@ -395,7 +546,9 @@ TEST_F(URLParserTest, ParseRelative)
     checkRelativeURL("http://example\t.\norg", "http://example.org/foo/bar", {"http", "", "", "example.org", 0, "/", "", "", "http://example.org/"});
     checkRelativeURL("test", "file:///path1/path2", {"file", "", "", "", 0, "/path1/test", "", "", "file:///path1/test"});
     checkRelativeURL(utf16String(u"http://www.foo。bar.com"), "http://other.com/", {"http", "", "", "www.foo.bar.com", 0, "/", "", "", "http://www.foo.bar.com/"});
-    checkRelativeURL(utf16String(u"sc://ñ.test/"), "about:blank", {"sc", "", "", "xn--ida.test", 0, "/", "", "", "sc://xn--ida.test/"});
+    checkRelativeURLDifferences(utf16String(u"sc://ñ.test/"), "about:blank",
+        {"sc", "", "", "%C3%B1.test", 0, "/", "", "", "sc://%C3%B1.test/"},
+        {"sc", "", "", "xn--ida.test", 0, "/", "", "", "sc://xn--ida.test/"});
     checkRelativeURL("#fragment", "http://host/path", {"http", "", "", "host", 0, "/path", "", "fragment", "http://host/path#fragment"});
     checkRelativeURL("#fragment", "file:///path", {"file", "", "", "", 0, "/path", "", "fragment", "file:///path#fragment"});
     checkRelativeURL("#fragment", "file:///path#old", {"file", "", "", "", 0, "/path", "", "fragment", "file:///path#fragment"});
@@ -467,96 +620,6 @@ TEST_F(URLParserTest, ParseRelative)
     checkRelativeURL("http:\\\\host\\foo", "about:blank", {"http", "", "", "host", 0, "/foo", "", "", "http://host/foo"});
 }
 
-static void checkURLDifferences(const String& urlString, const ExpectedParts& partsNew, const ExpectedParts& partsOld, TestTabs testTabs = TestTabs::Yes)
-{
-    bool wasEnabled = URLParser::enabled();
-    URLParser::setEnabled(true);
-    auto url = URL(URL(), urlString);
-    URLParser::setEnabled(false);
-    auto oldURL = URL(URL(), urlString);
-    URLParser::setEnabled(wasEnabled);
-
-    EXPECT_TRUE(eq(partsNew.protocol, url.protocol().toString()));
-    EXPECT_TRUE(eq(partsNew.user, url.user()));
-    EXPECT_TRUE(eq(partsNew.password, url.pass()));
-    EXPECT_TRUE(eq(partsNew.host, url.host()));
-    EXPECT_EQ(partsNew.port, url.port().valueOr(0));
-    EXPECT_TRUE(eq(partsNew.path, url.path()));
-    EXPECT_TRUE(eq(partsNew.query, url.query()));
-    EXPECT_TRUE(eq(partsNew.fragment, url.fragmentIdentifier()));
-    EXPECT_TRUE(eq(partsNew.string, url.string()));
-    
-    EXPECT_TRUE(eq(partsOld.protocol, oldURL.protocol().toString()));
-    EXPECT_TRUE(eq(partsOld.user, oldURL.user()));
-    EXPECT_TRUE(eq(partsOld.password, oldURL.pass()));
-    EXPECT_TRUE(eq(partsOld.host, oldURL.host()));
-    EXPECT_EQ(partsOld.port, oldURL.port().valueOr(0));
-    EXPECT_TRUE(eq(partsOld.path, oldURL.path()));
-    EXPECT_TRUE(eq(partsOld.query, oldURL.query()));
-    EXPECT_TRUE(eq(partsOld.fragment, oldURL.fragmentIdentifier()));
-    EXPECT_TRUE(eq(partsOld.string, oldURL.string()));
-    
-    EXPECT_FALSE(URLParser::allValuesEqual(url, oldURL));
-    EXPECT_TRUE(URLParser::internalValuesConsistent(url));
-    EXPECT_TRUE(URLParser::internalValuesConsistent(oldURL));
-    
-    if (testTabs == TestTabs::No)
-        return;
-
-    for (size_t i = 0; i < urlString.length(); ++i) {
-        String urlStringWithTab = insertTabAtLocation(urlString, i);
-        checkURLDifferences(urlStringWithTab,
-            partsNew.isInvalid() ? invalidParts(urlStringWithTab) : partsNew,
-            partsOld.isInvalid() ? invalidParts(urlStringWithTab) : partsOld,
-            TestTabs::No);
-    }
-}
-
-static void checkRelativeURLDifferences(const String& urlString, const String& baseURLString, const ExpectedParts& partsNew, const ExpectedParts& partsOld, TestTabs testTabs = TestTabs::Yes)
-{
-    bool wasEnabled = URLParser::enabled();
-    URLParser::setEnabled(true);
-    auto url = URL(URL(URL(), baseURLString), urlString);
-    URLParser::setEnabled(false);
-    auto oldURL = URL(URL(URL(), baseURLString), urlString);
-    URLParser::setEnabled(wasEnabled);
-
-    EXPECT_TRUE(eq(partsNew.protocol, url.protocol().toString()));
-    EXPECT_TRUE(eq(partsNew.user, url.user()));
-    EXPECT_TRUE(eq(partsNew.password, url.pass()));
-    EXPECT_TRUE(eq(partsNew.host, url.host()));
-    EXPECT_EQ(partsNew.port, url.port().valueOr(0));
-    EXPECT_TRUE(eq(partsNew.path, url.path()));
-    EXPECT_TRUE(eq(partsNew.query, url.query()));
-    EXPECT_TRUE(eq(partsNew.fragment, url.fragmentIdentifier()));
-    EXPECT_TRUE(eq(partsNew.string, url.string()));
-    
-    EXPECT_TRUE(eq(partsOld.protocol, oldURL.protocol().toString()));
-    EXPECT_TRUE(eq(partsOld.user, oldURL.user()));
-    EXPECT_TRUE(eq(partsOld.password, oldURL.pass()));
-    EXPECT_TRUE(eq(partsOld.host, oldURL.host()));
-    EXPECT_EQ(partsOld.port, oldURL.port().valueOr(0));
-    EXPECT_TRUE(eq(partsOld.path, oldURL.path()));
-    EXPECT_TRUE(eq(partsOld.query, oldURL.query()));
-    EXPECT_TRUE(eq(partsOld.fragment, oldURL.fragmentIdentifier()));
-    EXPECT_TRUE(eq(partsOld.string, oldURL.string()));
-    
-    EXPECT_FALSE(URLParser::allValuesEqual(url, oldURL));
-    EXPECT_TRUE(URLParser::internalValuesConsistent(url));
-    EXPECT_TRUE(URLParser::internalValuesConsistent(oldURL));
-
-    if (testTabs == TestTabs::No)
-        return;
-
-    for (size_t i = 0; i < urlString.length(); ++i) {
-        String urlStringWithTab = insertTabAtLocation(urlString, i);
-        checkRelativeURLDifferences(urlStringWithTab, baseURLString,
-            partsNew.isInvalid() ? invalidParts(urlStringWithTab) : partsNew,
-            partsOld.isInvalid() ? invalidParts(urlStringWithTab) : partsOld,
-            TestTabs::No);
-    }
-}
-
 // These are differences between the new URLParser and the old URL::parse which make URLParser more standards compliant.
 TEST_F(URLParserTest, ParserDifferences)
 {
@@ -892,13 +955,11 @@ TEST_F(URLParserTest, ParserDifferences)
         {"http", "", "", "f", 10, "/c", "", "", "http://f:10/c"},
         {"http", "", "", "f", 10, "/c", "", "", "http://f:010/c"});
     checkURL("notspecial://HoSt", {"notspecial", "", "", "HoSt", 0, "", "", "", "notspecial://HoSt"});
-    checkURLDifferences("notspecial://H%6FSt",
-        {"notspecial", "", "", "HoSt", 0, "", "", "", "notspecial://HoSt"},
-        {"notspecial", "", "", "H%6FSt", 0, "", "", "", "notspecial://H%6FSt"});
-    checkURLDifferences("notspecial://H%4fSt",
-        {"notspecial", "", "", "HOSt", 0, "", "", "", "notspecial://HOSt"},
-        {"notspecial", "", "", "H%4fSt", 0, "", "", "", "notspecial://H%4fSt"});
-    checkURL(utf16String(u"notspecial://H😍ßt"), {"notspecial", "", "", "xn--hsst-qc83c", 0, "", "", "", "notspecial://xn--hsst-qc83c"}, testTabsValueForSurrogatePairs);
+    checkURL("notspecial://H%6FSt", {"notspecial", "", "", "H%6FSt", 0, "", "", "", "notspecial://H%6FSt"});
+    checkURL("notspecial://H%4fSt", {"notspecial", "", "", "H%4fSt", 0, "", "", "", "notspecial://H%4fSt"});
+    checkURLDifferences(utf16String(u"notspecial://H😍ßt"),
+        {"notspecial", "", "", "H%F0%9F%98%8D%C3%9Ft", 0, "", "", "", "notspecial://H%F0%9F%98%8D%C3%9Ft"},
+        {"notspecial", "", "", "xn--hsst-qc83c", 0, "", "", "", "notspecial://xn--hsst-qc83c"}, testTabsValueForSurrogatePairs);
     checkURLDifferences("http://[ffff:aaaa:cccc:eeee:bbbb:dddd:255.255.255.255]/",
         {"http", "", "", "[ffff:aaaa:cccc:eeee:bbbb:dddd:ffff:ffff]", 0, "/", "", "", "http://[ffff:aaaa:cccc:eeee:bbbb:dddd:ffff:ffff]/"},
         {"http", "", "", "[ffff:aaaa:cccc:eeee:bbbb:dddd:255.255.255.255]", 0, "/", "", "", "http://[ffff:aaaa:cccc:eeee:bbbb:dddd:255.255.255.255]/"}, TestTabs::No);
@@ -981,7 +1042,37 @@ TEST_F(URLParserTest, ParserDifferences)
         {"http", "", "", "[0:0:0:0:a:b:c:d]", 0, "/", "", "", "http://[0:0:0:0:a:b:c:d]/"});
     checkURLDifferences("asdf://[0:0:0:0:a:b:c:d]",
         {"asdf", "", "", "[::a:b:c:d]", 0, "", "", "", "asdf://[::a:b:c:d]"},
-        {"asdf", "", "", "[0:0:0:0:a:b:c:d]", 0, "", "", "", "asdf://[0:0:0:0:a:b:c:d]"});
+        {"asdf", "", "", "[0:0:0:0:a:b:c:d]", 0, "", "", "", "asdf://[0:0:0:0:a:b:c:d]"}, TestTabs::No);
+    shouldFail("a://%:a");
+    checkURL("a://%:/", {"a", "", "", "%", 0, "/", "", "", "a://%/"});
+    checkURL("a://%:", {"a", "", "", "%", 0, "", "", "", "a://%"});
+    checkURL("a://%:1/", {"a", "", "", "%", 1, "/", "", "", "a://%:1/"});
+    checkURLDifferences("http://%:",
+        {"", "", "", "", 0, "", "", "", "http://%:"},
+        {"http", "", "", "%", 0, "/", "", "", "http://%/"});
+    checkURL("a://123456", {"a", "", "", "123456", 0, "", "", "", "a://123456"});
+    checkURL("a://123456:7", {"a", "", "", "123456", 7, "", "", "", "a://123456:7"});
+    checkURL("a://123456:7/", {"a", "", "", "123456", 7, "/", "", "", "a://123456:7/"});
+    checkURL("a://A", {"a", "", "", "A", 0, "", "", "", "a://A"});
+    checkURL("a://A:2", {"a", "", "", "A", 2, "", "", "", "a://A:2"});
+    checkURL("a://A:2/", {"a", "", "", "A", 2, "/", "", "", "a://A:2/"});
+    checkURLDifferences("asd://ß",
+        {"asd", "", "", "%C3%83%C2%9F", 0, "", "", "", "asd://%C3%83%C2%9F"},
+        {"", "", "", "", 0, "", "", "", "about:blank"}, TestTabs::No);
+    checkURLDifferences("asd://ß:4",
+        {"asd", "", "", "%C3%83%C2%9F", 4, "", "", "", "asd://%C3%83%C2%9F:4"},
+        {"", "", "", "", 0, "", "", "", "about:blank"}, TestTabs::No);
+    checkURLDifferences("asd://ß:4/",
+        {"asd", "", "", "%C3%83%C2%9F", 4, "/", "", "", "asd://%C3%83%C2%9F:4/"},
+        {"", "", "", "", 0, "", "", "", "about:blank"}, TestTabs::No);
+    checkURLDifferences("a://[A::b]:4",
+        {"a", "", "", "[a::b]", 4, "", "", "", "a://[a::b]:4"},
+        {"a", "", "", "[A::b]", 4, "", "", "", "a://[A::b]:4"});
+    shouldFail("http://[~]");
+    shouldFail("a://[~]");
+    checkRelativeURLDifferences("a://b", "//[aBc]",
+        {"a", "", "", "b", 0, "", "", "", "a://b"},
+        {"", "", "", "", 0, "", "", "", "a://b"});
 }
 
 TEST_F(URLParserTest, DefaultPort)
@@ -1073,16 +1164,6 @@ TEST_F(URLParserTest, DefaultPort)
         {"", "", "", "", 0, "", "", "", "file://:0/path"},
         {"file", "", "", "", 0, "/path", "", "", "file://:0/path"});
 }
-    
-static void shouldFail(const String& urlString)
-{
-    checkURL(urlString, {"", "", "", "", 0, "", "", "", urlString});
-}
-
-static void shouldFail(const String& urlString, const String& baseString)
-{
-    checkRelativeURL(urlString, baseString, {"", "", "", "", 0, "", "", "", urlString});
-}
 
 TEST_F(URLParserTest, ParserFailures)
 {
@@ -1137,7 +1218,9 @@ TEST_F(URLParserTest, ParserFailures)
     shouldFail("http://[a:b:c:d:e:f:g:h:127.0.0.1]");
     shouldFail("http://[a:b:c:d:e:f:127.0.0.0x11]"); // Chrome treats this as hex, Firefox and the spec fail
     shouldFail("http://[a:b:c:d:e:f:127.0.-0.1]");
-    shouldFail("asdf://space InHost");
+    checkURLDifferences("asdf://space In\aHost",
+        {"asdf", "", "", "space In%07Host", 0, "", "", "", "asdf://space In%07Host"},
+        {"", "", "", "", 0, "", "", "", "asdf://space In\aHost"});
     shouldFail("asdf://[0:0:0:0:a:b:c:d");
 }
 
@@ -1179,57 +1262,6 @@ TEST_F(URLParserTest, AdditionalTests)
     // FIXME: Write more invalid surrogate pair tests based on feedback from https://bugs.webkit.org/show_bug.cgi?id=162105
 }
 
-static void checkURL(const String& urlString, const TextEncoding& encoding, const ExpectedParts& parts, TestTabs testTabs = TestTabs::Yes)
-{
-    URLParser parser(urlString, { }, encoding);
-    auto url = parser.result();
-    EXPECT_TRUE(eq(parts.protocol, url.protocol().toString()));
-    EXPECT_TRUE(eq(parts.user, url.user()));
-    EXPECT_TRUE(eq(parts.password, url.pass()));
-    EXPECT_TRUE(eq(parts.host, url.host()));
-    EXPECT_EQ(parts.port, url.port().valueOr(0));
-    EXPECT_TRUE(eq(parts.path, url.path()));
-    EXPECT_TRUE(eq(parts.query, url.query()));
-    EXPECT_TRUE(eq(parts.fragment, url.fragmentIdentifier()));
-    EXPECT_TRUE(eq(parts.string, url.string()));
-
-    if (testTabs == TestTabs::No)
-        return;
-
-    for (size_t i = 0; i < urlString.length(); ++i) {
-        String urlStringWithTab = insertTabAtLocation(urlString, i);
-        checkURL(urlStringWithTab, encoding,
-            parts.isInvalid() ? invalidParts(urlStringWithTab) : parts,
-            TestTabs::No);
-    }
-}
-
-static void checkURL(const String& urlString, const String& baseURLString, const TextEncoding& encoding, const ExpectedParts& parts, TestTabs testTabs = TestTabs::Yes)
-{
-    URLParser baseParser(baseURLString, { }, encoding);
-    URLParser parser(urlString, baseParser.result(), encoding);
-    auto url = parser.result();
-    EXPECT_TRUE(eq(parts.protocol, url.protocol().toString()));
-    EXPECT_TRUE(eq(parts.user, url.user()));
-    EXPECT_TRUE(eq(parts.password, url.pass()));
-    EXPECT_TRUE(eq(parts.host, url.host()));
-    EXPECT_EQ(parts.port, url.port().valueOr(0));
-    EXPECT_TRUE(eq(parts.path, url.path()));
-    EXPECT_TRUE(eq(parts.query, url.query()));
-    EXPECT_TRUE(eq(parts.fragment, url.fragmentIdentifier()));
-    EXPECT_TRUE(eq(parts.string, url.string()));
-    
-    if (testTabs == TestTabs::No)
-        return;
-
-    for (size_t i = 0; i < urlString.length(); ++i) {
-        String urlStringWithTab = insertTabAtLocation(urlString, i);
-        checkURL(urlStringWithTab, baseURLString, encoding,
-            parts.isInvalid() ? invalidParts(urlStringWithTab) : parts,
-            TestTabs::No);
-    }
-}
-
 TEST_F(URLParserTest, QueryEncoding)
 {
     checkURL(utf16String(u"http://host?ß😍#ß😍"), UTF8Encoding(), {"http", "", "", "host", 0, "/", "%C3%9F%F0%9F%98%8D", "%C3%9F%F0%9F%98%8D", utf16String(u"http://host/?%C3%9F%F0%9F%98%8D#%C3%9F%F0%9F%98%8D")}, testTabsValueForSurrogatePairs);