[JSC] Relax line terminators in String to make JSON subset of JS
authorutatane.tea@gmail.com <utatane.tea@gmail.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 30 Jan 2018 07:34:44 +0000 (07:34 +0000)
committerutatane.tea@gmail.com <utatane.tea@gmail.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 30 Jan 2018 07:34:44 +0000 (07:34 +0000)
https://bugs.webkit.org/show_bug.cgi?id=182232

Reviewed by Keith Miller.

JSTests:

* ChakraCore/test/es5/Lex_u3.baseline-jsc:
* stress/relaxed-line-terminators-in-string.js: Added.
(shouldBe):

Source/JavaScriptCore:

"Subsume JSON" spec is now stage 3[1]. Before this spec change,
JSON can accept \u2028 / \u2029 in string while JS cannot do that.
It accidentally made JSON non subset of JS.

Now we extend our JS string to accept \u2028 / \u2029 to make JSON
subset of JS in this spec change.

[1]: https://github.com/tc39/proposal-json-superset

* parser/Lexer.cpp:
(JSC::Lexer<T>::parseStringSlowCase):

LayoutTests:

* sputnik/Conformance/07_Lexical_Conventions/7.3_Line_Terminators/S7.3_A2.3-expected.txt:
* sputnik/Conformance/07_Lexical_Conventions/7.3_Line_Terminators/S7.3_A2.3.html:
* sputnik/Conformance/07_Lexical_Conventions/7.3_Line_Terminators/S7.3_A2.4-expected.txt:
* sputnik/Conformance/07_Lexical_Conventions/7.3_Line_Terminators/S7.3_A2.4.html:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@227775 268f45cc-cd09-0410-ab3c-d52691b4dbfc

JSTests/ChakraCore/test/es5/Lex_u3.baseline-jsc
JSTests/ChangeLog
JSTests/stress/relaxed-line-terminators-in-string.js [new file with mode: 0644]
LayoutTests/ChangeLog
LayoutTests/sputnik/Conformance/07_Lexical_Conventions/7.3_Line_Terminators/S7.3_A2.3-expected.txt
LayoutTests/sputnik/Conformance/07_Lexical_Conventions/7.3_Line_Terminators/S7.3_A2.3.html
LayoutTests/sputnik/Conformance/07_Lexical_Conventions/7.3_Line_Terminators/S7.3_A2.4-expected.txt
LayoutTests/sputnik/Conformance/07_Lexical_Conventions/7.3_Line_Terminators/S7.3_A2.4.html
Source/JavaScriptCore/ChangeLog
Source/JavaScriptCore/parser/Lexer.cpp

index 48aefc6..703ad30 100644 (file)
@@ -2,7 +2,7 @@ undefined
 undefined
 str const Left 
  str const right
-LS in string -  compile failure in ES5: expected.SyntaxError: Unexpected EOF
+str%20const%20Left%20%u2028%20str%20const%20right
 LS in regex literal -  compile failure in ES5: expected.SyntaxError: Unterminated regular expression literal '/str const regex '
 LS%20in%20escape%20sequence%20string%20literal%20%20%3Amore%20string
 BOM is WS :  91
index 450a3ad..a1fe362 100644 (file)
@@ -1,3 +1,14 @@
+2018-01-29  Yusuke Suzuki  <utatane.tea@gmail.com>
+
+        [JSC] Relax line terminators in String to make JSON subset of JS
+        https://bugs.webkit.org/show_bug.cgi?id=182232
+
+        Reviewed by Keith Miller.
+
+        * ChakraCore/test/es5/Lex_u3.baseline-jsc:
+        * stress/relaxed-line-terminators-in-string.js: Added.
+        (shouldBe):
+
 2018-01-29  Michael Saboff  <msaboff@apple.com>
 
         REGRESSION (r227341): DFG_ASSERT failure at JSC::DFG::AtTailAbstractState::forNode()
diff --git a/JSTests/stress/relaxed-line-terminators-in-string.js b/JSTests/stress/relaxed-line-terminators-in-string.js
new file mode 100644 (file)
index 0000000..c675ac8
--- /dev/null
@@ -0,0 +1,14 @@
+function shouldBe(actual, expected) {
+    if (actual !== expected)
+        throw new Error('bad value: ' + actual);
+}
+
+shouldBe(eval("'\u2028Japanese'").charCodeAt(0), 0x2028);
+shouldBe(eval("'\u2029Japanese'").charCodeAt(0), 0x2029);
+shouldBe(eval("'\u2028日本語'").charCodeAt(0), 0x2028);
+shouldBe(eval("'\u2029日本語'").charCodeAt(0), 0x2029);
+
+shouldBe(eval("'\u2028Japanese' + 'hello' + 'world'").charCodeAt(0), 0x2028);
+shouldBe(eval("'\u2029Japanese' + 'hello' + 'world'").charCodeAt(0), 0x2029);
+shouldBe(eval("'\u2028日本語' + 'hello' + 'world'").charCodeAt(0), 0x2028);
+shouldBe(eval("'\u2029日本語' + 'hello' + 'world'").charCodeAt(0), 0x2029);
index 465196f..d78323a 100644 (file)
@@ -1,3 +1,15 @@
+2018-01-29  Yusuke Suzuki  <utatane.tea@gmail.com>
+
+        [JSC] Relax line terminators in String to make JSON subset of JS
+        https://bugs.webkit.org/show_bug.cgi?id=182232
+
+        Reviewed by Keith Miller.
+
+        * sputnik/Conformance/07_Lexical_Conventions/7.3_Line_Terminators/S7.3_A2.3-expected.txt:
+        * sputnik/Conformance/07_Lexical_Conventions/7.3_Line_Terminators/S7.3_A2.3.html:
+        * sputnik/Conformance/07_Lexical_Conventions/7.3_Line_Terminators/S7.3_A2.4-expected.txt:
+        * sputnik/Conformance/07_Lexical_Conventions/7.3_Line_Terminators/S7.3_A2.4.html:
+
 2018-01-29  Youenn Fablet  <youenn@apple.com>
 
         ServiceWorkerClientFetch should not consider responses without Location headers as redirection responses
index e5bca64..fe0a8ea 100644 (file)
@@ -68,14 +68,14 @@ try {
 /**
  * @name: S7.3_A2.3;
  * @section: 7.3;
- * @assertion: LINE SEPARATOR (U+2028) within strings is not allowed;
+ * @assertion: LINE SEPARATOR (U+2028) within strings is allowed;
  * @description: Insert LINE SEPARATOR (\u2028) into string;  
  * @negative
 */
 
 // CHECK#1
-if (eval("'\u2028str\u2028ing\u2028'") === "\u2028str\u2028ing\u2028") {
-  testFailed('#1: eval("\'\\u2028str\\u2028ing\\u2028\'") === "\\u2028str\\u2028ing\\u2028"');
+if (eval("'\u2028str\u2028ing\u2028'") !== "\u2028str\u2028ing\u2028") {
+  testFailed('#1: eval("\'\\u2028str\\u2028ing\\u2028\'") !== "\\u2028str\\u2028ing\\u2028"');
 }
 
 } catch (ex) {
@@ -87,11 +87,11 @@ var successfullyParsed = true;
 
 <script>
 if (!successfullyParsed)
-    printTestPassed('Expected parsing failure');
+    printTestFailed('successfullyParsed is not set');
 else if (sputnikException)
-    printTestPassed(sputnikException);
+    printTestFailed(sputnikException);
 else
-    printTestFailed("No error detected");
+    printTestPassed("");
 testPrint('<br /><span class="pass">TEST COMPLETE</span>');
 </script>
 </body>
index 1b2bd06..5850591 100644 (file)
@@ -68,14 +68,14 @@ try {
 /**
  * @name: S7.3_A2.4;
  * @section: 7.3;
- * @assertion: PARAGRAPH SEPARATOR (U+2029) within strings is not allowed;
+ * @assertion: PARAGRAPH SEPARATOR (U+2029) within strings is allowed;
  * @description: Insert PARAGRAPH SEPARATOR (\u2029) into string;  
  * @negative
 */
 
 // CHECK#1
-if (eval("'\u2029str\u2029ing\u2029'") === "\u2029str\u2029ing\u2029") {
-  testFailed('#1: eval("\'\\u2029str\\u2029ing\\u2029\'") === "\\u2029str\\u2029ing\\u2029"');
+if (eval("'\u2029str\u2029ing\u2029'") !== "\u2029str\u2029ing\u2029") {
+  testFailed('#1: eval("\'\\u2029str\\u2029ing\\u2029\'") !== "\\u2029str\\u2029ing\\u2029"');
 }
 
 } catch (ex) {
@@ -87,11 +87,11 @@ var successfullyParsed = true;
 
 <script>
 if (!successfullyParsed)
-    printTestPassed('Expected parsing failure');
+    printTestFailed('successfullyParsed is not set');
 else if (sputnikException)
-    printTestPassed(sputnikException);
+    printTestFailed(sputnikException);
 else
-    printTestFailed("No error detected");
+    printTestPassed("");
 testPrint('<br /><span class="pass">TEST COMPLETE</span>');
 </script>
 </body>
index 9504777..e10a1c3 100644 (file)
@@ -1,3 +1,22 @@
+2018-01-29  Yusuke Suzuki  <utatane.tea@gmail.com>
+
+        [JSC] Relax line terminators in String to make JSON subset of JS
+        https://bugs.webkit.org/show_bug.cgi?id=182232
+
+        Reviewed by Keith Miller.
+
+        "Subsume JSON" spec is now stage 3[1]. Before this spec change,
+        JSON can accept \u2028 / \u2029 in string while JS cannot do that.
+        It accidentally made JSON non subset of JS.
+
+        Now we extend our JS string to accept \u2028 / \u2029 to make JSON
+        subset of JS in this spec change.
+
+        [1]: https://github.com/tc39/proposal-json-superset
+
+        * parser/Lexer.cpp:
+        (JSC::Lexer<T>::parseStringSlowCase):
+
 2018-01-29  Jiewen Tan  <jiewen_tan@apple.com>
 
         [WebAuthN] Add a compile-time feature flag
index ab585d2..7b8e97e 100644 (file)
@@ -1355,11 +1355,11 @@ template <bool shouldBuildStrings> auto Lexer<T>::parseStringSlowCase(JSTokenDat
             continue;
         }
         // Fast check for characters that require special handling.
-        // Catches 0, \n, \r, 0x2028, and 0x2029 as efficiently
-        // as possible, and lets through all common ASCII characters.
-        if (UNLIKELY(((static_cast<unsigned>(m_current) - 0xE) & 0x2000))) {
+        // Catches 0, \n, and \r as efficiently as possible, and lets through all common ASCII characters.
+        static_assert(std::is_unsigned<T>::value, "Lexer expects an unsigned character type");
+        if (UNLIKELY(m_current < 0xE)) {
             // New-line or end of input is not allowed
-            if (atEnd() || isLineTerminator(m_current)) {
+            if (atEnd() || m_current == '\r' || m_current == '\n') {
                 m_lexErrorMessage = ASCIILiteral("Unexpected EOF");
                 return atEnd() ? StringUnterminated : StringCannotBeParsed;
             }