Arrow functions with concise bodies cannot return regular expressions
authorutatane.tea@gmail.com <utatane.tea@gmail.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 25 Oct 2016 02:35:59 +0000 (02:35 +0000)
committerutatane.tea@gmail.com <utatane.tea@gmail.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 25 Oct 2016 02:35:59 +0000 (02:35 +0000)
commitd00a2371c62a22dfb2e050ed0e5fc0bf6e86b131
tree1e7818b7abbca341cb5b5b3125d913fefe4575b3
parent5a84b8407e9c57f07cf679db3a7e153855206617
Arrow functions with concise bodies cannot return regular expressions
https://bugs.webkit.org/show_bug.cgi?id=163162

Reviewed by Filip Pizlo.

JSTests:

* ChakraCore/test/Operators/instanceof.baseline-jsc:
* ChakraCore/test/Regex/nul_character.baseline-jsc:
* ChakraCore/test/es5/Lex_u3.baseline-jsc:
* stress/parse-regexp-as-token.js: Added.
(shouldBe):

Source/JavaScriptCore:

When we encounter the RegExp in the parser, we first scan it as / or /=.
And if / or /= is parsed under the primary expression context, we rescan it
as RegExp. However, we did not update the token record information. So the
token record still says "I'm / or /=".

When we parse the string "() => /hello/", the last token becomes "/", which is
the first character of the RegExp, instead of "/hello/". Since the arrow
function parsing utilizes the end offset of the last token, we accidentally
recognize the range of the above arrow function as "() => /".

In this patch, we update the token when rescanning under the RegExp context.
This logic is similar to parsing Tail Template Literal token.

We also refine the error message for regular expression literals. And since
the REGEXP token is now introduced, the other error messages using that token
are improved too.

Currently, unterminated error messages can be seen in Parser.cpp. However,
these messages cannot be shown to users if the lexer has m_error. So these
code is meaningless. I'll move these tokenizing errors to the lexer in the
subsequent patch[1].

[1]: https://bugs.webkit.org/show_bug.cgi?id=163928

* parser/Lexer.cpp:
(JSC::Lexer<T>::fillTokenInfo):
(JSC::Lexer<T>::lex):
(JSC::Lexer<T>::scanRegExp):
(JSC::Lexer<T>::scanTrailingTemplateString):
(JSC::Lexer<T>::skipRegExp): Deleted.
* parser/Lexer.h:
(JSC::Lexer::getToken):
* parser/Parser.cpp:
(JSC::Parser<LexerType>::parseAssignmentExpression):
* parser/Parser.h:
(JSC::Parser::getToken):
* parser/ParserTokens.h:

LayoutTests:

* fast/regex/dom/non-pattern-characters-expected.txt:
* js/arrowfunction-syntax-errors-expected.txt:
* js/regexp-compile-crash-expected.txt:
* sputnik/Conformance/07_Lexical_Conventions/7.4_Comments/S7.4_A4_T1-expected.txt:
* sputnik/Conformance/07_Lexical_Conventions/7.4_Comments/S7.4_A4_T4-expected.txt:
* sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A1.2_T2-expected.txt:
* sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A1.3_T1-expected.txt:
* sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A1.3_T3-expected.txt:
* sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A1.5_T1-expected.txt:
* sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A1.5_T3-expected.txt:
* sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A2.2_T1-expected.txt:
* sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A2.3_T1-expected.txt:
* sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A2.3_T3-expected.txt:
* sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A2.5_T1-expected.txt:
* sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A2.5_T3-expected.txt:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@207798 268f45cc-cd09-0410-ab3c-d52691b4dbfc
27 files changed:
JSTests/ChakraCore/test/Operators/instanceof.baseline-jsc
JSTests/ChakraCore/test/Regex/nul_character.baseline-jsc
JSTests/ChakraCore/test/es5/Lex_u3.baseline-jsc
JSTests/ChangeLog
JSTests/stress/parse-regexp-as-token.js [new file with mode: 0644]
LayoutTests/ChangeLog
LayoutTests/fast/regex/dom/non-pattern-characters-expected.txt
LayoutTests/js/arrowfunction-syntax-errors-expected.txt
LayoutTests/js/regexp-compile-crash-expected.txt
LayoutTests/sputnik/Conformance/07_Lexical_Conventions/7.4_Comments/S7.4_A4_T1-expected.txt
LayoutTests/sputnik/Conformance/07_Lexical_Conventions/7.4_Comments/S7.4_A4_T4-expected.txt
LayoutTests/sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A1.2_T2-expected.txt
LayoutTests/sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A1.3_T1-expected.txt
LayoutTests/sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A1.3_T3-expected.txt
LayoutTests/sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A1.5_T1-expected.txt
LayoutTests/sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A1.5_T3-expected.txt
LayoutTests/sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A2.2_T1-expected.txt
LayoutTests/sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A2.3_T1-expected.txt
LayoutTests/sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A2.3_T3-expected.txt
LayoutTests/sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A2.5_T1-expected.txt
LayoutTests/sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A2.5_T3-expected.txt
Source/JavaScriptCore/ChangeLog
Source/JavaScriptCore/parser/Lexer.cpp
Source/JavaScriptCore/parser/Lexer.h
Source/JavaScriptCore/parser/Parser.cpp
Source/JavaScriptCore/parser/Parser.h
Source/JavaScriptCore/parser/ParserTokens.h