[WHLSL] Add a handwritten lexer
authormmaxfield@apple.com <mmaxfield@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 19 Dec 2018 23:27:45 +0000 (23:27 +0000)
committermmaxfield@apple.com <mmaxfield@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 19 Dec 2018 23:27:45 +0000 (23:27 +0000)
commite350c3d9d0a278d5dca8654f1b63b3e69e4afdd9
tree1d2322603f6911cfe5e970772b6b357cf7ef8810
parenta20782ec4171b003bede9af680da628fe2e5b1b8
[WHLSL] Add a handwritten lexer
https://bugs.webkit.org/show_bug.cgi?id=192294

Reviewed by Jon Lee.

This is infrastructure necessary for https://bugs.webkit.org/show_bug.cgi?id=192355. The
implementation matches the lexing rules in the spec (specifically, the rules that start
with an uppercase letter). The spec is at
https://github.com/gpuweb/WHLSL/blob/master/Spec/WHLSL.g4.

This patch also modifies the lexer according to https://github.com/gpuweb/WHLSL/pull/283.

No new tests because the lexer isn't hooked up yet; there are tests in the parser,
once that gets committed.

* Modules/webgpu/WHLSL/WHLSLLexer.cpp: Added.
(WebCore::WHLSL::Lexer::Token::typeName):
(WebCore::WHLSL::Lexer::recognizeKeyword):
(WebCore::WHLSL::Lexer::consumeTokenFromStream):
(WebCore::WHLSL::Lexer::skipWhitespaceAndComments):
(WebCore::WHLSL::isWhitespace):
(WebCore::WHLSL::isNewline):
(WebCore::WHLSL::Lexer::skipWhitespace):
(WebCore::WHLSL::Lexer::skipLineComment):
(WebCore::WHLSL::Lexer::skipLongComment):
(WebCore::WHLSL::Lexer::coreDecimalIntLiteral const):
(WebCore::WHLSL::Lexer::decimalIntLiteral const):
(WebCore::WHLSL::Lexer::decimalUintLiteral const):
(WebCore::WHLSL::isHexadecimalCharacter):
(WebCore::WHLSL::Lexer::coreHexadecimalIntLiteral const):
(WebCore::WHLSL::Lexer::hexadecimalIntLiteral const):
(WebCore::WHLSL::Lexer::hexadecimalUintLiteral const):
(WebCore::WHLSL::Lexer::intLiteral const):
(WebCore::WHLSL::Lexer::uintLiteral const):
(WebCore::WHLSL::Lexer::digit const):
(WebCore::WHLSL::Lexer::digitStar const):
(WebCore::WHLSL::Lexer::character const):
(WebCore::WHLSL::Lexer::coreFloatLiteralType1 const):
(WebCore::WHLSL::Lexer::coreFloatLiteral const):
(WebCore::WHLSL::Lexer::floatLiteral const):
(WebCore::WHLSL::Lexer::validIdentifier const):
(WebCore::WHLSL::Lexer::identifier const):
(WebCore::WHLSL::Lexer::operatorName const):
* Modules/webgpu/WHLSL/WHLSLLexer.h: Added.
(WebCore::WHLSL::Lexer::Lexer):
(WebCore::WHLSL::Lexer::consumeToken):
(WebCore::WHLSL::Lexer::unconsumeToken):
(WebCore::WHLSL::Lexer::state const):
(WebCore::WHLSL::Lexer::setState):
(WebCore::WHLSL::Lexer::isFullyConsumed const):
(WebCore::WHLSL::Lexer::errorString):
(WebCore::WHLSL::Lexer::string const):
(WebCore::WHLSL::Lexer::anyCharacter const):
* Sources.txt:
* WebCore.xcodeproj/project.pbxproj:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@239398 268f45cc-cd09-0410-ab3c-d52691b4dbfc
Source/WebCore/ChangeLog
Source/WebCore/Modules/webgpu/WHLSL/WHLSLLexer.cpp [new file with mode: 0644]
Source/WebCore/Modules/webgpu/WHLSL/WHLSLLexer.h [new file with mode: 0644]
Source/WebCore/Sources.txt
Source/WebCore/WebCore.xcodeproj/project.pbxproj