Refactor HTML tokenizer code in preparation for a new XML tokenizer
authorjpfau@apple.com <jpfau@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Sun, 24 Jul 2011 10:07:26 +0000 (10:07 +0000)
committerjpfau@apple.com <jpfau@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Sun, 24 Jul 2011 10:07:26 +0000 (10:07 +0000)
commit09073e86b98ff5bf546f6b1c31c38bc192227990
tree15a3d39919901028945925ea8f73e8d0fd9f74cf
parent07f27f021016b89d586b5156b25a1c0bc81d9452
Refactor HTML tokenizer code in preparation for a new XML tokenizer
https://bugs.webkit.org/show_bug.cgi?id=65000

Reviewed by Adam Barth.

* GNUmakefile.list.am:
* WebCore.gypi:
* WebCore.pro:
* WebCore.vcproj/WebCore.vcproj:
* WebCore.xcodeproj/project.pbxproj:
* html/parser/HTMLDocumentParser.cpp:
(WebCore::HTMLNames::tokenizerStateForContextElement):
(WebCore::HTMLDocumentParser::pumpTokenizer):
* html/parser/HTMLTokenizer.cpp:
(WebCore::HTMLNames::isEndTagBufferingState):
(WebCore::HTMLTokenizer::HTMLTokenizer):
(WebCore::::shouldSkipNullCharacters):
(WebCore::HTMLTokenizer::reset):
(WebCore::HTMLTokenizer::flushEmitAndResumeIn):
(WebCore::HTMLTokenizer::nextToken):
(WebCore::HTMLTokenizer::updateStateFor):
* html/parser/HTMLTokenizer.h:
(WebCore::HTMLTokenizer::emitAndResumeIn):
(WebCore::HTMLTokenizer::emitAndReconsumeIn):
(WebCore::HTMLTokenizer::saveEndTagNameIfNeeded):
* html/parser/HTMLTreeBuilder.cpp:
(WebCore::HTMLTreeBuilder::processStartTagForInBody):
(WebCore::HTMLTreeBuilder::processEndTag):
(WebCore::HTMLTreeBuilder::processGenericRCDATAStartTag):
(WebCore::HTMLTreeBuilder::processGenericRawTextStartTag):
(WebCore::HTMLTreeBuilder::processScriptStartTag):
* html/parser/TextDocumentParser.cpp:
(WebCore::TextDocumentParser::TextDocumentParser):
* html/parser/TextViewSourceParser.cpp:
(WebCore::TextViewSourceParser::TextViewSourceParser):
* xml/parser/MarkupTokenBase.h:
(WebCore::MarkupTokenBase::type):
* xml/parser/MarkupTokenizerBase.h: Added.
(WebCore::MarkupTokenizerBase::~MarkupTokenizerBase):
(WebCore::MarkupTokenizerBase::lineNumber):
(WebCore::MarkupTokenizerBase::columnNumber):
(WebCore::MarkupTokenizerBase::state):
(WebCore::MarkupTokenizerBase::setState):
(WebCore::MarkupTokenizerBase::forceNullCharacterReplacement):
(WebCore::MarkupTokenizerBase::setForceNullCharacterReplacement):
(WebCore::MarkupTokenizerBase::InputStreamPreprocessor::InputStreamPreprocessor):
(WebCore::MarkupTokenizerBase::InputStreamPreprocessor::nextInputCharacter):
(WebCore::MarkupTokenizerBase::InputStreamPreprocessor::peek):
(WebCore::MarkupTokenizerBase::InputStreamPreprocessor::advance):
(WebCore::MarkupTokenizerBase::InputStreamPreprocessor::shouldTreatNullAsEndOfFileMarker):
(WebCore::MarkupTokenizerBase::MarkupTokenizerBase):
(WebCore::MarkupTokenizerBase::bufferCharacter):
(WebCore::MarkupTokenizerBase::emitAndResumeIn):
(WebCore::MarkupTokenizerBase::emitAndReconsumeIn):
(WebCore::MarkupTokenizerBase::emitEndOfFile):
(WebCore::MarkupTokenizerBase::reset):
(WebCore::MarkupTokenizerBase::haveBufferedCharacterToken):
* xml/parser/MarkupTokenizerInlineMethods.h: Added.
(WebCore::isTokenizerWhitespace):
(WebCore::advanceStringAndASSERTIgnoringCase):
(WebCore::advanceStringAndASSERT):

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@91643 268f45cc-cd09-0410-ab3c-d52691b4dbfc
15 files changed:
Source/WebCore/ChangeLog
Source/WebCore/GNUmakefile.list.am
Source/WebCore/WebCore.gypi
Source/WebCore/WebCore.pro
Source/WebCore/WebCore.vcproj/WebCore.vcproj
Source/WebCore/WebCore.xcodeproj/project.pbxproj
Source/WebCore/html/parser/HTMLDocumentParser.cpp
Source/WebCore/html/parser/HTMLTokenizer.cpp
Source/WebCore/html/parser/HTMLTokenizer.h
Source/WebCore/html/parser/HTMLTreeBuilder.cpp
Source/WebCore/html/parser/TextDocumentParser.cpp
Source/WebCore/html/parser/TextViewSourceParser.cpp
Source/WebCore/xml/parser/MarkupTokenBase.h
Source/WebCore/xml/parser/MarkupTokenizerBase.h [new file with mode: 0644]
Source/WebCore/xml/parser/MarkupTokenizerInlineMethods.h [new file with mode: 0644]