Web Inspector: Formatter: Pretty Print HTML resources (including inline <script>...
authorpecoraro@apple.com <pecoraro@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 13 Sep 2019 10:05:36 +0000 (10:05 +0000)
committerpecoraro@apple.com <pecoraro@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 13 Sep 2019 10:05:36 +0000 (10:05 +0000)
commitb2071abbd7e39ae219e0c761f8646002b69e5056
treef0dabcaf18270f4c9714d2a1ae6c7807ccd76a81
parentec53328c3476771b0bf361bb9d6d9809a0269f6e
Web Inspector: Formatter: Pretty Print HTML resources (including inline <script>/<style>)
https://bugs.webkit.org/show_bug.cgi?id=201535
<rdar://problem/29119232>

Reviewed by Devin Rousso.

Source/JavaScriptCore:

* debugger/Debugger.cpp:
(JSC::Debugger::resolveBreakpoint):
When resolving a breakpoint inside of an inline <script> we need to adjust
based on the starting position of the <script> in the HTML resource.

Source/WebInspectorUI:

Introduce HTMLFormatter, HTMLParser, and HTMLTreeBuilderFormatter.

    - HTMLParser tokenizes HTML content and passes it to a tree builder.
    - HTMLTreeBuilderFormatter uses those tokens to create a formatted dom tree of fake nodes.
    - HTMLFormatter then builds formatted content and a FormatterSourceMap from that DOM tree.

The output of HTMLFormatter is intended to just modify whitespace. So it won't be a
100% compatible DOM, but will be useful for debugging and reading inline <style>
and <script>, which is a more common debugging scenario. It also maintains original
author attribute quotations or lack therefore, because again it tries to only
change whitespace. A separate tree builder could be used for tidying content.

This introduces a few tools for working with formatting:

    - Tools/HTMLFormatter - debug the parser, tree builder, and output format
    - Tools/SourceMaps - debug formatted content and source mappings

This also:

    - renames "EsprimaFormatter" to "JSFormatter"
    - eliminates an extra trailing newline in CSSFormatter output

* Tools/HTMLFormatter/HTMLTreeBuilderDebug.js: Added.
* Tools/HTMLFormatter/index.html: Added.
* Tools/HTMLFormatter/styles.css: Added.
Tool for debugging the HTML formatter components.

* Tools/SourceMaps/.gitignore: Added.
* Tools/SourceMaps/index.html: Added.
* Tools/SourceMaps/setup.sh: Added.
* Tools/SourceMaps/styles.css: Added.
Tool for debugging Formatter generated Source Maps.

* Tools/JSFormatter/JSFormatterDebug.js: Renamed from Source/WebInspectorUI/Tools/Formatting/EsprimaFormatterDebug.js.
* Tools/JSFormatter/codemirror-additions.css: Renamed from Source/WebInspectorUI/Tools/Formatting/codemirror-additions.css.
* Tools/JSFormatter/index.html: Renamed from Source/WebInspectorUI/Tools/Formatting/index.html.
Renamed this tool. EsprimaFormatter => JSFormatter.

* UserInterface/Controllers/FormatterSourceMap.js:
(WI.FormatterSourceMap.prototype.originalPositionToFormattedPosition):
(WI.FormatterSourceMap.prototype.formattedPositionToOriginalPosition):
New APIs used by tools and tests for translating "position" offsets and not line/columns.

* UserInterface/Workers/Formatter/JSFormatter.js: Renamed from Source/WebInspectorUI/UserInterface/Workers/Formatter/EsprimaFormatter.js.
(JSFormatter):
* UserInterface/Workers/Formatter/CSSFormatter.js:
(CSSFormatter):
Allow using a supplied builder, which is useful when formatting
inline content from HTMLFormatter (<style> / <script>).

* UserInterface/Workers/Formatter/FormatterContentBuilder.js:
(FormatterContentBuilder.prototype.get indentString):
(FormatterContentBuilder.prototype.get originalContent):
(FormatterContentBuilder.prototype.get indentLevel):
(FormatterContentBuilder.prototype.get originalOffset):
(FormatterContentBuilder.prototype.set originalOffset):
(FormatterContentBuilder.prototype.appendNonToken):
(FormatterContentBuilder.prototype.appendToken):
(FormatterContentBuilder.prototype.appendTokenWithPossibleNewlines):
(FormatterContentBuilder.prototype.appendMapping):
(FormatterContentBuilder.prototype.indentToLevel):
(FormatterContentBuilder.prototype.finish):
Provide API for getting / setting the original offset and indent level.
This is useful when the HTMLFormatter runs the CSSFormatter/JSFormatter
in a nested context (<style> / <script>). Add a few more APIs for
conveniences when appending tokens or important positions.

* UserInterface/Workers/Formatter/HTMLParser.js: Added.
(HTMLParser.prototype.parseDocument):
(HTMLParser.prototype._isEOF):
(HTMLParser.prototype._peek):
(HTMLParser.prototype._peekCharacterRegex):
(HTMLParser.prototype._peekString):
(HTMLParser.prototype._peekCaseInsensitiveString):
(HTMLParser.prototype._consumeWhitespace):
(HTMLParser.prototype._consumeUntilString):
(HTMLParser.prototype._consumeUntilRegex):
(HTMLParser.prototype._consumeDoubleQuotedString):
(HTMLParser):
Tokenize HTML content into nodes.

* UserInterface/Workers/Formatter/HTMLTreeBuilderFormatter.js: Added.
(HTMLTreeBuilderFormatter.prototype.get dom):
(HTMLTreeBuilderFormatter.prototype.begin):
(HTMLTreeBuilderFormatter.prototype.pushParserNode):
(HTMLTreeBuilderFormatter.prototype.end):
(HTMLTreeBuilderFormatter.prototype._pushParserNodeTopLevel):
(HTMLTreeBuilderFormatter.prototype._pushParserNodeStack):
(HTMLTreeBuilderFormatter.prototype._isEmptyNode):
(HTMLTreeBuilderFormatter.prototype._buildDOMNodeFromOpenTag):
(HTMLTreeBuilderFormatter.prototype._buildErrorNodeFromCloseTag):
(HTMLTreeBuilderFormatter.prototype._buildSimpleNodeFromParserNode):
(HTMLTreeBuilderFormatter):
Build a simple DOM tree from parser nodes.

* UserInterface/Workers/Formatter/HTMLFormatter.js: Added.
(HTMLFormatter.let.dom):
(HTMLFormatter):
(HTMLFormatter.prototype.get success):
(HTMLFormatter.prototype.get formattedText):
(HTMLFormatter.prototype.get sourceMapData):
(HTMLFormatter.prototype._walk):
(HTMLFormatter.prototype._walkChildren):
(HTMLFormatter.prototype._isNodeSelfClosing):
(HTMLFormatter.prototype._shouldHaveInlineContent):
(HTMLFormatter.prototype._hasMultipleNewLines):
(HTMLFormatter.prototype._buildAttributeString):
(HTMLFormatter.prototype._before):
(HTMLFormatter.prototype._after):
(HTMLFormatter.prototype._formatWithNestedFormatter):
(HTMLFormatter.prototype._formatScript):
(HTMLFormatter.prototype._formatStyle):
Output pretty printed text from a HTMLTreeBuilderFormatter dom tree.

* UserInterface/Proxies/FormatterWorkerProxy.js:
(WI.FormatterWorkerProxy.prototype.formatHTML):
* UserInterface/Workers/Formatter/FormatterWorker.js:
(FormatterWorker.prototype.formatJavaScript):
(FormatterWorker.prototype.formatCSS):
(FormatterWorker.prototype.formatHTML):
* UserInterface/Views/TextEditor.js:
(WI.TextEditor.prototype.hasFormatter):
(WI.TextEditor.prototype._canUseFormatterWorker):
(WI.TextEditor.prototype._startWorkerPrettyPrint):
Allow pretty printing HTML resources.

* UserInterface/Main.html:
* UserInterface/Test.html:
New resources.

LayoutTests:

* TestExpectations:
Mark slow test.

* inspector/formatting/formatting-css.html:
* inspector/formatting/formatting-html-expected.txt: Added.
* inspector/formatting/formatting-html.html: Added.
* inspector/formatting/formatting-javascript-expected.txt:
* inspector/formatting/formatting-javascript.html:
* inspector/formatting/formatting-json-expected.txt:
* inspector/formatting/formatting-json.html:
* inspector/formatting/resources/formatting-utilities.js: Renamed from LayoutTests/inspector/formatting/resources/utilities.js.
Update formatting tests to use new path to utilities.

* inspector/formatting/resources/css-tests/basic-expected.css:
* inspector/formatting/resources/css-tests/gradient-expected.css:
* inspector/formatting/resources/css-tests/keyframes-expected.css:
* inspector/formatting/resources/css-tests/media-query-expected.css:
* inspector/formatting/resources/css-tests/selectors-expected.css:
* inspector/formatting/resources/css-tests/wrapping-expected.css:
* inspector/codemirror/resources/prettyprinting/css-tests/basic-expected.css:
* inspector/codemirror/resources/prettyprinting/css-tests/calc-expected.css:
* inspector/codemirror/resources/prettyprinting/css-tests/gradient-expected.css:
* inspector/codemirror/resources/prettyprinting/css-tests/keyframes-expected.css:
* inspector/codemirror/resources/prettyprinting/css-tests/media-query-expected.css:
* inspector/codemirror/resources/prettyprinting/css-tests/selectors-expected.css:
* inspector/codemirror/resources/prettyprinting/css-tests/wrapping-expected.css:
Eliminate an extra trailing newline in CSS formatted results.

* inspector/formatting/resources/html-tests/attributes-expected.html: Added.
* inspector/formatting/resources/html-tests/attributes.html: Added.
* inspector/formatting/resources/html-tests/auto-close-expected.html: Added.
* inspector/formatting/resources/html-tests/auto-close.html: Added.
* inspector/formatting/resources/html-tests/basic-1-expected.html: Added.
* inspector/formatting/resources/html-tests/basic-1.html: Added.
* inspector/formatting/resources/html-tests/basic-2-expected.html: Added.
* inspector/formatting/resources/html-tests/basic-2.html: Added.
* inspector/formatting/resources/html-tests/comments-expected.html: Added.
* inspector/formatting/resources/html-tests/comments.html: Added.
* inspector/formatting/resources/html-tests/eof-1-expected.html: Added.
* inspector/formatting/resources/html-tests/eof-1.html: Added.
* inspector/formatting/resources/html-tests/eof-2-expected.html: Added.
* inspector/formatting/resources/html-tests/eof-2.html: Added.
* inspector/formatting/resources/html-tests/eof-3-expected.html: Added.
* inspector/formatting/resources/html-tests/eof-3.html: Added.
* inspector/formatting/resources/html-tests/eof-4-expected.html: Added.
* inspector/formatting/resources/html-tests/eof-4.html: Added.
* inspector/formatting/resources/html-tests/eof-5-expected.html: Added.
* inspector/formatting/resources/html-tests/eof-5.html: Added.
* inspector/formatting/resources/html-tests/eof-6-expected.html: Added.
* inspector/formatting/resources/html-tests/eof-6.html: Added.
* inspector/formatting/resources/html-tests/inline-script-expected.html: Added.
* inspector/formatting/resources/html-tests/inline-script.html: Added.
* inspector/formatting/resources/html-tests/inline-style-expected.html: Added.
* inspector/formatting/resources/html-tests/inline-style.html: Added.
* inspector/formatting/resources/html-tests/list-expected.html: Added.
* inspector/formatting/resources/html-tests/list.html: Added.
* inspector/formatting/resources/html-tests/not-well-formed-1-expected.html: Added.
* inspector/formatting/resources/html-tests/not-well-formed-1.html: Added.
* inspector/formatting/resources/html-tests/not-well-formed-2-expected.html: Added.
* inspector/formatting/resources/html-tests/not-well-formed-2.html: Added.
* inspector/formatting/resources/html-tests/not-well-formed-3-expected.html: Added.
* inspector/formatting/resources/html-tests/not-well-formed-3.html: Added.
* inspector/formatting/resources/html-tests/self-closing-expected.html: Added.
* inspector/formatting/resources/html-tests/self-closing.html: Added.
HTMLFormatter tests.

* inspector/formatting/resources/html-source-map-tests/1.html: Added.
* inspector/formatting/resources/html-source-map-tests/2.html: Added.
* inspector/formatting/resources/css-source-map-tests/1.css: Added.
* inspector/formatting/resources/javascript-source-map-tests/1.js: Added.
* inspector/formatting/source-map-css-1-expected.txt: Added.
* inspector/formatting/source-map-css-1.html: Added.
* inspector/formatting/source-map-html-1-expected.txt: Added.
* inspector/formatting/source-map-html-1.html: Added.
* inspector/formatting/source-map-html-2-expected.txt: Added.
* inspector/formatting/source-map-html-2.html: Added.
* inspector/formatting/source-map-javascript-1-expected.txt: Added.
* inspector/formatting/source-map-javascript-1.html: Added.
* inspector/formatting/resources/source-map-utilities.js: Added.
(TestPage.registerInitializer.logLocationWithContext):
(TestPage.registerInitializer.async.runSourceMapTest):
(TestPage.registerInitializer.):
(TestPage.registerInitializer.async.loadFormattedContentAndSourceMap):
(TestPage.registerInitializer.async.loadSourceMapTestResource):
(TestPage.registerInitializer.window.addSourceMapTest):
(TestPage.registerInitializer):
New formatter source map tests to ensure we can go back and forth reasonably
between formatted content and original content.

* inspector/debugger/breakpoints/resolved-dump-all-inline-script-pause-locations-expected.txt: Added.
* inspector/debugger/breakpoints/resolved-dump-all-inline-script-pause-locations.html: Added.
* inspector/debugger/breakpoints/resources/dump-inline-scripts.html: Added.
* inspector/debugger/breakpoints/resources/dump.js:
(TestPage.registerInitializer.window.addDumpAllPauseLocationsTestCase):
(TestPage.registerInitializer.addDumpAllPauseLocationsTestCaseForScript):
(TestPage.registerInitializer):
* inspector/debugger/resources/log-pause-location.js:
(TestPage.registerInitializer.window.findScript):
(TestPage.registerInitializer.window.findResource):
Test for resolving breakpoints at column locations in <script> which was
not previously possible and would not have worked previously.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@249831 268f45cc-cd09-0410-ab3c-d52691b4dbfc
102 files changed:
LayoutTests/ChangeLog
LayoutTests/TestExpectations
LayoutTests/inspector/debugger/breakpoints/resolved-dump-all-inline-script-pause-locations-expected.txt [new file with mode: 0644]
LayoutTests/inspector/debugger/breakpoints/resolved-dump-all-inline-script-pause-locations.html [new file with mode: 0644]
LayoutTests/inspector/debugger/breakpoints/resources/dump-inline-scripts.html [new file with mode: 0644]
LayoutTests/inspector/debugger/breakpoints/resources/dump.js
LayoutTests/inspector/debugger/resources/log-pause-location.js
LayoutTests/inspector/debugger/tail-deleted-frames/tail-deleted-frames-vm-entry.html
LayoutTests/inspector/formatting/formatting-css.html
LayoutTests/inspector/formatting/formatting-html-expected.txt [new file with mode: 0644]
LayoutTests/inspector/formatting/formatting-html.html [new file with mode: 0644]
LayoutTests/inspector/formatting/formatting-javascript-expected.txt
LayoutTests/inspector/formatting/formatting-javascript.html
LayoutTests/inspector/formatting/formatting-json-expected.txt
LayoutTests/inspector/formatting/formatting-json.html
LayoutTests/inspector/formatting/resources/css-source-map-tests/1.css [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/css-tests/basic-expected.css
LayoutTests/inspector/formatting/resources/css-tests/gradient-expected.css
LayoutTests/inspector/formatting/resources/css-tests/keyframes-expected.css
LayoutTests/inspector/formatting/resources/css-tests/media-query-expected.css
LayoutTests/inspector/formatting/resources/css-tests/selectors-expected.css
LayoutTests/inspector/formatting/resources/css-tests/wrapping-expected.css
LayoutTests/inspector/formatting/resources/formatting-utilities.js [moved from LayoutTests/inspector/formatting/resources/utilities.js with 95% similarity]
LayoutTests/inspector/formatting/resources/html-source-map-tests/1.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-source-map-tests/2.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/attributes-expected.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/attributes.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/auto-close-expected.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/auto-close.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/basic-1-expected.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/basic-1.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/basic-2-expected.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/basic-2.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/comments-expected.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/comments.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/eof-1-expected.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/eof-1.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/eof-2-expected.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/eof-2.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/eof-3-expected.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/eof-3.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/eof-4-expected.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/eof-4.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/eof-5-expected.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/eof-5.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/eof-6-expected.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/eof-6.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/inline-script-expected.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/inline-script.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/inline-style-expected.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/inline-style.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/list-expected.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/list.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/not-well-formed-1-expected.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/not-well-formed-1.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/not-well-formed-2-expected.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/not-well-formed-2.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/not-well-formed-3-expected.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/not-well-formed-3.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/self-closing-expected.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/html-tests/self-closing.html [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/javascript-source-map-tests/1.js [new file with mode: 0644]
LayoutTests/inspector/formatting/resources/source-map-utilities.js [new file with mode: 0644]
LayoutTests/inspector/formatting/source-map-css-1-expected.txt [new file with mode: 0644]
LayoutTests/inspector/formatting/source-map-css-1.html [new file with mode: 0644]
LayoutTests/inspector/formatting/source-map-html-1-expected.txt [new file with mode: 0644]
LayoutTests/inspector/formatting/source-map-html-1.html [new file with mode: 0644]
LayoutTests/inspector/formatting/source-map-html-2-expected.txt [new file with mode: 0644]
LayoutTests/inspector/formatting/source-map-html-2.html [new file with mode: 0644]
LayoutTests/inspector/formatting/source-map-javascript-1-expected.txt [new file with mode: 0644]
LayoutTests/inspector/formatting/source-map-javascript-1.html [new file with mode: 0644]
Source/JavaScriptCore/ChangeLog
Source/JavaScriptCore/debugger/Debugger.cpp
Source/WebInspectorUI/ChangeLog
Source/WebInspectorUI/Tools/HTMLFormatter/HTMLTreeBuilderDebug.js [new file with mode: 0644]
Source/WebInspectorUI/Tools/HTMLFormatter/index.html [new file with mode: 0644]
Source/WebInspectorUI/Tools/HTMLFormatter/styles.css [new file with mode: 0644]
Source/WebInspectorUI/Tools/JSFormatter/JSFormatterDebug.js [moved from Source/WebInspectorUI/Tools/Formatting/EsprimaFormatterDebug.js with 98% similarity]
Source/WebInspectorUI/Tools/JSFormatter/codemirror-additions.css [moved from Source/WebInspectorUI/Tools/Formatting/codemirror-additions.css with 100% similarity]
Source/WebInspectorUI/Tools/JSFormatter/index.html [moved from Source/WebInspectorUI/Tools/Formatting/index.html with 92% similarity]
Source/WebInspectorUI/Tools/SourceMaps/.gitignore [new file with mode: 0644]
Source/WebInspectorUI/Tools/SourceMaps/index.html [new file with mode: 0644]
Source/WebInspectorUI/Tools/SourceMaps/setup.sh [new file with mode: 0755]
Source/WebInspectorUI/Tools/SourceMaps/styles.css [new file with mode: 0644]
Source/WebInspectorUI/UserInterface/Controllers/FormatterSourceMap.js
Source/WebInspectorUI/UserInterface/Models/Resource.js
Source/WebInspectorUI/UserInterface/Proxies/FormatterWorkerProxy.js
Source/WebInspectorUI/UserInterface/Test.html
Source/WebInspectorUI/UserInterface/Views/DebuggerSidebarPanel.js
Source/WebInspectorUI/UserInterface/Views/LocalResourceOverridePopover.js
Source/WebInspectorUI/UserInterface/Views/LocalResourceOverrideTreeElement.js
Source/WebInspectorUI/UserInterface/Views/SettingsTabContentView.js
Source/WebInspectorUI/UserInterface/Views/SourcesNavigationSidebarPanel.js
Source/WebInspectorUI/UserInterface/Views/TextEditor.js
Source/WebInspectorUI/UserInterface/Views/TextResourceContentView.js
Source/WebInspectorUI/UserInterface/Workers/Formatter/CSSFormatter.js
Source/WebInspectorUI/UserInterface/Workers/Formatter/FormatterContentBuilder.js
Source/WebInspectorUI/UserInterface/Workers/Formatter/FormatterWorker.js
Source/WebInspectorUI/UserInterface/Workers/Formatter/HTMLFormatter.js [new file with mode: 0644]
Source/WebInspectorUI/UserInterface/Workers/Formatter/HTMLParser.js [new file with mode: 0644]
Source/WebInspectorUI/UserInterface/Workers/Formatter/HTMLTreeBuilderFormatter.js [new file with mode: 0644]
Source/WebInspectorUI/UserInterface/Workers/Formatter/JSFormatter.js [moved from Source/WebInspectorUI/UserInterface/Workers/Formatter/EsprimaFormatter.js with 98% similarity]