Update custom line breaking iterators to the latest version of Unicode
authormmaxfield@apple.com <mmaxfield@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 13 Feb 2017 19:07:43 +0000 (19:07 +0000)
committermmaxfield@apple.com <mmaxfield@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 13 Feb 2017 19:07:43 +0000 (19:07 +0000)
commit0125d4d306f76eab6cec54518e4846ed74d6b5f0
tree9ba6344673107e7e44159e8268727a072ba971c7
parent6ec1c0495259d02071932c55c187ba103febad10
Update custom line breaking iterators to the latest version of Unicode
https://bugs.webkit.org/show_bug.cgi?id=168182

Reviewed by Zalan Bujtas.

Source/WebCore:

Clean up our breaking code to be more descriptive about the difference between
line-break: auto vs line-break: loose | normal | strict. The only difference is
that we have some hardcoded tables to speed up character iteration for
line-break: auto.

Tests: TestWebKitAPI WebKit2.LineBreaking

* rendering/BreakLines.h:
(WebCore::nextBreakablePosition):
(WebCore::nextBreakablePositionIgnoringNBSP):
(WebCore::nextBreakablePositionWithoutShortcut):
(WebCore::nextBreakablePositionIgnoringNBSPWithoutShortcut):
(WebCore::isBreakable):
(WebCore::nextBreakablePositionNonLoosely): Deleted.
(WebCore::nextBreakablePositionLoosely): Deleted.
(WebCore::nextBreakablePositionLoose): Deleted.
(WebCore::nextBreakablePositionIgnoringNBSPLoose): Deleted.
* rendering/RenderText.cpp:
(WebCore::RenderText::computePreferredLogicalWidths):
* rendering/SimpleLineLayoutTextFragmentIterator.cpp:
(WebCore::SimpleLineLayout::nextBreakablePositionInSegment):
* rendering/line/BreakingContext.h:
(WebCore::BreakingContext::handleText):
(WebCore::BreakingContext::optimalLineBreakLocationForTrailingWord):

Source/WTF:

ICU 55.1 supports loose / normal / strict line breaking rules. The oldest platform we ship
on has a version of ICU >= that one. Therefore, we don't need to compile our own rules;
we can just use ICU's rules.

* wtf/text/LineBreakIteratorPoolICU.h:
(WTF::LineBreakIteratorPool::makeLocaleWithBreakKeyword):
(WTF::LineBreakIteratorPool::take):
* wtf/text/TextBreakIterator.cpp:
(WTF::acquireLineBreakIterator):
(WTF::openLineBreakIterator):
(WTF::mapLineIteratorModeToRules): Deleted.
(WTF::isCJKLocale): Deleted.
* wtf/text/TextBreakIterator.h:
(WTF::LazyLineBreakIterator::LazyLineBreakIterator):
(WTF::LazyLineBreakIterator::mode):
(WTF::LazyLineBreakIterator::get):
(WTF::LazyLineBreakIterator::resetStringAndReleaseIterator):
(WTF::LazyLineBreakIterator::isLooseCJKMode): Deleted.

Tools:

Treat the system's ICU as the source of truth to compare breaking positions against.

* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WebCore/LineBreaking.mm: Added.
(generateJavaScriptForTest):
(breakingLocationsFromICU):
(testAFewStrings):
(TEST):
* TestWebKitAPI/Tests/WebKit2Cocoa/AllAhem.svg: Renamed from LayoutTests/css3/line-break/resources/AllAhem.svg.
* TestWebKitAPI/Tests/WebKit2Cocoa/LineBreaking.html: Added.

LayoutTests:

Migrated to TestWebKitAPI.

* css3/line-break/line-break-auto-centered-2-expected.html: Removed.
* css3/line-break/line-break-auto-centered-2.html: Removed.
* css3/line-break/line-break-auto-centered-expected.html: Removed.
* css3/line-break/line-break-auto-centered.html: Removed.
* css3/line-break/line-break-auto-half-kana-2-expected.html: Removed.
* css3/line-break/line-break-auto-half-kana-2.html: Removed.
* css3/line-break/line-break-auto-half-kana-3-expected.html: Removed.
* css3/line-break/line-break-auto-half-kana-3.html: Removed.
* css3/line-break/line-break-auto-half-kana-4-expected.html: Removed.
* css3/line-break/line-break-auto-half-kana-4.html: Removed.
* css3/line-break/line-break-auto-half-kana-5-expected.html: Removed.
* css3/line-break/line-break-auto-half-kana-5.html: Removed.
* css3/line-break/line-break-auto-half-kana-expected.html: Removed.
* css3/line-break/line-break-auto-half-kana.html: Removed.
* css3/line-break/line-break-auto-hyphens-expected.html: Removed.
* css3/line-break/line-break-auto-hyphens.html: Removed.
* css3/line-break/line-break-auto-inseparables-expected.html: Removed.
* css3/line-break/line-break-auto-inseparables.html: Removed.
* css3/line-break/line-break-auto-iteration-marks-expected.html: Removed.
* css3/line-break/line-break-auto-iteration-marks.html: Removed.
* css3/line-break/line-break-auto-postfixes-expected.html: Removed.
* css3/line-break/line-break-auto-postfixes.html: Removed.
* css3/line-break/line-break-auto-prefixes-expected.html: Removed.
* css3/line-break/line-break-auto-prefixes.html: Removed.
* css3/line-break/line-break-auto-sound-marks-expected.html: Removed.
* css3/line-break/line-break-auto-sound-marks.html: Removed.
* css3/line-break/line-break-loose-centered-2-expected.html: Removed.
* css3/line-break/line-break-loose-centered-2.html: Removed.
* css3/line-break/line-break-loose-centered-expected.html: Removed.
* css3/line-break/line-break-loose-centered.html: Removed.
* css3/line-break/line-break-loose-half-kana-2-expected.html: Removed.
* css3/line-break/line-break-loose-half-kana-2.html: Removed.
* css3/line-break/line-break-loose-half-kana-3-expected.html: Removed.
* css3/line-break/line-break-loose-half-kana-3.html: Removed.
* css3/line-break/line-break-loose-half-kana-4-expected.html: Removed.
* css3/line-break/line-break-loose-half-kana-4.html: Removed.
* css3/line-break/line-break-loose-half-kana-5-expected.html: Removed.
* css3/line-break/line-break-loose-half-kana-5.html: Removed.
* css3/line-break/line-break-loose-half-kana-expected.html: Removed.
* css3/line-break/line-break-loose-half-kana.html: Removed.
* css3/line-break/line-break-loose-hyphens-expected.html: Removed.
* css3/line-break/line-break-loose-hyphens.html: Removed.
* css3/line-break/line-break-loose-inseparables-expected.html: Removed.
* css3/line-break/line-break-loose-inseparables.html: Removed.
* css3/line-break/line-break-loose-iteration-marks-expected.html: Removed.
* css3/line-break/line-break-loose-iteration-marks.html: Removed.
* css3/line-break/line-break-loose-postfixes-expected.html: Removed.
* css3/line-break/line-break-loose-postfixes.html: Removed.
* css3/line-break/line-break-loose-prefixes-expected.html: Removed.
* css3/line-break/line-break-loose-prefixes.html: Removed.
* css3/line-break/line-break-loose-sound-marks-expected.html: Removed.
* css3/line-break/line-break-loose-sound-marks.html: Removed.
* css3/line-break/line-break-normal-centered-2-expected.html: Removed.
* css3/line-break/line-break-normal-centered-2.html: Removed.
* css3/line-break/line-break-normal-centered-expected.html: Removed.
* css3/line-break/line-break-normal-centered.html: Removed.
* css3/line-break/line-break-normal-half-kana-2-expected.html: Removed.
* css3/line-break/line-break-normal-half-kana-2.html: Removed.
* css3/line-break/line-break-normal-half-kana-3-expected.html: Removed.
* css3/line-break/line-break-normal-half-kana-3.html: Removed.
* css3/line-break/line-break-normal-half-kana-4-expected.html: Removed.
* css3/line-break/line-break-normal-half-kana-4.html: Removed.
* css3/line-break/line-break-normal-half-kana-5-expected.html: Removed.
* css3/line-break/line-break-normal-half-kana-5.html: Removed.
* css3/line-break/line-break-normal-half-kana-expected.html: Removed.
* css3/line-break/line-break-normal-half-kana.html: Removed.
* css3/line-break/line-break-normal-hyphens-expected.html: Removed.
* css3/line-break/line-break-normal-hyphens.html: Removed.
* css3/line-break/line-break-normal-inseparables-expected.html: Removed.
* css3/line-break/line-break-normal-inseparables.html: Removed.
* css3/line-break/line-break-normal-iteration-marks-expected.html: Removed.
* css3/line-break/line-break-normal-iteration-marks.html: Removed.
* css3/line-break/line-break-normal-postfixes-expected.html: Removed.
* css3/line-break/line-break-normal-postfixes.html: Removed.
* css3/line-break/line-break-normal-prefixes-expected.html: Removed.
* css3/line-break/line-break-normal-prefixes.html: Removed.
* css3/line-break/line-break-normal-sound-marks-expected.html: Removed.
* css3/line-break/line-break-normal-sound-marks.html: Removed.
* css3/line-break/line-break-strict-centered-2-expected.html: Removed.
* css3/line-break/line-break-strict-centered-2.html: Removed.
* css3/line-break/line-break-strict-centered-expected.html: Removed.
* css3/line-break/line-break-strict-centered.html: Removed.
* css3/line-break/line-break-strict-half-kana-2-expected.html: Removed.
* css3/line-break/line-break-strict-half-kana-2.html: Removed.
* css3/line-break/line-break-strict-half-kana-3-expected.html: Removed.
* css3/line-break/line-break-strict-half-kana-3.html: Removed.
* css3/line-break/line-break-strict-half-kana-4-expected.html: Removed.
* css3/line-break/line-break-strict-half-kana-4.html: Removed.
* css3/line-break/line-break-strict-half-kana-5-expected.html: Removed.
* css3/line-break/line-break-strict-half-kana-5.html: Removed.
* css3/line-break/line-break-strict-half-kana-expected.html: Removed.
* css3/line-break/line-break-strict-half-kana.html: Removed.
* css3/line-break/line-break-strict-hyphens-expected.html: Removed.
* css3/line-break/line-break-strict-hyphens.html: Removed.
* css3/line-break/line-break-strict-inseparables-expected.html: Removed.
* css3/line-break/line-break-strict-inseparables.html: Removed.
* css3/line-break/line-break-strict-iteration-marks-expected.html: Removed.
* css3/line-break/line-break-strict-iteration-marks.html: Removed.
* css3/line-break/line-break-strict-postfixes-expected.html: Removed.
* css3/line-break/line-break-strict-postfixes.html: Removed.
* css3/line-break/line-break-strict-prefixes-expected.html: Removed.
* css3/line-break/line-break-strict-prefixes.html: Removed.
* css3/line-break/line-break-strict-sound-marks-expected.html: Removed.
* css3/line-break/line-break-strict-sound-marks.html: Removed.
* platform/ios-simulator-wk1/TestExpectations:
* platform/ios-simulator/TestExpectations:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@212235 268f45cc-cd09-0410-ab3c-d52691b4dbfc
121 files changed:
LayoutTests/ChangeLog
LayoutTests/css3/line-break/line-break-auto-centered-2-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-auto-centered-2.html [deleted file]
LayoutTests/css3/line-break/line-break-auto-centered-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-auto-centered.html [deleted file]
LayoutTests/css3/line-break/line-break-auto-half-kana-2-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-auto-half-kana-2.html [deleted file]
LayoutTests/css3/line-break/line-break-auto-half-kana-3-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-auto-half-kana-3.html [deleted file]
LayoutTests/css3/line-break/line-break-auto-half-kana-4-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-auto-half-kana-4.html [deleted file]
LayoutTests/css3/line-break/line-break-auto-half-kana-5-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-auto-half-kana-5.html [deleted file]
LayoutTests/css3/line-break/line-break-auto-half-kana-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-auto-half-kana.html [deleted file]
LayoutTests/css3/line-break/line-break-auto-hyphens-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-auto-hyphens.html [deleted file]
LayoutTests/css3/line-break/line-break-auto-inseparables-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-auto-inseparables.html [deleted file]
LayoutTests/css3/line-break/line-break-auto-iteration-marks-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-auto-iteration-marks.html [deleted file]
LayoutTests/css3/line-break/line-break-auto-postfixes-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-auto-postfixes.html [deleted file]
LayoutTests/css3/line-break/line-break-auto-prefixes-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-auto-prefixes.html [deleted file]
LayoutTests/css3/line-break/line-break-auto-sound-marks-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-auto-sound-marks.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-centered-2-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-centered-2.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-centered-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-centered.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-half-kana-2-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-half-kana-2.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-half-kana-3-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-half-kana-3.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-half-kana-4-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-half-kana-4.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-half-kana-5-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-half-kana-5.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-half-kana-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-half-kana.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-hyphens-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-hyphens.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-inseparables-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-inseparables.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-iteration-marks-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-iteration-marks.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-postfixes-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-postfixes.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-prefixes-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-prefixes.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-sound-marks-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-loose-sound-marks.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-centered-2-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-centered-2.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-centered-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-centered.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-half-kana-2-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-half-kana-2.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-half-kana-3-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-half-kana-3.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-half-kana-4-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-half-kana-4.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-half-kana-5-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-half-kana-5.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-half-kana-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-half-kana.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-hyphens-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-hyphens.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-inseparables-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-inseparables.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-iteration-marks-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-iteration-marks.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-postfixes-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-postfixes.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-prefixes-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-prefixes.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-sound-marks-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-normal-sound-marks.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-centered-2-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-centered-2.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-centered-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-centered.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-half-kana-2-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-half-kana-2.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-half-kana-3-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-half-kana-3.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-half-kana-4-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-half-kana-4.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-half-kana-5-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-half-kana-5.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-half-kana-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-half-kana.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-hyphens-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-hyphens.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-inseparables-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-inseparables.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-iteration-marks-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-iteration-marks.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-postfixes-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-postfixes.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-prefixes-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-prefixes.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-sound-marks-expected.html [deleted file]
LayoutTests/css3/line-break/line-break-strict-sound-marks.html [deleted file]
LayoutTests/platform/ios-simulator-wk1/TestExpectations
LayoutTests/platform/ios-simulator/TestExpectations
Source/WTF/ChangeLog
Source/WTF/wtf/text/LineBreakIteratorPoolICU.h
Source/WTF/wtf/text/TextBreakIterator.cpp
Source/WTF/wtf/text/TextBreakIterator.h
Source/WebCore/ChangeLog
Source/WebCore/rendering/BreakLines.h
Source/WebCore/rendering/RenderText.cpp
Source/WebCore/rendering/SimpleLineLayoutTextFragmentIterator.cpp
Source/WebCore/rendering/line/BreakingContext.h
Tools/ChangeLog
Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj
Tools/TestWebKitAPI/Tests/WebCore/LineBreaking.mm [new file with mode: 0644]
Tools/TestWebKitAPI/Tests/WebKit2Cocoa/AllAhem.svg [moved from LayoutTests/css3/line-break/resources/AllAhem.svg with 100% similarity]
Tools/TestWebKitAPI/Tests/WebKit2Cocoa/LineBreaking.html [new file with mode: 0644]