2010-08-24 Dirk Pranke <dpranke@chromium.org>
authordpranke@chromium.org <dpranke@chromium.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 25 Aug 2010 00:02:27 +0000 (00:02 +0000)
committerdpranke@chromium.org <dpranke@chromium.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 25 Aug 2010 00:02:27 +0000 (00:02 +0000)
        Reviewed by Eric Seidel.

        new-run-webkit-tests: revise unit testing strategy for test failures

        This change revamps the way we store test data for new-run-webkit-tests.
        Previously we had a few copies of test files from the main test tree.
        Now we have a bunch of completely fake test data and use the test
        data to generate different kinds of test failures, getting
        much better coverage.

        https://bugs.webkit.org/show_bug.cgi?id=44556

        * Scripts/webkitpy/layout_tests/data/failures/expected/crash.html: Added.
        * Scripts/webkitpy/layout_tests/data/failures/expected/image-expected.checksum: Added.
        * Scripts/webkitpy/layout_tests/data/failures/expected/image-expected.png: Added.
        * Scripts/webkitpy/layout_tests/data/failures/expected/image-expected.txt: Added.
        * Scripts/webkitpy/layout_tests/data/failures/expected/image.html: Added.
        * Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum-expected.checksum: Added.
        * Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum-expected.png: Added.
        * Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum-expected.txt: Added.
        * Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum.html: Added.
        * Scripts/webkitpy/layout_tests/data/failures/expected/missing_image.html: Added.
        * Scripts/webkitpy/layout_tests/data/failures/expected/missing_text.html: Added.
        * Scripts/webkitpy/layout_tests/data/failures/expected/text-expected.txt: Added.
        * Scripts/webkitpy/layout_tests/data/failures/expected/text.html: Added.
        * Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum-expected.checksum: Added.
        * Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum-expected.png: Added.
        * Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum-expected.txt: Added.
        * Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum.html: Added.
        * Scripts/webkitpy/layout_tests/data/image/canvas-bg.html: Removed.
        * Scripts/webkitpy/layout_tests/data/image/canvas-zoom-expected.checksum: Removed.
        * Scripts/webkitpy/layout_tests/data/image/canvas-zoom-expected.png: Removed.
        * Scripts/webkitpy/layout_tests/data/image/canvas-zoom-expected.txt: Removed.
        * Scripts/webkitpy/layout_tests/data/image/canvas-zoom.html: Removed.
        * Scripts/webkitpy/layout_tests/data/misc/crash-expected.txt: Removed.
        * Scripts/webkitpy/layout_tests/data/misc/crash.html: Removed.
        * Scripts/webkitpy/layout_tests/data/misc/missing-expectation.html: Removed.
        * Scripts/webkitpy/layout_tests/data/misc/passing-expected.txt: Removed.
        * Scripts/webkitpy/layout_tests/data/misc/passing.html: Removed.
        * Scripts/webkitpy/layout_tests/data/passes/image-expected.checksum: Added.
        * Scripts/webkitpy/layout_tests/data/passes/image-expected.png: Added.
        * Scripts/webkitpy/layout_tests/data/passes/image-expected.txt: Added.
        * Scripts/webkitpy/layout_tests/data/passes/image.html: Added.
        * Scripts/webkitpy/layout_tests/data/passes/platform_image-expected.checksum: Added.
        * Scripts/webkitpy/layout_tests/data/passes/platform_image-expected.png: Added.
        * Scripts/webkitpy/layout_tests/data/passes/platform_image-expected.txt: Added.
        * Scripts/webkitpy/layout_tests/data/passes/platform_image.html: Added.
        * Scripts/webkitpy/layout_tests/data/passes/text-expected.txt: Added.
        * Scripts/webkitpy/layout_tests/data/passes/text.html: Added.
        * Scripts/webkitpy/layout_tests/data/platform/test/image/canvas-bg-expected.checksum: Removed.
        * Scripts/webkitpy/layout_tests/data/platform/test/image/canvas-bg-expected.png: Removed.
        * Scripts/webkitpy/layout_tests/data/platform/test/image/canvas-bg-expected.txt: Removed.
        * Scripts/webkitpy/layout_tests/data/platform/test/passes/platform_image-expected.checksum: Added.
        * Scripts/webkitpy/layout_tests/data/platform/test/passes/platform_image-expected.png: Added.
        * Scripts/webkitpy/layout_tests/data/platform/test/passes/platform_image-expected.txt: Added.
        * Scripts/webkitpy/layout_tests/data/platform/test/test_expectations.txt:
        * Scripts/webkitpy/layout_tests/data/text/article-element-expected.txt: Removed.
        * Scripts/webkitpy/layout_tests/data/text/article-element.html: Removed.
        * Scripts/webkitpy/layout_tests/layout_package/test_expectations_unittest.py:
        * Scripts/webkitpy/layout_tests/port/test.py:
        * Scripts/webkitpy/layout_tests/run_webkit_tests_unittest.py:
        * Scripts/webkitpy/style/checkers/test_expectations_unittest.py:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@65951 268f45cc-cd09-0410-ab3c-d52691b4dbfc

51 files changed:
WebKitTools/ChangeLog
WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/crash.html [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/image-expected.checksum [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/image-expected.png [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/image-expected.txt [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/image.html [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum-expected.checksum [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum-expected.png [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum-expected.txt [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum.html [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/missing_image.html [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/missing_text.html [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/text-expected.txt [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/text.html [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum-expected.checksum [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum-expected.png [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum-expected.txt [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum.html [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/image/canvas-bg.html [deleted file]
WebKitTools/Scripts/webkitpy/layout_tests/data/image/canvas-zoom-expected.checksum [deleted file]
WebKitTools/Scripts/webkitpy/layout_tests/data/image/canvas-zoom-expected.png [deleted file]
WebKitTools/Scripts/webkitpy/layout_tests/data/image/canvas-zoom-expected.txt [deleted file]
WebKitTools/Scripts/webkitpy/layout_tests/data/image/canvas-zoom.html [deleted file]
WebKitTools/Scripts/webkitpy/layout_tests/data/misc/crash-expected.txt [deleted file]
WebKitTools/Scripts/webkitpy/layout_tests/data/misc/crash.html [deleted file]
WebKitTools/Scripts/webkitpy/layout_tests/data/misc/missing-expectation.html [deleted file]
WebKitTools/Scripts/webkitpy/layout_tests/data/misc/passing-expected.txt [deleted file]
WebKitTools/Scripts/webkitpy/layout_tests/data/misc/passing.html [deleted file]
WebKitTools/Scripts/webkitpy/layout_tests/data/passes/image-expected.checksum [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/passes/image-expected.png [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/passes/image-expected.txt [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/passes/image.html [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/passes/platform_image-expected.checksum [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/passes/platform_image-expected.png [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/passes/platform_image-expected.txt [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/passes/platform_image.html [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/passes/text-expected.txt [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/passes/text.html [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/platform/test/image/canvas-bg-expected.checksum [deleted file]
WebKitTools/Scripts/webkitpy/layout_tests/data/platform/test/image/canvas-bg-expected.png [deleted file]
WebKitTools/Scripts/webkitpy/layout_tests/data/platform/test/image/canvas-bg-expected.txt [deleted file]
WebKitTools/Scripts/webkitpy/layout_tests/data/platform/test/passes/platform_image-expected.checksum [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/platform/test/passes/platform_image-expected.png [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/platform/test/passes/platform_image-expected.txt [new file with mode: 0644]
WebKitTools/Scripts/webkitpy/layout_tests/data/platform/test/test_expectations.txt
WebKitTools/Scripts/webkitpy/layout_tests/data/text/article-element-expected.txt [deleted file]
WebKitTools/Scripts/webkitpy/layout_tests/data/text/article-element.html [deleted file]
WebKitTools/Scripts/webkitpy/layout_tests/layout_package/test_expectations_unittest.py
WebKitTools/Scripts/webkitpy/layout_tests/port/test.py
WebKitTools/Scripts/webkitpy/layout_tests/run_webkit_tests_unittest.py
WebKitTools/Scripts/webkitpy/style/checkers/test_expectations_unittest.py

index 84671f2..ccdff3e 100644 (file)
@@ -2,6 +2,71 @@
 
         Reviewed by Eric Seidel.
 
+        new-run-webkit-tests: revise unit testing strategy for test failures
+    
+        This change revamps the way we store test data for new-run-webkit-tests.
+        Previously we had a few copies of test files from the main test tree.
+        Now we have a bunch of completely fake test data and use the test
+        data to generate different kinds of test failures, getting
+        much better coverage.
+
+        https://bugs.webkit.org/show_bug.cgi?id=44556
+
+        * Scripts/webkitpy/layout_tests/data/failures/expected/crash.html: Added.
+        * Scripts/webkitpy/layout_tests/data/failures/expected/image-expected.checksum: Added.
+        * Scripts/webkitpy/layout_tests/data/failures/expected/image-expected.png: Added.
+        * Scripts/webkitpy/layout_tests/data/failures/expected/image-expected.txt: Added.
+        * Scripts/webkitpy/layout_tests/data/failures/expected/image.html: Added.
+        * Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum-expected.checksum: Added.
+        * Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum-expected.png: Added.
+        * Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum-expected.txt: Added.
+        * Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum.html: Added.
+        * Scripts/webkitpy/layout_tests/data/failures/expected/missing_image.html: Added.
+        * Scripts/webkitpy/layout_tests/data/failures/expected/missing_text.html: Added.
+        * Scripts/webkitpy/layout_tests/data/failures/expected/text-expected.txt: Added.
+        * Scripts/webkitpy/layout_tests/data/failures/expected/text.html: Added.
+        * Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum-expected.checksum: Added.
+        * Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum-expected.png: Added.
+        * Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum-expected.txt: Added.
+        * Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum.html: Added.
+        * Scripts/webkitpy/layout_tests/data/image/canvas-bg.html: Removed.
+        * Scripts/webkitpy/layout_tests/data/image/canvas-zoom-expected.checksum: Removed.
+        * Scripts/webkitpy/layout_tests/data/image/canvas-zoom-expected.png: Removed.
+        * Scripts/webkitpy/layout_tests/data/image/canvas-zoom-expected.txt: Removed.
+        * Scripts/webkitpy/layout_tests/data/image/canvas-zoom.html: Removed.
+        * Scripts/webkitpy/layout_tests/data/misc/crash-expected.txt: Removed.
+        * Scripts/webkitpy/layout_tests/data/misc/crash.html: Removed.
+        * Scripts/webkitpy/layout_tests/data/misc/missing-expectation.html: Removed.
+        * Scripts/webkitpy/layout_tests/data/misc/passing-expected.txt: Removed.
+        * Scripts/webkitpy/layout_tests/data/misc/passing.html: Removed.
+        * Scripts/webkitpy/layout_tests/data/passes/image-expected.checksum: Added.
+        * Scripts/webkitpy/layout_tests/data/passes/image-expected.png: Added.
+        * Scripts/webkitpy/layout_tests/data/passes/image-expected.txt: Added.
+        * Scripts/webkitpy/layout_tests/data/passes/image.html: Added.
+        * Scripts/webkitpy/layout_tests/data/passes/platform_image-expected.checksum: Added.
+        * Scripts/webkitpy/layout_tests/data/passes/platform_image-expected.png: Added.
+        * Scripts/webkitpy/layout_tests/data/passes/platform_image-expected.txt: Added.
+        * Scripts/webkitpy/layout_tests/data/passes/platform_image.html: Added.
+        * Scripts/webkitpy/layout_tests/data/passes/text-expected.txt: Added.
+        * Scripts/webkitpy/layout_tests/data/passes/text.html: Added.
+        * Scripts/webkitpy/layout_tests/data/platform/test/image/canvas-bg-expected.checksum: Removed.
+        * Scripts/webkitpy/layout_tests/data/platform/test/image/canvas-bg-expected.png: Removed.
+        * Scripts/webkitpy/layout_tests/data/platform/test/image/canvas-bg-expected.txt: Removed.
+        * Scripts/webkitpy/layout_tests/data/platform/test/passes/platform_image-expected.checksum: Added.
+        * Scripts/webkitpy/layout_tests/data/platform/test/passes/platform_image-expected.png: Added.
+        * Scripts/webkitpy/layout_tests/data/platform/test/passes/platform_image-expected.txt: Added.
+        * Scripts/webkitpy/layout_tests/data/platform/test/test_expectations.txt:
+        * Scripts/webkitpy/layout_tests/data/text/article-element-expected.txt: Removed.
+        * Scripts/webkitpy/layout_tests/data/text/article-element.html: Removed.
+        * Scripts/webkitpy/layout_tests/layout_package/test_expectations_unittest.py:
+        * Scripts/webkitpy/layout_tests/port/test.py:
+        * Scripts/webkitpy/layout_tests/run_webkit_tests_unittest.py:
+        * Scripts/webkitpy/style/checkers/test_expectations_unittest.py:
+
+2010-08-24  Dirk Pranke  <dpranke@chromium.org>
+
+        Reviewed by Eric Seidel.
+
         speed up new-run-webkit-tests unit tests
 
         Add a --no-record-results flag that turns off generating the JSON
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/crash.html b/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/crash.html
new file mode 100644 (file)
index 0000000..0bc3798
--- /dev/null
@@ -0,0 +1 @@
+crash
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/image-expected.checksum b/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/image-expected.checksum
new file mode 100644 (file)
index 0000000..24b887a
--- /dev/null
@@ -0,0 +1 @@
+image-checksum
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/image-expected.png b/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/image-expected.png
new file mode 100644 (file)
index 0000000..4c23996
--- /dev/null
@@ -0,0 +1 @@
+image-png
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/image-expected.txt b/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/image-expected.txt
new file mode 100644 (file)
index 0000000..c6ee718
--- /dev/null
@@ -0,0 +1 @@
+image-txt
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/image.html b/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/image.html
new file mode 100644 (file)
index 0000000..53e4b27
--- /dev/null
@@ -0,0 +1 @@
+image_failure
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum-expected.checksum b/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum-expected.checksum
new file mode 100644 (file)
index 0000000..8fa0851
--- /dev/null
@@ -0,0 +1 @@
+image_checksum-checksum
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum-expected.png b/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum-expected.png
new file mode 100644 (file)
index 0000000..d677d2e
--- /dev/null
@@ -0,0 +1 @@
+image_checksum-png
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum-expected.txt b/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum-expected.txt
new file mode 100644 (file)
index 0000000..453f213
--- /dev/null
@@ -0,0 +1 @@
+image_checksum-txt
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum.html b/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum.html
new file mode 100644 (file)
index 0000000..2b78d31
--- /dev/null
@@ -0,0 +1 @@
+image_checksum
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/missing_image.html b/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/missing_image.html
new file mode 100644 (file)
index 0000000..0af8000
--- /dev/null
@@ -0,0 +1 @@
+missing_image
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/missing_text.html b/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/missing_text.html
new file mode 100644 (file)
index 0000000..47b8ad6
--- /dev/null
@@ -0,0 +1 @@
+missing_text
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/text-expected.txt b/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/text-expected.txt
new file mode 100644 (file)
index 0000000..e21ea45
--- /dev/null
@@ -0,0 +1 @@
+text_failures-txt
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/text.html b/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/expected/text.html
new file mode 100644 (file)
index 0000000..91f5fc7
--- /dev/null
@@ -0,0 +1 @@
+text_failure
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum-expected.checksum b/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum-expected.checksum
new file mode 100644 (file)
index 0000000..0c4f6da
--- /dev/null
@@ -0,0 +1 @@
+fail_checksum
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum-expected.png b/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum-expected.png
new file mode 100644 (file)
index 0000000..db483ee
--- /dev/null
@@ -0,0 +1 @@
+fail_png
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum-expected.txt b/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum-expected.txt
new file mode 100644 (file)
index 0000000..a1f3c24
--- /dev/null
@@ -0,0 +1 @@
+fail_output
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum.html b/WebKitTools/Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum.html
new file mode 100644 (file)
index 0000000..b325924
--- /dev/null
@@ -0,0 +1 @@
+Google
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/image/canvas-bg.html b/WebKitTools/Scripts/webkitpy/layout_tests/data/image/canvas-bg.html
deleted file mode 100644 (file)
index 2022676..0000000
+++ /dev/null
@@ -1,22 +0,0 @@
-<html>
- <head>
- <style>
- div { background: -webkit-canvas(squares); width:600px; height:600px; border:2px solid black }
- </style>
-  <script type="application/x-javascript">
-function draw(w, h) {
- var ctx = document.getCSSCanvasContext("2d", "squares", w, h);
-
- ctx.fillStyle = "rgb(200,0,0)";
- ctx.fillRect (10, 10, 55, 50);
-
- ctx.fillStyle = "rgba(0, 0, 200, 0.5)";
- ctx.fillRect (30, 30, 55, 50);
-}
-  </script>
- </head>
- <body onload="draw(300, 300)">
-   <div></div>
- </body>
-</html>
\ No newline at end of file
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/image/canvas-zoom-expected.checksum b/WebKitTools/Scripts/webkitpy/layout_tests/data/image/canvas-zoom-expected.checksum
deleted file mode 100644 (file)
index 7373fe2..0000000
+++ /dev/null
@@ -1 +0,0 @@
-afa0f2d246120c180005d67d47636b92
\ No newline at end of file
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/image/canvas-zoom-expected.png b/WebKitTools/Scripts/webkitpy/layout_tests/data/image/canvas-zoom-expected.png
deleted file mode 100644 (file)
index 44952b4..0000000
Binary files a/WebKitTools/Scripts/webkitpy/layout_tests/data/image/canvas-zoom-expected.png and /dev/null differ
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/image/canvas-zoom-expected.txt b/WebKitTools/Scripts/webkitpy/layout_tests/data/image/canvas-zoom-expected.txt
deleted file mode 100644 (file)
index 288458d..0000000
+++ /dev/null
@@ -1,22 +0,0 @@
-layer at (0,0) size 800x600
-  RenderView at (0,0) size 800x600
-layer at (0,0) size 800x600
-  RenderBlock {HTML} at (0,0) size 800x600
-    RenderBody {BODY} at (8,8) size 784x584
-      RenderBlock {P} at (0,0) size 784x18
-        RenderText {#text} at (0,0) size 624x18
-          text run at (0,0) width 624: "These should be four green hollow boxes with dimensions 600x300, 100x300, 600x100, 100x100."
-      RenderBlock (anonymous) at (0,34) size 784x420
-        RenderHTMLCanvas {CANVAS} at (0,0) size 606x306 [border: (3px solid #008000)]
-        RenderText {#text} at (606,292) size 4x18
-          text run at (606,292) width 4: " "
-        RenderText {#text} at (0,0) size 0x0
-        RenderHTMLCanvas {CANVAS} at (610,0) size 106x306 [border: (3px solid #008000)]
-        RenderText {#text} at (0,0) size 0x0
-        RenderText {#text} at (0,0) size 0x0
-        RenderHTMLCanvas {CANVAS} at (0,310) size 606x106 [border: (3px solid #008000)]
-        RenderText {#text} at (606,402) size 4x18
-          text run at (606,402) width 4: " "
-        RenderText {#text} at (0,0) size 0x0
-        RenderHTMLCanvas {CANVAS} at (610,310) size 106x106 [border: (3px solid #008000)]
-        RenderText {#text} at (0,0) size 0x0
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/image/canvas-zoom.html b/WebKitTools/Scripts/webkitpy/layout_tests/data/image/canvas-zoom.html
deleted file mode 100644 (file)
index 4dabce1..0000000
+++ /dev/null
@@ -1,15 +0,0 @@
-<style>
-    canvas { border: solid green;
-             zoom: 2; }
-</style>
-<p>
-    These should be four green hollow boxes with dimensions 600x300, 100x300, 600x100, 100x100.
-</p>
-<!-- 300x150 -->
-<canvas id="canvas"></canvas>
-<!-- 50x150 -->
-<canvas id="canvas" width="50"></canvas>
-<!-- 300x50 -->
-<canvas id="canvas" height="50"></canvas>
-<!-- 50x50 -->
-<canvas id="canvas" width="50" height="50"></canvas>
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/misc/crash-expected.txt b/WebKitTools/Scripts/webkitpy/layout_tests/data/misc/crash-expected.txt
deleted file mode 100644 (file)
index 521c3f5..0000000
+++ /dev/null
@@ -1 +0,0 @@
-This test is expected to crash.
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/misc/crash.html b/WebKitTools/Scripts/webkitpy/layout_tests/data/misc/crash.html
deleted file mode 100644 (file)
index b9820d6..0000000
+++ /dev/null
@@ -1,5 +0,0 @@
-<html>
-<body>
-This test is expected to crash.
-</body>
-</html> 
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/misc/missing-expectation.html b/WebKitTools/Scripts/webkitpy/layout_tests/data/misc/missing-expectation.html
deleted file mode 100644 (file)
index 1f00b50..0000000
+++ /dev/null
@@ -1,5 +0,0 @@
-<html>
-<body>
-This test intentionally doesn't have an expected result checked in.
-</body>
-</html> 
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/misc/passing-expected.txt b/WebKitTools/Scripts/webkitpy/layout_tests/data/misc/passing-expected.txt
deleted file mode 100644 (file)
index 26bd316..0000000
+++ /dev/null
@@ -1 +0,0 @@
-This test is expected to pass.
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/misc/passing.html b/WebKitTools/Scripts/webkitpy/layout_tests/data/misc/passing.html
deleted file mode 100644 (file)
index db7e3de..0000000
+++ /dev/null
@@ -1,5 +0,0 @@
-<html>
-<body>
-This test is expected to pass.
-</body>
-</html>
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/passes/image-expected.checksum b/WebKitTools/Scripts/webkitpy/layout_tests/data/passes/image-expected.checksum
new file mode 100644 (file)
index 0000000..24b887a
--- /dev/null
@@ -0,0 +1 @@
+image-checksum
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/passes/image-expected.png b/WebKitTools/Scripts/webkitpy/layout_tests/data/passes/image-expected.png
new file mode 100644 (file)
index 0000000..4c23996
--- /dev/null
@@ -0,0 +1 @@
+image-png
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/passes/image-expected.txt b/WebKitTools/Scripts/webkitpy/layout_tests/data/passes/image-expected.txt
new file mode 100644 (file)
index 0000000..c6ee718
--- /dev/null
@@ -0,0 +1 @@
+image-txt
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/passes/image.html b/WebKitTools/Scripts/webkitpy/layout_tests/data/passes/image.html
new file mode 100644 (file)
index 0000000..773b222
--- /dev/null
@@ -0,0 +1 @@
+image
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/passes/platform_image-expected.checksum b/WebKitTools/Scripts/webkitpy/layout_tests/data/passes/platform_image-expected.checksum
new file mode 100644 (file)
index 0000000..52038ae
--- /dev/null
@@ -0,0 +1 @@
+platform_image-generic-checksum
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/passes/platform_image-expected.png b/WebKitTools/Scripts/webkitpy/layout_tests/data/passes/platform_image-expected.png
new file mode 100644 (file)
index 0000000..087872b
--- /dev/null
@@ -0,0 +1 @@
+platform_image-generic-png
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/passes/platform_image-expected.txt b/WebKitTools/Scripts/webkitpy/layout_tests/data/passes/platform_image-expected.txt
new file mode 100644 (file)
index 0000000..f71680c
--- /dev/null
@@ -0,0 +1 @@
+platform_image-generic-txt
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/passes/platform_image.html b/WebKitTools/Scripts/webkitpy/layout_tests/data/passes/platform_image.html
new file mode 100644 (file)
index 0000000..ca48a7b
--- /dev/null
@@ -0,0 +1 @@
+platform_image
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/passes/text-expected.txt b/WebKitTools/Scripts/webkitpy/layout_tests/data/passes/text-expected.txt
new file mode 100644 (file)
index 0000000..2b38a06
--- /dev/null
@@ -0,0 +1 @@
+text-txt
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/passes/text.html b/WebKitTools/Scripts/webkitpy/layout_tests/data/passes/text.html
new file mode 100644 (file)
index 0000000..8e27be7
--- /dev/null
@@ -0,0 +1 @@
+text
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/platform/test/image/canvas-bg-expected.checksum b/WebKitTools/Scripts/webkitpy/layout_tests/data/platform/test/image/canvas-bg-expected.checksum
deleted file mode 100644 (file)
index 4cd8dac..0000000
+++ /dev/null
@@ -1 +0,0 @@
-790b681a41697634fcf2a2587afb89c6
\ No newline at end of file
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/platform/test/image/canvas-bg-expected.png b/WebKitTools/Scripts/webkitpy/layout_tests/data/platform/test/image/canvas-bg-expected.png
deleted file mode 100644 (file)
index 3d00450..0000000
Binary files a/WebKitTools/Scripts/webkitpy/layout_tests/data/platform/test/image/canvas-bg-expected.png and /dev/null differ
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/platform/test/image/canvas-bg-expected.txt b/WebKitTools/Scripts/webkitpy/layout_tests/data/platform/test/image/canvas-bg-expected.txt
deleted file mode 100644 (file)
index 2411c0a..0000000
+++ /dev/null
@@ -1,6 +0,0 @@
-layer at (0,0) size 785x620
-  RenderView at (0,0) size 785x600
-layer at (0,0) size 785x620
-  RenderBlock {HTML} at (0,0) size 785x620
-    RenderBody {BODY} at (8,8) size 769x604
-      RenderBlock {DIV} at (0,0) size 604x604 [border: (2px solid #000000)]
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/platform/test/passes/platform_image-expected.checksum b/WebKitTools/Scripts/webkitpy/layout_tests/data/platform/test/passes/platform_image-expected.checksum
new file mode 100644 (file)
index 0000000..ea557cf
--- /dev/null
@@ -0,0 +1 @@
+platform_image-checksum
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/platform/test/passes/platform_image-expected.png b/WebKitTools/Scripts/webkitpy/layout_tests/data/platform/test/passes/platform_image-expected.png
new file mode 100644 (file)
index 0000000..ec42fc1
--- /dev/null
@@ -0,0 +1 @@
+platform_image-png
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/platform/test/passes/platform_image-expected.txt b/WebKitTools/Scripts/webkitpy/layout_tests/data/platform/test/passes/platform_image-expected.txt
new file mode 100644 (file)
index 0000000..ff8bf43
--- /dev/null
@@ -0,0 +1 @@
+platform_image-txt
index b78a01c..68d5641 100644 (file)
@@ -1 +1,5 @@
-WONTFIX : misc/missing-expectation.html = MISSING PASS
+WONTFIX : failures/expected/text.html = TEXT
+WONTFIX SKIP : failures/expected/crash.html = CRASH
+WONTFIX : failures/expected/image_checksum.html = IMAGE
+WONTFIX : failures/expected/missing_text.html = MISSING PASS
+WONTFIX : failures/expected/missing_image.html = MISSING PASS
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/text/article-element-expected.txt b/WebKitTools/Scripts/webkitpy/layout_tests/data/text/article-element-expected.txt
deleted file mode 100644 (file)
index f60ac38..0000000
+++ /dev/null
@@ -1,20 +0,0 @@
-Various tests for the article element.
-
-On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
-
-
-<article> closes <p>:
-PASS article1.parentNode.nodeName == "p" is false
-<p> does not close <article>:
-PASS p1.parentNode.nodeName is "ARTICLE"
-<article> can be nested inside <article>:
-PASS article3.parentNode.id is "article2"
-Residual style:
-PASS getWeight("article4") is "bold"
-PASS getWeight("span1") is "bold"
-FormatBlock:
-PASS document.getElementById("span2").parentNode.nodeName is "ARTICLE"
-PASS successfullyParsed is true
-
-TEST COMPLETE
-
diff --git a/WebKitTools/Scripts/webkitpy/layout_tests/data/text/article-element.html b/WebKitTools/Scripts/webkitpy/layout_tests/data/text/article-element.html
deleted file mode 100644 (file)
index c0f4547..0000000
+++ /dev/null
@@ -1,13 +0,0 @@
-<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
-<html>
-<head>
-<link rel="stylesheet" href="../../fast/js/resources/js-test-style.css">
-<script src="../../fast/js/resources/js-test-pre.js"></script>
-</head>
-<body>
-<p id="description"></p>
-<div id="console"></div>
-<script src="script-tests/article-element.js"></script>
-<script src="../../fast/js/resources/js-test-post.js"></script>
-</body>
-</html>
index cf3c560..394807a 100644 (file)
@@ -33,16 +33,8 @@ import os
 import sys
 import unittest
 
-try:
-   d = os.path.dirname(__file__)
-except NameError:
-   d = os.path.dirname(sys.argv[0])
-
-sys.path.append(os.path.abspath(os.path.join(d, '..')))
-sys.path.append(os.path.abspath(os.path.join(d, '../../thirdparty')))
-
-import port
-from test_expectations import *
+from webkitpy.layout_tests import port
+from webkitpy.layout_tests.layout_package.test_expectations import *
 
 class FunctionsTest(unittest.TestCase):
     def test_result_was_expected(self):
@@ -98,18 +90,18 @@ class TestExpectationsTest(unittest.TestCase):
         return os.path.join(self._port.layout_tests_dir(), test_name)
 
     def get_basic_tests(self):
-        return [self.get_test('text/article-element.html'),
-                self.get_test('image/canvas-bg.html'),
-                self.get_test('image/canvas-zoom.html'),
-                self.get_test('misc/crash.html'),
-                self.get_test('misc/passing.html')]
+        return [self.get_test('failures/expected/text.html'),
+                self.get_test('failures/expected/image_checksum.html'),
+                self.get_test('failures/expected/crash.html'),
+                self.get_test('failures/expected/missing_text.html'),
+                self.get_test('passes/text.html')]
 
     def get_basic_expectations(self):
         return """
-BUG_TEST : text/article-element.html = TEXT
-BUG_TEST SKIP : misc/crash.html = CRASH
-BUG_TEST REBASELINE : misc/missing-expectation.html = MISSING
-BUG_TEST : image = IMAGE
+BUG_TEST : failures/expected/text.html = TEXT
+BUG_TEST SKIP : failures/expected/crash.html = CRASH
+BUG_TEST REBASELINE : failure/expected/missing_image.html = MISSING
+BUG_TEST : failures/expected/image_checksum.html = IMAGE
 """
 
     def parse_exp(self, expectations, overrides=None):
@@ -128,23 +120,23 @@ BUG_TEST : image = IMAGE
 
     def test_basic(self):
         self.parse_exp(self.get_basic_expectations())
-        self.assert_exp('text/article-element.html', TEXT)
-        self.assert_exp('image/canvas-zoom.html', IMAGE)
-        self.assert_exp('misc/passing.html', PASS)
+        self.assert_exp('failures/expected/text.html', TEXT)
+        self.assert_exp('failures/expected/image_checksum.html', IMAGE)
+        self.assert_exp('passes/text.html', PASS)
 
     def test_duplicates(self):
         self.assertRaises(SyntaxError, self.parse_exp, """
-BUG_TEST : text/article-element.html = TEXT
-BUG_TEST : text/article-element.html = IMAGE""")
+BUG_TEST : failures/expected/text.html = TEXT
+BUG_TEST : failures/expected/text.html = IMAGE""")
         self.assertRaises(SyntaxError, self.parse_exp,
             self.get_basic_expectations(), """
-BUG_TEST : text/article-element.html = TEXT
-BUG_TEST : text/article-element.html = IMAGE""")
+BUG_TEST : failures/expected/text.html = TEXT
+BUG_TEST : failures/expected/text.html = IMAGE""")
 
     def test_overrides(self):
         self.parse_exp(self.get_basic_expectations(), """
-BUG_OVERRIDE : text/article-element.html = IMAGE""")
-        self.assert_exp('text/article-element.html', IMAGE)
+BUG_OVERRIDE : failures/expected/text.html = IMAGE""")
+        self.assert_exp('failures/expected/text.html', IMAGE)
 
     def test_matches_an_expected_result(self):
 
@@ -153,16 +145,14 @@ BUG_OVERRIDE : text/article-element.html = IMAGE""")
                 self.get_test(test), result, pixel_tests_enabled)
 
         self.parse_exp(self.get_basic_expectations())
-        self.assertTrue(match('text/article-element.html', TEXT, True))
-        self.assertTrue(match('text/article-element.html', TEXT, False))
-        self.assertFalse(match('text/article-element.html', CRASH, True))
-        self.assertFalse(match('text/article-element.html', CRASH, False))
-
-        self.assertTrue(match('image/canvas-bg.html', IMAGE, True))
-        self.assertTrue(match('image/canvas-bg.html', PASS, False))
-
-        self.assertTrue(match('misc/crash.html', SKIP, False))
-        self.assertTrue(match('misc/passing.html', PASS, False))
+        self.assertTrue(match('failures/expected/text.html', TEXT, True))
+        self.assertTrue(match('failures/expected/text.html', TEXT, False))
+        self.assertFalse(match('failures/expected/text.html', CRASH, True))
+        self.assertFalse(match('failures/expected/text.html', CRASH, False))
+        self.assertTrue(match('failures/expected/image_checksum.html', IMAGE, True))
+        self.assertTrue(match('failures/expected/image_checksum.html', PASS, False))
+        self.assertTrue(match('failures/expected/crash.html', SKIP, False))
+        self.assertTrue(match('passes/text.html', PASS, False))
 
 if __name__ == '__main__':
     unittest.main()
index 9c9ab0a..63c6896 100644 (file)
@@ -48,7 +48,7 @@ class TestPort(base.Port):
 
     def baseline_path(self):
         return os.path.join(self.layout_tests_dir(), 'platform',
-                            self.name())
+                            self.name() + self.version())
 
     def baseline_search_path(self):
         return [self.baseline_path()]
@@ -56,16 +56,18 @@ class TestPort(base.Port):
     def check_build(self, needs_http):
         return True
 
-    def compare_text(self, expected_text, actual_text):
-        return False
-
     def diff_image(self, expected_filename, actual_filename,
                    diff_filename=None, tolerance=0):
-        return False
-
-    def diff_text(self, expected_text, actual_text,
-                  expected_filename, actual_filename):
-        return ''
+        with codecs.open(actual_filename, "r", "utf-8") as actual_fh:
+            actual_contents = actual_fh.read()
+        with codecs.open(expected_filename, "r", "utf-8") as expected_fh:
+            expected_contents = expected_fh.read()
+        diffed = actual_contents != expected_contents
+        if diffed and diff_filename:
+            with codecs.open(diff_filename, "w", "utf-8") as diff_fh:
+                diff_fh.write("< %s\n---\n> %s\n" %
+                              (expected_contents, actual_contents))
+        return diffed
 
     def layout_tests_dir(self):
         return self.path_from_webkit_base('WebKitTools', 'Scripts',
@@ -82,6 +84,9 @@ class TestPort(base.Port):
             'webkitpy', 'layout_tests', 'data', 'platform', 'test',
             'test_expectations.txt')
 
+    def _path_to_wdiff(self):
+        return None
+
     def results_directory(self):
         return '/tmp/' + self._options.results_directory
 
@@ -127,10 +132,7 @@ class TestPort(base.Port):
     def test_platform_name_to_name(self, test_platform_name):
         return test_platform_name
 
-    def version():
-        return ''
-
-    def wdiff_text(self, expected_filename, actual_filename):
+    def version(self):
         return ''
 
 
@@ -150,16 +152,58 @@ class TestDriver(base.Driver):
         return 0
 
     def run_test(self, uri, timeoutms, image_hash):
-        if not self._image_written and self._port._options.pixel_tests:
-            with open(self._image_path, "w") as f:
-                f.write("bad png file from TestDriver")
-                self._image_written = True
-
-        # We special-case this because we can't fake an image hash for a
-        # missing expectation.
-        if uri.find('misc/missing-expectation') != -1:
-            return (False, False, 'deadbeefdeadbeefdeadbeefdeadbeef', '', None)
-        return (False, False, image_hash, '', None)
+        basename = uri[(uri.rfind("/") + 1):uri.rfind(".html")]
+
+        error = ''
+        checksum = None
+        # There are four currently supported types of tests: text, image,
+        # image hash (checksum), and stderr output. The fake output
+        # is the basename of the file + "-" plus the type of test output
+        # (or a blank string for stderr).
+        #
+        # If 'image' or 'check' appears in the basename, we assume this is
+        # simulating a pixel test.
+        #
+        # If 'failures' appears in the URI, then we assume this test should
+        # fail. Which type of failures are determined by which strings appear
+        # in the basename of the test. For failures that produce outputs,
+        # we change the fake output to basename + "_failed-".
+        #
+        # The fact that each test produces (more or less) unique output data
+        # will allow us to see if any results get crossed by the rest of the
+        # program.
+        if 'failures' in uri:
+            crash = 'crash' in basename
+            timeout = 'timeout' in basename
+            if 'error' in basename:
+                error = basename + "_error\n"
+            if 'text' in basename:
+                output = basename + '_failed-txt\n'
+            else:
+                output = basename + '-txt\n'
+            if self._port.options().pixel_tests:
+                if ('image' in basename or 'check' in basename):
+                    checksum = basename + "-checksum\n"
+
+                if 'image' in basename:
+                    with open(self._image_path, "w") as f:
+                        f.write(basename + "_failed-png\n")
+                elif 'check' in basename:
+                    with open(self._image_path, "w") as f:
+                        f.write(basename + "-png\n")
+                if 'checksum' in basename:
+                    checksum = basename + "_failed-checksum\n"
+        else:
+            crash = False
+            timeout = False
+            output = basename + '-txt\n'
+            if self._port.options().pixel_tests and (
+                'image' in basename or 'check' in basename):
+                checksum = basename + '-checksum\n'
+                with open(self._image_path, "w") as f:
+                    f.write(basename + "-png")
+
+        return (crash, timeout, checksum, output, error)
 
     def start(self):
         pass
index adba1ff..922d913 100644 (file)
@@ -48,8 +48,11 @@ from webkitpy.layout_tests.layout_package import dump_render_tree_thread
 from webkitpy.thirdparty.mock import Mock
 
 
-def passing_run(args, port_obj=None, record_results=False):
+def passing_run(args, port_obj=None, record_results=False,
+                tests_included=False):
     args.extend(['--print', 'nothing'])
+    if not tests_included:
+        args.extend(['passes', 'failures/expected'])
     if not record_results:
         args.append('--no-record-results')
     options, args = run_webkit_tests.parse_args(args)
@@ -75,7 +78,7 @@ class MainTest(unittest.TestCase):
         self.assertTrue(passing_run(['--platform', 'test']))
         self.assertTrue(passing_run(['--platform', 'test', '--run-singly']))
         self.assertTrue(passing_run(['--platform', 'test',
-                                     'text/article-element.html']))
+                                     'passes/text.html'], tests_included=True))
 
     def test_one_child_process(self):
         (res, buildbot_output, regular_output) = logging_run(
@@ -122,21 +125,19 @@ class RebaselineTest(unittest.TestCase):
         original_open = codecs.open
         try:
             # Test that we update expectations in place. If the expectation
-            # is mssing, update the expected generic location.
+            # is missing, update the expected generic location.
             file_list = []
             codecs.open = _mocked_open(original_open, file_list)
             passing_run(['--platform', 'test', '--pixel-tests',
                          '--reset-results',
-                         'image/canvas-bg.html',
-                         'image/canvas-zoom.html',
-                         'misc/missing-expectation.html'])
-            self.assertEqual(len(file_list), 9)
+                         'passes/image.html',
+                         'failures/expected/missing_image.html'],
+                         tests_included=True)
+            self.assertEqual(len(file_list), 6)
             self.assertBaselines(file_list,
-                "data/image/canvas-zoom")
+                "data/passes/image")
             self.assertBaselines(file_list,
-                "data/platform/test/image/canvas-bg")
-            self.assertBaselines(file_list,
-                "data/misc/missing-expectation")
+                "data/failures/expected/missing_image")
         finally:
             codecs.open = original_open
 
@@ -150,16 +151,14 @@ class RebaselineTest(unittest.TestCase):
             codecs.open = _mocked_open(original_open, file_list)
             passing_run(['--platform', 'test', '--pixel-tests',
                          '--new-baseline',
-                         'image/canvas-zoom.html',
-                         'image/canvas-bg.html',
-                         'misc/missing-expectation.html'])
-            self.assertEqual(len(file_list), 9)
-            self.assertBaselines(file_list,
-                "data/platform/test/image/canvas-zoom")
+                         'passes/image.html',
+                         'failures/expected/missing_image.html'],
+                        tests_included=True)
+            self.assertEqual(len(file_list), 6)
             self.assertBaselines(file_list,
-                "data/platform/test/image/canvas-bg")
+                "data/platform/test/passes/image")
             self.assertBaselines(file_list,
-                "data/platform/test/misc/missing-expectation")
+                "data/platform/test/failures/expected/missing_image")
         finally:
             codecs.open = original_open
 
index aa219b2..c0bb4ac 100644 (file)
@@ -67,7 +67,7 @@ class TestExpectationsTestCase(unittest.TestCase):
     def setUp(self):
         self._error_collector = ErrorCollector()
         port_obj = port.get('test')
-        self._test_file = os.path.join(port_obj.layout_tests_dir(), 'misc/passing.html')
+        self._test_file = os.path.join(port_obj.layout_tests_dir(), 'passes/text.html')
 
     def process_expectations(self, expectations, overrides=None):
         self._checker = TestExpectationsChecker()
@@ -84,88 +84,88 @@ class TestExpectationsTestCase(unittest.TestCase):
 
     def test_valid_expectations(self):
         self.assert_lines_lint(
-            ["misc/passing.html = PASS"],
+            ["passes/text.html = PASS"],
             "")
         self.assert_lines_lint(
-            ["misc/passing.html = FAIL PASS"],
+            ["passes/text.html = FAIL PASS"],
             "")
         self.assert_lines_lint(
-            ["misc/passing.html = CRASH TIMEOUT FAIL PASS"],
+            ["passes/text.html = CRASH TIMEOUT FAIL PASS"],
             "")
         self.assert_lines_lint(
-            ["BUG1234 TEST : misc/passing.html = PASS FAIL"],
+            ["BUG1234 TEST : passes/text.html = PASS FAIL"],
             "")
         self.assert_lines_lint(
-            ["SKIP BUG1234 : misc/passing.html = TIMEOUT PASS"],
+            ["SKIP BUG1234 : passes/text.html = TIMEOUT PASS"],
             "")
         self.assert_lines_lint(
-            ["BUG1234 DEBUG : misc/passing.html = TIMEOUT PASS"],
+            ["BUG1234 DEBUG : passes/text.html = TIMEOUT PASS"],
             "")
         self.assert_lines_lint(
-            ["BUG1234 DEBUG SKIP : misc/passing.html = TIMEOUT PASS"],
+            ["BUG1234 DEBUG SKIP : passes/text.html = TIMEOUT PASS"],
             "")
         self.assert_lines_lint(
-            ["BUG1234 TEST DEBUG SKIP : misc/passing.html = TIMEOUT PASS"],
+            ["BUG1234 TEST DEBUG SKIP : passes/text.html = TIMEOUT PASS"],
             "")
         self.assert_lines_lint(
-            ["BUG1234 DEBUG TEST : misc/passing.html = TIMEOUT PASS"],
+            ["BUG1234 DEBUG TEST : passes/text.html = TIMEOUT PASS"],
             "")
         self.assert_lines_lint(
-            ["SLOW DEFER BUG1234 : misc/passing.html = PASS"],
+            ["SLOW DEFER BUG1234 : passes/text.html = PASS"],
             "")
         self.assert_lines_lint(
-            ["WONTFIX SKIP : misc/passing.html = TIMEOUT"],
+            ["WONTFIX SKIP : passes/text.html = TIMEOUT"],
             "")
 
     def test_valid_modifiers(self):
         self.assert_lines_lint(
-            ["INVALID-MODIFIER : misc/passing.html = PASS"],
+            ["INVALID-MODIFIER : passes/text.html = PASS"],
             "Invalid modifier for test: invalid-modifier "
-            "misc/passing.html  [test/expectations] [5]")
+            "passes/text.html  [test/expectations] [5]")
         self.assert_lines_lint(
-            ["SKIP : misc/passing.html = PASS"],
+            ["SKIP : passes/text.html = PASS"],
             "Test lacks BUG modifier. "
-            "misc/passing.html  [test/expectations] [2]")
+            "passes/text.html  [test/expectations] [2]")
         self.assert_lines_lint(
-            ["WONTFIX DEFER : misc/passing.html = PASS"],
+            ["WONTFIX DEFER : passes/text.html = PASS"],
             "Test cannot be both DEFER and WONTFIX. "
-            "misc/passing.html  [test/expectations] [5]")
+            "passes/text.html  [test/expectations] [5]")
 
     def test_expectation_errors(self):
         self.assert_lines_lint(
             ["missing expectations"],
             "Missing expectations. ['missing expectations']  [test/expectations] [5]")
         self.assert_lines_lint(
-            ["SLOW : misc/passing.html = TIMEOUT"],
+            ["SLOW : passes/text.html = TIMEOUT"],
             "A test can not be both slow and timeout. "
             "If it times out indefinitely, then it should be just timeout. "
-            "misc/passing.html  [test/expectations] [5]")
+            "passes/text.html  [test/expectations] [5]")
         self.assert_lines_lint(
             ["does/not/exist.html = FAIL"],
             "Path does not exist. does/not/exist.html  [test/expectations] [2]")
 
     def test_parse_expectations(self):
         self.assert_lines_lint(
-            ["misc/passing.html = PASS"],
+            ["passes/text.html = PASS"],
             "")
         self.assert_lines_lint(
-            ["misc/passing.html = UNSUPPORTED"],
+            ["passes/text.html = UNSUPPORTED"],
             "Unsupported expectation: unsupported "
-            "misc/passing.html  [test/expectations] [5]")
+            "passes/text.html  [test/expectations] [5]")
         self.assert_lines_lint(
-            ["misc/passing.html = PASS UNSUPPORTED"],
+            ["passes/text.html = PASS UNSUPPORTED"],
             "Unsupported expectation: unsupported "
-            "misc/passing.html  [test/expectations] [5]")
+            "passes/text.html  [test/expectations] [5]")
 
     def test_already_seen_test(self):
         self.assert_lines_lint(
-            ["misc/passing.html = PASS",
-             "misc/passing.html = TIMEOUT"],
+            ["passes/text.html = PASS",
+             "passes/text.html = TIMEOUT"],
             "Duplicate expectation. %s  [test/expectations] [5]" % self._test_file)
 
     def test_tab(self):
         self.assert_lines_lint(
-            ["\tmisc/passing.html = PASS"],
+            ["\tpasses/text.html = PASS"],
             "Line contains tab character.  [whitespace/tab] [5]")
 
 if __name__ == '__main__':