Web Inspector: add TestSuite hooks for per-testcase setup and teardown actions
authorbburg@apple.com <bburg@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 1 Sep 2015 17:37:16 +0000 (17:37 +0000)
committerbburg@apple.com <bburg@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 1 Sep 2015 17:37:16 +0000 (17:37 +0000)
https://bugs.webkit.org/show_bug.cgi?id=148641

Reviewed by Timothy Hatcher.

Source/WebInspectorUI:

It should be possible to run setup and teardown actions for each test case.
The use case here is to add listeners, set up testcase-specific state, etc.
To match how test functions work in each suite, the sync and async implementations
take functions with no arguments and(resolve, reject), respectively.

* UserInterface/Test/TestSuite.js:
(TestSuite.prototype.addTestCase):
(TestSuite.messageFromThrownObject):
(TestSuite):
(AsyncTestSuite.prototype.runTestCases): Stick the extra functions in the chain.
(AsyncTestSuite):
(SyncTestSuite.prototype.runTestCases):
(SyncTestSuite):

LayoutTests:

Extend existing unit tests to also exercise setup and teardown actions.

* inspector/unit-tests/async-test-suite.html:
* inspector/unit-tests/async-test-suite-expected.txt:
* inspector/unit-tests/sync-test-suite.html:
* inspector/unit-tests/sync-test-suite-expected.txt:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@189213 268f45cc-cd09-0410-ab3c-d52691b4dbfc

LayoutTests/ChangeLog
LayoutTests/inspector/unit-tests/async-test-suite-expected.txt
LayoutTests/inspector/unit-tests/async-test-suite.html
LayoutTests/inspector/unit-tests/sync-test-suite-expected.txt
LayoutTests/inspector/unit-tests/sync-test-suite.html
Source/WebInspectorUI/ChangeLog
Source/WebInspectorUI/UserInterface/Test/TestSuite.js

index 97b1c2f..2e11c84 100644 (file)
@@ -1,3 +1,17 @@
+2015-09-01  Brian Burg  <bburg@apple.com>
+
+        Web Inspector: add TestSuite hooks for per-testcase setup and teardown actions
+        https://bugs.webkit.org/show_bug.cgi?id=148641
+
+        Reviewed by Timothy Hatcher.
+
+        Extend existing unit tests to also exercise setup and teardown actions.
+
+        * inspector/unit-tests/async-test-suite.html:
+        * inspector/unit-tests/async-test-suite-expected.txt:
+        * inspector/unit-tests/sync-test-suite.html:
+        * inspector/unit-tests/sync-test-suite-expected.txt:
+
 2015-09-01  Zalan Bujtas  <zalan@apple.com>
 
         Repaint cleanup:
index b913390..665300e 100644 (file)
@@ -7,6 +7,12 @@ PASS: should not be able to add non-object test case.
 PASS: test case should require string name.
 PASS: test case should require non-whitespace name.
 PASS: test case should require test function.
+PASS: should not be able to specify non-Function `setup` parameter.
+PASS: should not be able to specify non-Function `setup` parameter.
+PASS: should not be able to specify non-Function `setup` parameter.
+PASS: should not be able to specify non-Function `teardown` parameter.
+PASS: should not be able to specify non-Function `teardown` parameter.
+PASS: should not be able to specify non-Function `teardown` parameter.
 PASS: should not be able to run empty test suite.
 
 == Running test suite: AsyncTestSuite.RunTwiceSuite
@@ -38,3 +44,36 @@ PASS: abortOnFailureSuite should have passed one test.
 PASS: abortOnFailureSuite should have failed one test.
 PASS: abortOnFailureSuite should have skipped one test.
 
+== Running test suite: AsyncTestSuite.SetupAndTeardown
+-- Running test setup.
+-- Running test case: TestWithSetupAndTeardown
+PASS: Test should see side effects of running setup() action.
+-- Running test teardown.
+PASS: Teardown should see side effects of running setup() action.
+
+-- Running test case: TestRunningAfterTeardown
+PASS: Test should see side effects of previous test's teardown() action.
+PASS: Promise from setupAndTeardownTestSuite.runTestCases() should resolve.
+
+== Running test suite: AsyncTestSuite.SetupException
+-- Running test setup.
+!! EXCEPTION: 
+PASS: Promise from setupExceptionTestSuite.runTestCases() should reject.
+
+== Running test suite: AsyncTestSuite.SetupFailure
+-- Running test setup.
+!! EXCEPTION: undefined
+PASS: Promise from setupFailureTestSuite.runTestCases() should reject.
+
+== Running test suite: SyncTestSuite.TeardownException
+-- Running test case: TestWithExceptionDuringTeardown
+-- Running test teardown.
+!! EXCEPTION: 
+PASS: Promise from teardownExceptionTestSuite.runTestCases() should reject.
+
+== Running test suite: SyncTestSuite.TeardownFailure
+-- Running test case: TestWithExceptionDuringTeardown
+-- Running test teardown.
+!! EXCEPTION: undefined
+PASS: Promise from teardownFailureTestSuite.runTestCases() should reject.
+
index 9eee43c..9398371 100644 (file)
@@ -73,6 +73,66 @@ function test()
     } catch (e) {
         ProtocolTest.log("PASS: test case should require test function.");
     }
+    try {
+        badArgsSuite.addTestCase({
+            name: "foo",
+            test: () => {},
+            setup: "astd"
+        });
+        ProtocolTest.log("FAIL: should not be able to specify non-Function `setup` parameter.");
+    } catch (e) {
+        ProtocolTest.log("PASS: should not be able to specify non-Function `setup` parameter.");
+    }
+    try {
+        badArgsSuite.addTestCase({
+            name: "foo",
+            test: () => {},
+            setup: 123
+        });
+        ProtocolTest.log("FAIL: should not be able to specify non-Function `setup` parameter.");
+    } catch (e) {
+        ProtocolTest.log("PASS: should not be able to specify non-Function `setup` parameter.");
+    }
+    try {
+        badArgsSuite.addTestCase({
+            name: "foo",
+            test: () => {},
+            setup: {}
+        });
+        ProtocolTest.log("FAIL: should not be able to specify non-Function `setup` parameter.");
+    } catch (e) {
+        ProtocolTest.log("PASS: should not be able to specify non-Function `setup` parameter.");
+    }
+    try {
+        badArgsSuite.addTestCase({
+            name: "foo",
+            test: () => {},
+            teardown: "astd"
+        });
+        ProtocolTest.log("FAIL: should not be able to specify non-Function `teardown` parameter.");
+    } catch (e) {
+        ProtocolTest.log("PASS: should not be able to specify non-Function `teardown` parameter.");
+    }
+    try {
+        badArgsSuite.addTestCase({
+            name: "foo",
+            test: () => {},
+            teardown: 123
+        });
+        ProtocolTest.log("FAIL: should not be able to specify non-Function `teardown` parameter.");
+    } catch (e) {
+        ProtocolTest.log("PASS: should not be able to specify non-Function `teardown` parameter.");
+    }
+    try {
+        badArgsSuite.addTestCase({
+            name: "foo",
+            test: () => {},
+            teardown: {}
+        });
+        ProtocolTest.log("FAIL: should not be able to specify non-Function `teardown` parameter.");
+    } catch (e) {
+        ProtocolTest.log("PASS: should not be able to specify non-Function `teardown` parameter.");
+    }
 
     let runEmptySuite = ProtocolTest.createAsyncSuite("AsyncTestSuite.RunEmptySuite");
     try {
@@ -176,6 +236,157 @@ function test()
         return Promise.resolve(); // Continue this test.
     });
 
+    var setupAndTeardownSymbol = Symbol("async-suite-setup-and-teardown-token");
+    window[setupAndTeardownSymbol] = 0;
+
+    let setupAndTeardownTestSuite = ProtocolTest.createAsyncSuite("AsyncTestSuite.SetupAndTeardown");
+    setupAndTeardownTestSuite.addTestCase({
+        name: "TestWithSetupAndTeardown",
+        description: "Check execution order for setup and teardown actions.",
+        setup: (resolve, reject) => {
+            window[setupAndTeardownSymbol] = 1;
+            resolve();
+        },
+        test: (resolve, reject) => {
+            ProtocolTest.expectThat(window[setupAndTeardownSymbol] === 1, "Test should see side effects of running setup() action.");
+            window[setupAndTeardownSymbol] = 2;
+            resolve();
+        },
+        teardown: (resolve, reject) => {
+            ProtocolTest.expectThat(window[setupAndTeardownSymbol] === 2, "Teardown should see side effects of running setup() action.");
+            window[setupAndTeardownSymbol] = 3;
+            resolve();
+        }
+    });
+    setupAndTeardownTestSuite.addTestCase({
+        name: "TestRunningAfterTeardown",
+        description: "Check execution order for test after a teardown action.",
+        test: (resolve, reject) => {
+            ProtocolTest.expectThat(window[setupAndTeardownSymbol] === 3, "Test should see side effects of previous test's teardown() action.");
+            resolve();
+        },
+    });
+
+    result = result.then(() => {
+        return setupAndTeardownTestSuite.runTestCases();
+    }).then(function resolved() {
+        ProtocolTest.log("PASS: Promise from setupAndTeardownTestSuite.runTestCases() should resolve.");
+        return Promise.resolve(); // Continue this test.
+    }, function rejected(e) {
+        ProtocolTest.log("FAIL: Promise from setupAndTeardownTestSuite.runTestCases() should resolve.");
+        return Promise.resolve(); // Continue this test.
+    });
+
+    let setupExceptionTestSuite = ProtocolTest.createAsyncSuite("AsyncTestSuite.SetupException");
+    setupExceptionTestSuite.addTestCase({
+        name: "TestWithExceptionDuringSetup",
+        description: "Check execution order for setup action that throws an exception.",
+        setup: (resolve, reject) => { throw new Error() },
+        test: (resolve, reject) => {
+            ProtocolTest.assert(false, "Test should not execute if its setup action threw an exception.");
+            reject();
+        },
+        teardown: (resolve, reject) => {
+            ProtocolTest.assert(false, "Teardown action should not execute if its setup action threw an exception.");
+            reject();           
+        }
+    });
+
+    result = result.then(() => {
+        return setupExceptionTestSuite.runTestCases();
+    }).then(function resolved() {
+        ProtocolTest.log("FAIL: Promise from setupExceptionTestSuite.runTestCases() should reject.");
+        return Promise.resolve(); // Continue this test.
+    }, function rejected(e) {
+        ProtocolTest.log("PASS: Promise from setupExceptionTestSuite.runTestCases() should reject.");
+        return Promise.resolve(); // Continue this test.
+    });
+
+    let setupFailureTestSuite = ProtocolTest.createAsyncSuite("AsyncTestSuite.SetupFailure");
+    setupFailureTestSuite.addTestCase({
+        name: "TestWithFailureDuringSetup",
+        description: "Check execution order for setup action that has failed.",
+        setup: (resolve, reject) => { reject(); },
+        test: (resolve, reject) => {
+            ProtocolTest.assert(false, "Test should not execute if its setup action failed.")
+            reject();
+        },
+        teardown: (resolve, reject) => {
+            ProtocolTest.assert(false, "Teardown action should not execute if its setup action failed.")
+            reject();
+        }
+    });
+
+    result = result.then(() => {
+        return setupFailureTestSuite.runTestCases();
+    }).then(function resolved() {
+        ProtocolTest.log("FAIL: Promise from setupFailureTestSuite.runTestCases() should reject.");
+        return Promise.resolve(); // Continue this test.
+    }, function rejected(e) {
+        ProtocolTest.log("PASS: Promise from setupFailureTestSuite.runTestCases() should reject.");
+        return Promise.resolve(); // Continue this test.
+    });
+
+    let teardownExceptionTestSuite = ProtocolTest.createAsyncSuite("SyncTestSuite.TeardownException");
+    teardownExceptionTestSuite.addTestCase({
+        name: "TestWithExceptionDuringTeardown",
+        description: "Check execution order for teardown action that throws an exception.",
+        test: (resolve, reject) => { resolve(); },
+        teardown: (resolve, reject) => { throw new Error() }
+    });
+    teardownExceptionTestSuite.addTestCase({
+        name: "TestAfterTeardownException",
+        descrption: "Check execution order for test after previous test's teardown throws an exception.",
+        setup: (resolve, reject) => {
+            ProtocolTest.assert(false, "Setup action should not execute if previous test's teardown action threw an exception.");
+            reject();
+        },
+        test: (resolve, reject) => {
+            ProtocolTest.assert(false, "Test should not execute if previous test's teardown action threw an exception.");
+            reject();
+        }
+    });
+
+    result = result.then(() => {
+        return teardownExceptionTestSuite.runTestCases();
+    }).then(function resolved() {
+        ProtocolTest.log("FAIL: Promise from teardownExceptionTestSuite.runTestCases() should reject.");
+        return Promise.resolve(); // Continue this test.
+    }, function rejected(e) {
+        ProtocolTest.log("PASS: Promise from teardownExceptionTestSuite.runTestCases() should reject.");
+        return Promise.resolve(); // Continue this test.
+    });
+
+    let teardownFailureTestSuite = ProtocolTest.createAsyncSuite("SyncTestSuite.TeardownFailure");
+    teardownFailureTestSuite.addTestCase({
+        name: "TestWithExceptionDuringTeardown",
+        description: "Check execution order for teardown action that has failed.",
+        test: (resolve, reject) => { resolve(); },
+        teardown: (resolve, reject) => { reject(); },
+    });
+    teardownFailureTestSuite.addTestCase({
+        name: "TestAfterTeardownException",
+        description: "Check execution order for test after previous test's teardown throws an exception",
+        setup: (resolve, reject) => {
+            ProtocolTest.assert(false, "Setup action should not execute if previous test's teardown action failed.");
+            reject();
+        },
+        test: (resolve, reject) => {
+            ProtocolTest.assert(false, "Test should not execute if previous test's teardown action failed.");
+            reject();
+        }
+    });
+
+    result = result.then(() => {
+        return teardownFailureTestSuite.runTestCases();
+    }).then(function resolved() {
+        ProtocolTest.log("FAIL: Promise from teardownFailureTestSuite.runTestCases() should reject.");
+        return Promise.resolve(); // Continue this test.
+    }, function rejected(e) {
+        ProtocolTest.log("PASS: Promise from teardownFailureTestSuite.runTestCases() should reject.");
+        return Promise.resolve(); // Continue this test.
+    });
+
     // This will finish the test whether the chain was resolved or rejected.
     result = result.then(() => { ProtocolTest.completeTest(); });
 }
index 3babc59..f5130d0 100644 (file)
@@ -7,6 +7,12 @@ PASS: should not be able to add non-object test case.
 PASS: test case should require string name.
 PASS: test case should require non-whitespace name.
 PASS: test case should require test function.
+PASS: should not be able to specify non-Function `setup` parameter.
+PASS: should not be able to specify non-Function `setup` parameter.
+PASS: should not be able to specify non-Function `setup` parameter.
+PASS: should not be able to specify non-Function `teardown` parameter.
+PASS: should not be able to specify non-Function `teardown` parameter.
+PASS: should not be able to specify non-Function `teardown` parameter.
 PASS: should not be able to run empty test suite.
 
 == Running test suite: SyncTestSuite.RunTwiceSuite
@@ -35,3 +41,31 @@ PASS: abortOnFailureSuite should have passed one test.
 PASS: abortOnFailureSuite should have failed one test.
 PASS: abortOnFailureSuite should have skipped one test.
 
+== Running test suite: SyncTestSuite.SetupAndTeardown
+-- Running test setup.
+-- Running test case: TestWithSetupAndTeardown
+PASS: Test should see side effects of running setup() action.
+-- Running test teardown.
+PASS: Teardown should see side effects of running setup() action.
+
+-- Running test case: TestRunningAfterTeardown
+PASS: Test should see side effects of previous test's teardown() action.
+
+== Running test suite: SyncTestSuite.SetupException
+-- Running test setup.
+!! EXCEPTION: 
+
+== Running test suite: SyncTestSuite.SetupFailure
+-- Running test setup.
+!! EXCEPTION
+
+== Running test suite: SyncTestSuite.TeardownException
+-- Running test case: TestWithExceptionDuringTeardown
+-- Running test teardown.
+!! EXCEPTION: 
+
+== Running test suite: SyncTestSuite.TeardownFailure
+-- Running test case: TestWithExceptionDuringTeardown
+-- Running test teardown.
+!! EXCEPTION:
+
index 70342fe..c624f0f 100644 (file)
@@ -73,7 +73,66 @@ function test()
     } catch (e) {
         ProtocolTest.log("PASS: test case should require test function.");
     }
-
+    try {
+        badArgsSuite.addTestCase({
+            name: "foo",
+            test: () => {},
+            setup: "astd"
+        });
+        ProtocolTest.log("FAIL: should not be able to specify non-Function `setup` parameter.");
+    } catch (e) {
+        ProtocolTest.log("PASS: should not be able to specify non-Function `setup` parameter.");
+    }
+    try {
+        badArgsSuite.addTestCase({
+            name: "foo",
+            test: () => {},
+            setup: 123
+        });
+        ProtocolTest.log("FAIL: should not be able to specify non-Function `setup` parameter.");
+    } catch (e) {
+        ProtocolTest.log("PASS: should not be able to specify non-Function `setup` parameter.");
+    }
+    try {
+        badArgsSuite.addTestCase({
+            name: "foo",
+            test: () => {},
+            setup: Promise.resolve()
+        });
+        ProtocolTest.log("FAIL: should not be able to specify non-Function `setup` parameter.");
+    } catch (e) {
+        ProtocolTest.log("PASS: should not be able to specify non-Function `setup` parameter.");
+    }
+    try {
+        badArgsSuite.addTestCase({
+            name: "foo",
+            test: () => {},
+            teardown: "astd"
+        });
+        ProtocolTest.log("FAIL: should not be able to specify non-Function `teardown` parameter.");
+    } catch (e) {
+        ProtocolTest.log("PASS: should not be able to specify non-Function `teardown` parameter.");
+    }
+    try {
+        badArgsSuite.addTestCase({
+            name: "foo",
+            test: () => {},
+            teardown: 123
+        });
+        ProtocolTest.log("FAIL: should not be able to specify non-Function `teardown` parameter.");
+    } catch (e) {
+        ProtocolTest.log("PASS: should not be able to specify non-Function `teardown` parameter.");
+    }
+    try {
+        badArgsSuite.addTestCase({
+            name: "foo",
+            test: () => {},
+            teardown: Promise.resolve()
+        });
+        ProtocolTest.log("FAIL: should not be able to specify non-Function `teardown` parameter.");
+    } catch (e) {
+        ProtocolTest.log("PASS: should not be able to specify non-Function `teardown` parameter.");
+    }
     let runEmptySuite = ProtocolTest.createSyncSuite("SyncTestSuite.RunEmptySuite");
     try {
         runEmptySuite.runTestCases();
@@ -154,6 +213,113 @@ function test()
     ProtocolTest.expectThat(abortOnFailureSuite.failCount === 1, "abortOnFailureSuite should have failed one test.");
     ProtocolTest.expectThat(abortOnFailureSuite.skipCount === 1, "abortOnFailureSuite should have skipped one test.");
 
+    let setupAndTeardownSymbol = Symbol("sync-suite-setup-and-teardown-token");
+    window[setupAndTeardownSymbol] = 0;
+
+    let setupAndTeardownTestSuite = ProtocolTest.createSyncSuite("SyncTestSuite.SetupAndTeardown");
+    setupAndTeardownTestSuite.addTestCase({
+        name: "TestWithSetupAndTeardown",
+        description: "Check execution order for setup and teardown actions.",
+        setup: () => {
+            window[setupAndTeardownSymbol] = 1;
+            return true;
+        },
+        test: () => {
+            ProtocolTest.expectThat(window[setupAndTeardownSymbol] === 1, "Test should see side effects of running setup() action.");
+            window[setupAndTeardownSymbol] = 2;
+            return true;
+        },
+        teardown: () => {
+            ProtocolTest.expectThat(window[setupAndTeardownSymbol] === 2, "Teardown should see side effects of running setup() action.");
+            window[setupAndTeardownSymbol] = 3;
+            return true;
+        }
+    });
+    setupAndTeardownTestSuite.addTestCase({
+        name: "TestRunningAfterTeardown",
+        description: "Check execution order for test after a teardown action.",
+        test: () => {
+            ProtocolTest.expectThat(window[setupAndTeardownSymbol] === 3, "Test should see side effects of previous test's teardown() action.");
+            return true;
+        },
+    });
+
+    setupAndTeardownTestSuite.runTestCases();
+
+    let setupExceptionTestSuite = ProtocolTest.createSyncSuite("SyncTestSuite.SetupException");
+    setupExceptionTestSuite.addTestCase({
+        name: "TestWithExceptionDuringSetup",
+        description: "Check execution order for setup action that throws an exception.",
+        setup: () => { throw new Error() },
+        test: () => {
+            ProtocolTest.assert(false, "Test should not execute if its setup action threw an exception.");
+            return false;
+        },
+        teardown: () => {
+            ProtocolTest.assert(false, "Teardown action should not execute if its setup action threw an exception.");
+            return false;           
+        }
+    });
+    setupExceptionTestSuite.runTestCases();
+
+    let setupFailureTestSuite = ProtocolTest.createSyncSuite("SyncTestSuite.SetupFailure");
+    setupFailureTestSuite.addTestCase({
+        name: "TestWithFailureDuringSetup",
+        description: "Check execution order for setup action that has failed.",
+        setup: () => false,
+        test: () => {
+            ProtocolTest.assert(false, "Test should not execute if its setup action returned false.")
+            return false;
+        },
+        teardown: () => {
+            ProtocolTest.assert(false, "Teardown action should not execute if its setup action returned false.")
+            return false;           
+        }
+    });
+    setupFailureTestSuite.runTestCases();
+
+    let teardownExceptionTestSuite = ProtocolTest.createSyncSuite("SyncTestSuite.TeardownException");
+    teardownExceptionTestSuite.addTestCase({
+        name: "TestWithExceptionDuringTeardown",
+        description: "Check execution order for teardown action that throws an exception.",
+        test: () => true,
+        teardown: () => { throw new Error() }
+    });
+    teardownExceptionTestSuite.addTestCase({
+        name: "TestAfterTeardownException",
+        descrption: "Check execution order for test after previous test's teardown throws an exception",
+        setup: () => {
+            ProtocolTest.assert(false, "Setup action should not execute if previous test's teardown action threw an exception.");
+            return false;
+        },
+        test: () => {
+            ProtocolTest.assert(false, "Test should not execute if previous test's teardown action threw an exception.");
+            return false;
+        }
+    });
+    teardownExceptionTestSuite.runTestCases();
+
+    let teardownFailureTestSuite = ProtocolTest.createSyncSuite("SyncTestSuite.TeardownFailure");
+    teardownFailureTestSuite.addTestCase({
+        name: "TestWithExceptionDuringTeardown",
+        description: "Check execution order for teardown action that has failed.",
+        test: () => true,
+        teardown: () => false,
+    });
+    teardownFailureTestSuite.addTestCase({
+        name: "TestAfterTeardownException",
+        descrption: "Check execution order for test after previous test's teardown throws an exception",
+        setup: () => {
+            ProtocolTest.assert(false, "Setup action should not execute if previous test's teardown action failed.");
+            return false;
+        },
+        test: () => {
+            ProtocolTest.assert(false, "Test should not execute if previous test's teardown action failed.");
+            return false;
+        }
+    });
+    teardownFailureTestSuite.runTestCases();
+
     ProtocolTest.completeTest();
 }
 </script>
index a36384e..e02bee2 100644 (file)
@@ -1,3 +1,24 @@
+2015-09-01  Brian Burg  <bburg@apple.com>
+
+        Web Inspector: add TestSuite hooks for per-testcase setup and teardown actions
+        https://bugs.webkit.org/show_bug.cgi?id=148641
+
+        Reviewed by Timothy Hatcher.
+
+        It should be possible to run setup and teardown actions for each test case.
+        The use case here is to add listeners, set up testcase-specific state, etc.
+        To match how test functions work in each suite, the sync and async implementations
+        take functions with no arguments and(resolve, reject), respectively.
+
+        * UserInterface/Test/TestSuite.js:
+        (TestSuite.prototype.addTestCase):
+        (TestSuite.messageFromThrownObject):
+        (TestSuite):
+        (AsyncTestSuite.prototype.runTestCases): Stick the extra functions in the chain.
+        (AsyncTestSuite):
+        (SyncTestSuite.prototype.runTestCases):
+        (SyncTestSuite):
+
 2015-09-01  Matt Baker  <mattbaker@apple.com>
 
         Web Inspector: Visual style editor shouldn't allow alpha characters in numeric input fields
index fdd531e..01d854e 100644 (file)
@@ -78,8 +78,26 @@ TestSuite = class TestSuite extends WebInspector.Object
         if (typeof testcase.test !== "function")
             throw new Error("Tried to add test case without `test` function.");
 
+        if (testcase.setup && typeof testcase.setup !== "function")
+            throw new Error("Tried to add test case with invalid `setup` parameter (must be a function).")
+
+        if (testcase.teardown && typeof testcase.teardown !== "function")
+            throw new Error("Tried to add test case with invalid `teardown` parameter (must be a function).")
+
         this.testcases.push(testcase);
     }
+
+    static messageFromThrownObject(e)
+    {
+        let message = e;
+        if (e instanceof Error)
+            message = e.message;
+
+        if (typeof message !== "string")
+            message = JSON.stringify(message);
+
+        return message;
+    }
 };
 
 AsyncTestSuite = class AsyncTestSuite extends TestSuite
@@ -103,32 +121,42 @@ AsyncTestSuite = class AsyncTestSuite extends TestSuite
         this._startedRunning = true;
 
         this._harness.log("");
-        this._harness.log("== Running test suite: " + this.name);
+        this._harness.log(`== Running test suite: ${this.name}`);
 
         // Avoid adding newlines if nothing was logged.
         let priorLogCount = this._harness.logCount;
         let result = this.testcases.reduce((chain, testcase, i) => {
-            return chain.then(() => {
+            if (testcase.setup) {
+                chain = chain.then(() => {
+                    this._harness.log("-- Running test setup.");
+                    return new Promise(testcase.setup);
+                });
+            }
+
+            chain = chain.then(() => {
                 if (i > 0 && priorLogCount + 1 < this._harness.logCount)
                     this._harness.log("");
 
                 priorLogCount = this._harness.logCount;
-                this._harness.log("-- Running test case: " + testcase.name);
+                this._harness.log(`-- Running test case: ${testcase.name}`);
                 this.runCount++;
                 return new Promise(testcase.test);
             });
+
+            if (testcase.teardown) {
+                chain = chain.then(() => {
+                    this._harness.log("-- Running test teardown.");
+                    return new Promise(testcase.teardown);
+                });
+            }
+            return chain;
         }, Promise.resolve());
 
         return result.catch((e) => {
             this.failCount++;
-            let message = e;
-            if (e instanceof Error)
-                message = e.message;
+            let message = TestSuite.messageFromThrownObject(e);
+            this._harness.log(`!! EXCEPTION: ${message}`);
 
-            if (typeof message !== "string")
-                message = JSON.stringify(message);
-
-            this._harness.log("!! EXCEPTION: " + message);
             throw e; // Reject this promise by re-throwing the error.
         });
     }
@@ -152,7 +180,7 @@ SyncTestSuite = class SyncTestSuite extends TestSuite
         this._startedRunning = true;
 
         this._harness.log("");
-        this._harness.log("== Running test suite: " + this.name);
+        this._harness.log(`== Running test suite: ${this.name}`);
 
         let priorLogCount = this._harness.logCount;
         for (let i = 0; i < this.testcases.length; i++) {
@@ -162,6 +190,22 @@ SyncTestSuite = class SyncTestSuite extends TestSuite
 
             priorLogCount = this._harness.logCount;
 
+            // Run the setup action, if one was provided.
+            if (testcase.setup) {
+                this._harness.log("-- Running test setup.");
+                try {
+                    let result = testcase.setup.call(null);
+                    if (result === false) {
+                        this._harness.log("!! EXCEPTION");
+                        return false;
+                    }
+                } catch (e) {
+                    let message = TestSuite.messageFromThrownObject(e);
+                    this._harness.log(`!! EXCEPTION: ${message}`);
+                    return false;
+                }
+            }
+
             this._harness.log("-- Running test case: " + testcase.name);
             this.runCount++;
             try {
@@ -172,18 +216,26 @@ SyncTestSuite = class SyncTestSuite extends TestSuite
                 }
             } catch (e) {
                 this.failCount++;
-                let message = e;
-                if (e instanceof Error)
-                    message = e.message;
-                else
-                    e = new Error(e);
-
-                if (typeof message !== "string")
-                    message = JSON.stringify(message);
-
-                this._harness.log("!! EXCEPTION: " + message);
+                let message = TestSuite.messageFromThrownObject(e);
+                this._harness.log(`!! EXCEPTION: ${message}`);
                 return false;
             }
+
+            // Run the teardown action, if one was provided.
+            if (testcase.teardown) {
+                this._harness.log("-- Running test teardown.");
+                try {
+                    let result = testcase.teardown.call(null);
+                    if (result === false) {
+                        this._harness.log("!! EXCEPTION:");
+                        return false;
+                    }
+                } catch (e) {
+                    let message = TestSuite.messageFromThrownObject(e);
+                    this._harness.log(`!! EXCEPTION: ${message}`);
+                    return false;
+                }
+            }
         }
 
         return true;