AX: Audit Tab should have an Audit Manager
authorcommit-queue@webkit.org <commit-queue@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 16 Jul 2018 18:57:39 +0000 (18:57 +0000)
committercommit-queue@webkit.org <commit-queue@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 16 Jul 2018 18:57:39 +0000 (18:57 +0000)
https://bugs.webkit.org/show_bug.cgi?id=184071
<rdar://problem/38946364>

Patch by Aaron Chu <aaron_chu@apple.com> on 2018-07-16
Reviewed by Brian Burg.

Source/WebInspectorUI:

This implements the AuditManager for the audit feature. This patch revolves
around building out an AuditManager that facilitates an audit. The AuditManager
is responsible for managing and storing AuditReports and AuditTestSuites. It is
also tasked to decide how to run a test -- whether as a test case or as a test
suite. This patch also includes 4 models with which the AuditManager works to
perform an audit and to generate a report. These models include AuditTestCase,
which as a collection is stored inside an AuditTestSuite; and AuditResult,
which, as a collection is stored inside an AuditReport.

* UserInterface/Controllers/AuditManager.js: Added.
(WI.AuditManager):
(WI.AuditManager.prototype.get testSuites):
(WI.AuditManager.prototype.get reports):
(WI.AuditManager.prototype.async.runAuditTestByRepresentedObject):
(WI.AuditManager.prototype.reportForId):
(WI.AuditManager.prototype.removeAllReports):
(WI.AuditManager.prototype.async._runTestCase):
* UserInterface/Main.html:
* UserInterface/Models/AuditReport.js: Added.
(WI.AuditReport):
(WI.AuditReport.prototype.get representedTestCases):
(WI.AuditReport.prototype.get representedTestSuite):
(WI.AuditReport.prototype.get resultsData):
(WI.AuditReport.prototype.get isWritable):
(WI.AuditReport.prototype.get failedCount):
(WI.AuditReport.prototype.addResult):
(WI.AuditReport.prototype.close):
* UserInterface/Models/AuditResult.js: Added.
(WI.AuditResult):
(WI.AuditResult.prototype.get testResult):
(WI.AuditResult.prototype.get name):
(WI.AuditResult.prototype.get logLevel):
(WI.AuditResult.prototype.get failed):
* UserInterface/Models/AuditTestCase.js: Added.
(WI.AuditTestCase.prototype.get id):
(WI.AuditTestCase.prototype.get name):
(WI.AuditTestCase.prototype.get suite):
(WI.AuditTestCase.prototype.get test):
(WI.AuditTestCase.prototype.get setup):
(WI.AuditTestCase.prototype.get tearDown):
(WI.AuditTestCase.prototype.get errorDetails):
(WI.AuditTestCase):
* UserInterface/Models/AuditTestSuite.js: Added.
(WI.AuditTestSuite):
(WI.AuditTestSuite.testCaseDescriptors):
(WI.AuditTestSuite.prototype.get id):
(WI.AuditTestSuite.prototype.get name):
(WI.AuditTestSuite.prototype.get testCases):
(WI.AuditTestSuite.prototype._buildTestCasesFromDescriptors):
* UserInterface/Test.html:

LayoutTests:

Test cases for AuditManager, AuditTestCase, AuditTestSuite, AuditResult and AuditReport.

* inspector/audit/audit-manager-expected.txt: Added.
* inspector/audit/audit-manager.html: Added.
* inspector/audit/audit-report-expected.txt: Added.
* inspector/audit/audit-report.html: Added.
* inspector/audit/audit-test-case-expected.txt: Added.
* inspector/audit/audit-test-case.html: Added.
* inspector/audit/audit-test-suite-expected.txt: Added.
* inspector/audit/audit-test-suite.html: Added.
* inspector/audit/resources/audit-test-fixtures.js: Added.
(TestPage.registerInitializer.window.testSuiteFixture1):
(TestPage.registerInitializer.window.testSuiteFixture1.testCaseDescriptors):
(TestPage.registerInitializer.window.testSuiteFixture2):
(TestPage.registerInitializer.window.testSuiteFixture2.testCaseDescriptors):
(TestPage.registerInitializer):

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@233858 268f45cc-cd09-0410-ab3c-d52691b4dbfc

18 files changed:
LayoutTests/ChangeLog
LayoutTests/inspector/audit/audit-manager-expected.txt [new file with mode: 0644]
LayoutTests/inspector/audit/audit-manager.html [new file with mode: 0644]
LayoutTests/inspector/audit/audit-report-expected.txt [new file with mode: 0644]
LayoutTests/inspector/audit/audit-report.html [new file with mode: 0644]
LayoutTests/inspector/audit/audit-test-case-expected.txt [new file with mode: 0644]
LayoutTests/inspector/audit/audit-test-case.html [new file with mode: 0644]
LayoutTests/inspector/audit/audit-test-suite-expected.txt [new file with mode: 0644]
LayoutTests/inspector/audit/audit-test-suite.html [new file with mode: 0644]
LayoutTests/inspector/audit/resources/audit-test-fixtures.js [new file with mode: 0644]
Source/WebInspectorUI/ChangeLog
Source/WebInspectorUI/UserInterface/Controllers/AuditManager.js [new file with mode: 0644]
Source/WebInspectorUI/UserInterface/Main.html
Source/WebInspectorUI/UserInterface/Models/AuditReport.js [new file with mode: 0644]
Source/WebInspectorUI/UserInterface/Models/AuditResult.js [new file with mode: 0644]
Source/WebInspectorUI/UserInterface/Models/AuditTestCase.js [new file with mode: 0644]
Source/WebInspectorUI/UserInterface/Models/AuditTestSuite.js [new file with mode: 0644]
Source/WebInspectorUI/UserInterface/Test.html

index 27a9730..88a5bb4 100644 (file)
@@ -1,3 +1,28 @@
+2018-07-16  Aaron Chu  <aaron_chu@apple.com>
+
+        AX: Audit Tab should have an Audit Manager
+        https://bugs.webkit.org/show_bug.cgi?id=184071
+        <rdar://problem/38946364>
+
+        Reviewed by Brian Burg.
+
+        Test cases for AuditManager, AuditTestCase, AuditTestSuite, AuditResult and AuditReport.
+
+        * inspector/audit/audit-manager-expected.txt: Added.
+        * inspector/audit/audit-manager.html: Added.
+        * inspector/audit/audit-report-expected.txt: Added.
+        * inspector/audit/audit-report.html: Added.
+        * inspector/audit/audit-test-case-expected.txt: Added.
+        * inspector/audit/audit-test-case.html: Added.
+        * inspector/audit/audit-test-suite-expected.txt: Added.
+        * inspector/audit/audit-test-suite.html: Added.
+        * inspector/audit/resources/audit-test-fixtures.js: Added.
+        (TestPage.registerInitializer.window.testSuiteFixture1):
+        (TestPage.registerInitializer.window.testSuiteFixture1.testCaseDescriptors):
+        (TestPage.registerInitializer.window.testSuiteFixture2):
+        (TestPage.registerInitializer.window.testSuiteFixture2.testCaseDescriptors):
+        (TestPage.registerInitializer):
+
 2018-07-16  Truitt Savell  <tsavell@apple.com>
 
         [ iOS ] Layout Test fast/forms/submit-change-fragment.html is a flaky Timeout
diff --git a/LayoutTests/inspector/audit/audit-manager-expected.txt b/LayoutTests/inspector/audit/audit-manager-expected.txt
new file mode 100644 (file)
index 0000000..a8ef603
--- /dev/null
@@ -0,0 +1,41 @@
+Test for the AuditManager Instantiation.
+
+
+== Running test suite: AuditManager
+-- Running test case: Adding an AuditTestSuite
+PASS: AuditManager should have 0 testSuite.
+Adding an AuditTestSuite to AuditManager.
+PASS: AuditManager should have 1 test suite.
+PASS: New test suite has the correct name.
+PASS: New test suite is of AuditTestSuite.
+
+-- Running test case: Adding a duplicating AuditTestSuite
+PASS: Should produce an exception.
+Error: class testSuiteFixture1 already exists.
+
+-- Running test case: Perform tests by AuditTestSuite.
+PASS: Receive a report that is of instance AuditReport.
+PASS: AuditReport is not writable
+PASS: There are two results in AuditReport.
+PASS: auditResults 0  is an instance of AuditResult.
+PASS: auditReport 0 is expected for test case fakeTest1.
+PASS: auditResults 1  is an instance of AuditResult.
+PASS: auditReport 1 is expected for test case fakeTest2.
+AuditReport is not writable.
+Attempting to add another AuditResult to AuditReport.
+PASS: AuditReport no longer accepts new AuditResults.
+PASS: Report represents the expected AuditTestSuite.
+
+-- Running test case: Perform a test by AuditTestCase.
+PASS: Receive a report that is of instance AuditReport.
+PASS: AuditReport represents the expected AuditTestCase.
+
+-- Running test case: AuditReports are unique.
+Only the most recent AuditReport for a case/suite is retained.
+PASS: The report represents the correct AuditTestSuite.
+
+-- Running test case: Get AuditReport by AuditTestCase/Suite id.
+Running a test for an AuditTestSuite and an AuditTestCase.
+PASS: The report represents the correct AuditTestSuite.
+PASS: The report represents the correct AuditTestCase.
+
diff --git a/LayoutTests/inspector/audit/audit-manager.html b/LayoutTests/inspector/audit/audit-manager.html
new file mode 100644 (file)
index 0000000..5f07540
--- /dev/null
@@ -0,0 +1,139 @@
+<!doctype html>
+<html>
+<head>
+<script src="../../http/tests/inspector/resources/inspector-test.js"></script>
+<script src="./resources/audit-test-fixtures.js"></script>
+<script>
+function test()
+{
+    let suite = InspectorTest.createAsyncSuite("AuditManager");
+
+    suite.addTestCase({
+        name: "Adding an AuditTestSuite",
+        description: "AuditManager should have one instantiated AuditTestSuite.",
+        async test(){
+            let auditManager = new WI.AuditManager;
+
+            InspectorTest.expectThat(!auditManager.testSuites.length, "AuditManager should have 0 testSuite.");
+
+            InspectorTest.log("Adding an AuditTestSuite to AuditManager.");
+            auditManager.addTestSuite(testSuiteFixture1);
+            let testSuite = auditManager.testSuites[0];
+
+            InspectorTest.expectEqual(auditManager.testSuites.length, 1, "AuditManager should have 1 test suite.");
+            InspectorTest.expectEqual(testSuite.name, "FakeTestSuite1", "New test suite has the correct name.");
+            InspectorTest.expectThat(testSuite instanceof WI.AuditTestSuite, "New test suite is of AuditTestSuite.");
+        }
+    });
+
+    suite.addTestCase({
+        name: "Adding a duplicating AuditTestSuite",
+        description: "Should throw exception for duplicated test suite.",
+        async test() {
+            let auditManager = new WI.AuditManager;
+
+            auditManager.addTestSuite(testSuiteFixture1);
+            auditManager.addTestSuite(testSuiteFixture2);
+
+            InspectorTest.expectException(() => {
+               auditManager.addTestSuite(testSuiteFixture1);
+            });
+        }
+    });
+
+    suite.addTestCase({
+        name: "Perform tests by AuditTestSuite.",
+        description: "Should produce report for AuditTestSuite.",
+        async test() {
+            let auditManager = new WI.AuditManager;
+
+            InspectorTest.assert(!auditManager._reports.size, "auditManager has no reports.");
+
+            let testSuite = new testSuiteFixture1;
+            let testCaseNames = testSuite.testCases.map(testCase => {
+                return testCase.name;
+            });
+            
+            await auditManager.runAuditTestByRepresentedObject(testSuite);
+
+            let auditReport = auditManager.reports[0];
+
+            InspectorTest.expectThat(auditReport instanceof WI.AuditReport, "Receive a report that is of instance AuditReport."); 
+            InspectorTest.expectThat(!auditReport._isWritable, "AuditReport is not writable");
+            InspectorTest.expectEqual(auditReport.resultsData.length, 2, "There are two results in AuditReport.");
+            
+            for (let i = 0; i < auditReport.resultsData.length; i++) {
+                let resultToTest = auditReport.resultsData[i];
+                InspectorTest.expectThat(resultToTest instanceof WI.AuditResult, `auditResults ${i}  is an instance of AuditResult.`)
+                InspectorTest.expectThat(testCaseNames.indexOf(resultToTest.name) >= 0, `auditReport ${i} is expected for test case ${resultToTest.name}.`);
+            }
+            InspectorTest.log("AuditReport is not writable.");
+            InspectorTest.log("Attempting to add another AuditResult to AuditReport.");
+            let additionalReport = auditReport.resultsData[1];
+            auditReport.addResult(additionalReport);
+
+            InspectorTest.expectEqual(auditReport.resultsData.length, 2, "AuditReport no longer accepts new AuditResults.");
+            InspectorTest.expectEqual(auditReport.representedTestSuite.id, testSuite.id, "Report represents the expected AuditTestSuite.");
+        }
+    });
+
+
+    suite.addTestCase({
+        name: "Perform a test by AuditTestCase.",
+        description: "Should produce report for AuditTestCase.",
+        async test() {
+            let auditManager = new WI.AuditManager;
+            auditManager.addTestSuite(testSuiteFixture1);
+            let testCase = auditManager.testSuites[0].testCases[0];
+
+            await auditManager.runAuditTestByRepresentedObject(testCase);
+
+            let auditReport = auditManager.reports[0];
+
+            InspectorTest.expectThat(auditReport instanceof WI.AuditReport, "Receive a report that is of instance AuditReport.");
+            InspectorTest.expectEqual(auditReport.representedTestCases[0], testCase, "AuditReport represents the expected AuditTestCase.");
+        }
+    });
+
+    suite.addTestCase({
+        name: "AuditReports are unique.",
+        description: "No AuditReport should represent the same AuditTestCase/Suite",
+        async test() {
+            let auditManager = new WI.AuditManager;
+            let testSuite = new testSuiteFixture1;
+
+            InspectorTest.log("Only the most recent AuditReport for a case/suite is retained.");
+
+            let results = [await auditManager.runAuditTestByRepresentedObject(testSuite), await auditManager.runAuditTestByRepresentedObject(testSuite)];
+
+            InspectorTest.expectEqual(results[0].representedTestSuite, results[1].representedTestSuite, "The report represents the correct AuditTestSuite.");
+        }
+    });
+
+    suite.addTestCase({
+        name: "Get AuditReport by AuditTestCase/Suite id.",
+        description: "Should return the correct AuditReport.",
+        async test() {
+            let auditManager = new WI.AuditManager;
+            let testSuite = new testSuiteFixture1;
+            let testCase = testSuite.testCases[0];
+
+            InspectorTest.log("Running a test for an AuditTestSuite and an AuditTestCase.");
+            let results = [await auditManager.runAuditTestByRepresentedObject(testSuite), await auditManager.runAuditTestByRepresentedObject(testCase)];
+
+            let auditReportForTestSuite = auditManager.reportForId(testSuite.id);
+            let auditReportForTestCase = auditManager.reportForId(testCase.id);
+
+            InspectorTest.expectEqual(results[0], auditReportForTestSuite, "The report represents the correct AuditTestSuite.");
+            InspectorTest.expectEqual(results[1], auditReportForTestCase, "The report represents the correct AuditTestCase.");
+        }
+    });
+
+    suite.runTestCasesAndFinish();
+}
+</script>
+</head>
+<body onload="runTest()">
+<p>Test for the AuditManager Instantiation.</p>
+</body>
+</html>
diff --git a/LayoutTests/inspector/audit/audit-report-expected.txt b/LayoutTests/inspector/audit/audit-report-expected.txt
new file mode 100644 (file)
index 0000000..6369257
--- /dev/null
@@ -0,0 +1,7 @@
+Test for the AuditManager Instantiation.
+
+
+== Running test suite: AuditReport
+-- Running test case: Instantiation with test suite
+PASS: Instantiate AuditReport with AuditTestSuite.
+
diff --git a/LayoutTests/inspector/audit/audit-report.html b/LayoutTests/inspector/audit/audit-report.html
new file mode 100644 (file)
index 0000000..db122bd
--- /dev/null
@@ -0,0 +1,29 @@
+<!doctype html>
+<html>
+<head>
+<script src="../../http/tests/inspector/resources/inspector-test.js"></script>
+<script src="resources/audit-test-fixtures.js"></script>
+<script>
+function test()
+{
+    let suite = InspectorTest.createAsyncSuite("AuditReport");
+
+    suite.addTestCase({
+        name: "Instantiation with test suite",
+        description: "should instantiate correctly.",
+        async test() {
+            let testSuite = new testSuiteFixture1;
+            InspectorTest.assert(testSuite instanceof WI.AuditTestSuite, "testSuite is AuditTestSuite.");
+            let report = new WI.AuditReport(testSuite);
+            InspectorTest.expectThat(report instanceof WI.AuditReport, "Instantiate AuditReport with AuditTestSuite.");
+        }
+    });
+
+    suite.runTestCasesAndFinish();
+}
+</script>
+</head>
+<body onload="runTest()">
+<p>Test for the AuditManager Instantiation.</p>
+</body>
+</html>
diff --git a/LayoutTests/inspector/audit/audit-test-case-expected.txt b/LayoutTests/inspector/audit/audit-test-case-expected.txt
new file mode 100644 (file)
index 0000000..9215ac4
--- /dev/null
@@ -0,0 +1,8 @@
+Test for the AudtTestCase.
+
+
+== Running test suite: AuditTestCase
+-- Running test case: Test functions must be asynchronous.
+PASS: Should produce an exception.
+Error: Test functions must be async functions.
+
diff --git a/LayoutTests/inspector/audit/audit-test-case.html b/LayoutTests/inspector/audit/audit-test-case.html
new file mode 100644 (file)
index 0000000..456d108
--- /dev/null
@@ -0,0 +1,28 @@
+<!doctype html>
+<html>
+<head>
+<script src="../../http/tests/inspector/resources/inspector-test.js"></script>
+<script src="./resources/audit-test-fixtures.js"></script>
+<script>
+function test()
+{
+    let suite = InspectorTest.createAsyncSuite("AuditTestCase");
+
+    suite.addTestCase({
+        name: "Test functions must be asynchronous.",
+        description: "AuditTestCase should throw an exception when instantiated with a non-async function.",
+        async test() {
+            InspectorTest.expectException(() => {
+                new WI.AuditTestCase(new testSuiteFixture1, "fakeTest2", () => []);
+            });
+        }
+    })
+
+    suite.runTestCasesAndFinish();
+}
+</script>
+</head>
+<body onload="runTest()">
+<p>Test for the AudtTestCase.</p>
+</body>
+</html>
diff --git a/LayoutTests/inspector/audit/audit-test-suite-expected.txt b/LayoutTests/inspector/audit/audit-test-suite-expected.txt
new file mode 100644 (file)
index 0000000..8686f82
--- /dev/null
@@ -0,0 +1,19 @@
+Test for the AuditTestSuite.
+
+
+== Running test suite: AuditTestSuite
+-- Running test case: AuditTestSuite Id
+PASS: AuditTestSuite1 has ID with correct type.
+PASS: AuditTestSuite2 has ID with correct type.
+PASS: AuditTestSuites with same name have different unique IDs.
+
+-- Running test case: AuditTestSuite testCaseCount
+PASS: There are two tests.
+
+-- Running test case: AuditTestSuite should run tests sequentially.
+PASS: First test is ran.
+PASS: Second test is ran.
+PASS: Third test is ran.
+PASS: Fourth test is ran.
+PASS: Last test is ran.
+
diff --git a/LayoutTests/inspector/audit/audit-test-suite.html b/LayoutTests/inspector/audit/audit-test-suite.html
new file mode 100644 (file)
index 0000000..94c7f91
--- /dev/null
@@ -0,0 +1,106 @@
+<!doctype html>
+<html>
+<head>
+<script src="../../http/tests/inspector/resources/inspector-test.js"></script>
+<script src="./resources/audit-test-fixtures.js"></script>
+<script>
+function test()
+{
+
+    let suite = InspectorTest.createAsyncSuite("AuditTestSuite");
+
+    let auditTestSuite1 = new testSuiteFixture1("FakeTestSuite", "FakeTestSuite");
+    let auditTestSuite2 = new testSuiteFixture1("FakeTestSuite", "FakeTestSuite");
+
+    suite.addTestCase({
+        name: "AuditTestSuite Id",
+        description: "Should exist and be unique",
+        async test() {
+            InspectorTest.expectEqual(typeof(auditTestSuite1.id), "symbol", "AuditTestSuite1 has ID with correct type.");
+            InspectorTest.expectEqual(typeof(auditTestSuite2.id), "symbol", "AuditTestSuite2 has ID with correct type.");
+            InspectorTest.expectThat(auditTestSuite1.id !== auditTestSuite2.id, "AuditTestSuites with same name have different unique IDs.");
+        }
+    });
+
+    suite.addTestCase({
+        name: "AuditTestSuite testCaseCount",
+        description: "Should represents correct number of test case.",
+        async test() {
+            InspectorTest.expectEqual(auditTestSuite1.testCases.length, 2, "There are two tests.");
+        }
+    });
+
+
+
+    suite.addTestCase({
+        name: "AuditTestSuite should run tests sequentially.",
+        description: "Tests should be ran in the order that was defined in the testDescriptor.",
+        async test() {
+                
+            let testOrderSymbol = Symbol("test-order");
+            window[testOrderSymbol] = 0;
+
+            let TestOrderSuite = class TestOrderSuite extends WI.AuditTestSuite 
+            {
+                static testCaseDescriptors()
+                {
+                    return [
+                        {
+                            name: "test 1",
+                            description: "order 1",
+                            async test() {
+                                InspectorTest.expectEqual(window[testOrderSymbol], 0, "First test is ran.");
+                                window[testOrderSymbol] = 1;
+                            }
+                        },
+                        {
+                            name: "test 2",
+                            description: "order 2",
+                            async test() {
+                                InspectorTest.expectEqual(window[testOrderSymbol], 1, "Second test is ran.");
+                                window[testOrderSymbol] = 2;
+                            }
+                        },
+                        {
+                            name: "test 3",
+                            description: "order 3",
+                            async test() {
+                                InspectorTest.expectEqual(window[testOrderSymbol], 2, "Third test is ran.");
+                                window[testOrderSymbol] = 3;
+                            }
+                        },
+                        {
+                            name: "test 4",
+                            description: "order 4",
+                            async test() {
+                                InspectorTest.expectEqual(window[testOrderSymbol], 3, "Fourth test is ran.");
+                                window[testOrderSymbol] = 4;
+                            }
+                        },
+                        {
+                            name: "test 4",
+                            description: "order 4",
+                            async test() {
+                                InspectorTest.expectEqual(window[testOrderSymbol], 4, "Last test is ran.");
+                            }
+                        }
+                    ]
+                }
+            }
+
+            let auditManager = new WI.AuditManager;
+            auditManager.addTestSuite(TestOrderSuite);
+
+            let result = auditManager.runAuditTestByRepresentedObject(auditManager.testSuites[0]);
+        }
+    });
+
+    suite.runTestCasesAndFinish();
+}
+</script>
+</head>
+<body onload="runTest()">
+<p>Test for the AuditTestSuite.</p>
+</body>
+</html>
diff --git a/LayoutTests/inspector/audit/resources/audit-test-fixtures.js b/LayoutTests/inspector/audit/resources/audit-test-fixtures.js
new file mode 100644 (file)
index 0000000..c0c0535
--- /dev/null
@@ -0,0 +1,49 @@
+TestPage.registerInitializer(() => {
+
+    window.testSuiteFixture1 = class testSuiteFixture1 extends WI.AuditTestSuite
+    {
+        constructor()
+        {
+            super("FakeTestSuite1", "FakeTestSuite1");
+        }
+
+        static testCaseDescriptors()
+        {
+            return [
+                {
+                    name: "fakeTest1",
+                    async test(){}
+                },
+                {
+                    name: "fakeTest2",
+                    async test()
+                    {
+                        throw new Error([1, 2, 3, 4]);
+                    }
+                }
+            ];
+        }
+    }
+
+    window.testSuiteFixture2 = class testSuiteFixture2 extends WI.AuditTestSuite
+    {
+        constructor()
+        {
+            super("FakeTestSuite2", "FakeTestSuite2");
+        }
+
+        static testCaseDescriptors()
+        {
+            return [
+                {
+                    name: "fakeTest2",
+                    async test()
+                    {
+                        return [];
+                    }
+                }
+            ];
+        }
+    }
+  
+});
index ab565a9..3c3d36a 100644 (file)
@@ -1,3 +1,62 @@
+2018-07-16  Aaron Chu  <aaron_chu@apple.com>
+
+        AX: Audit Tab should have an Audit Manager
+        https://bugs.webkit.org/show_bug.cgi?id=184071
+        <rdar://problem/38946364>
+
+        Reviewed by Brian Burg.
+
+        This implements the AuditManager for the audit feature. This patch revolves
+        around building out an AuditManager that facilitates an audit. The AuditManager
+        is responsible for managing and storing AuditReports and AuditTestSuites. It is
+        also tasked to decide how to run a test -- whether as a test case or as a test
+        suite. This patch also includes 4 models with which the AuditManager works to
+        perform an audit and to generate a report. These models include AuditTestCase,
+        which as a collection is stored inside an AuditTestSuite; and AuditResult,
+        which, as a collection is stored inside an AuditReport.
+
+        * UserInterface/Controllers/AuditManager.js: Added.
+        (WI.AuditManager):
+        (WI.AuditManager.prototype.get testSuites):
+        (WI.AuditManager.prototype.get reports):
+        (WI.AuditManager.prototype.async.runAuditTestByRepresentedObject):
+        (WI.AuditManager.prototype.reportForId):
+        (WI.AuditManager.prototype.removeAllReports):
+        (WI.AuditManager.prototype.async._runTestCase):
+        * UserInterface/Main.html:
+        * UserInterface/Models/AuditReport.js: Added.
+        (WI.AuditReport):
+        (WI.AuditReport.prototype.get representedTestCases):
+        (WI.AuditReport.prototype.get representedTestSuite):
+        (WI.AuditReport.prototype.get resultsData):
+        (WI.AuditReport.prototype.get isWritable):
+        (WI.AuditReport.prototype.get failedCount):
+        (WI.AuditReport.prototype.addResult):
+        (WI.AuditReport.prototype.close):
+        * UserInterface/Models/AuditResult.js: Added.
+        (WI.AuditResult):
+        (WI.AuditResult.prototype.get testResult):
+        (WI.AuditResult.prototype.get name):
+        (WI.AuditResult.prototype.get logLevel):
+        (WI.AuditResult.prototype.get failed):
+        * UserInterface/Models/AuditTestCase.js: Added.
+        (WI.AuditTestCase.prototype.get id):
+        (WI.AuditTestCase.prototype.get name):
+        (WI.AuditTestCase.prototype.get suite):
+        (WI.AuditTestCase.prototype.get test):
+        (WI.AuditTestCase.prototype.get setup):
+        (WI.AuditTestCase.prototype.get tearDown):
+        (WI.AuditTestCase.prototype.get errorDetails):
+        (WI.AuditTestCase):
+        * UserInterface/Models/AuditTestSuite.js: Added.
+        (WI.AuditTestSuite):
+        (WI.AuditTestSuite.testCaseDescriptors):
+        (WI.AuditTestSuite.prototype.get id):
+        (WI.AuditTestSuite.prototype.get name):
+        (WI.AuditTestSuite.prototype.get testCases):
+        (WI.AuditTestSuite.prototype._buildTestCasesFromDescriptors):
+        * UserInterface/Test.html:
+
 2018-07-16  Nikita Vasilyev  <nvasilyev@apple.com>
 
         Web Inspector: Dark Mode: Console filter field buttons should be darker
diff --git a/Source/WebInspectorUI/UserInterface/Controllers/AuditManager.js b/Source/WebInspectorUI/UserInterface/Controllers/AuditManager.js
new file mode 100644 (file)
index 0000000..52f68a0
--- /dev/null
@@ -0,0 +1,139 @@
+/*
+ * Copyright (C) 2018 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+ WI.AuditManager = class AuditManager extends WI.Object
+{
+    constructor()
+    {
+        super();
+
+        this._testSuiteConstructors = [];
+        this._reports = new Map;
+        
+        // Transforming all the constructors into AuditTestSuite instances.
+        this._testSuites = this._testSuiteConstructors.map(suite =>  {
+            let newTestSuite = new suite;
+
+            if (!newTestSuite instanceof WI.AuditTestSuite)
+                throw new Error("Audit test suites must be of instance WI.AuditTestSuite.");
+
+            return newTestSuite;
+        });
+    }
+
+    // Public
+
+    get testSuites() { return this._testSuites.slice(); }
+    get reports() { return [...this._reports.values()]; }
+
+    async runAuditTestByRepresentedObject(representedObject)
+    {
+        let auditReport = new WI.AuditReport(representedObject);
+
+        if (representedObject instanceof WI.AuditTestCase) {
+            let auditResult = await this._runTestCase(representedObject);
+            auditReport.addResult(auditResult);
+        } else if (representedObject instanceof WI.AuditTestSuite) {
+            let testCases = representedObject.testCases;            
+            // Start reducing from testCases[0].
+            let result = testCases.slice(1).reduce((chain, testCase, index) => {
+                if (testCase.setup) {
+                    let setup = testCase.setup.call(testCase, testCase.suite)
+                    if (testCase.setup[Symbol.toStringTag] === "AsyncFunction")
+                        return setup;
+                    else
+                        return new Promise(setup);
+                }
+
+                chain = chain.then((auditResult) => {
+                    auditReport.addResult(auditResult);
+                    return this._runTestCase(testCase);
+                });
+
+                if (testCase.tearDown) {
+                    let tearDown = testCase.tearDown.call(testCase, testCase.suite)
+                    if (testCase.tearDown[Symbol.toStringTag] === "AsyncFunction")
+                        return tearDown;
+                    else
+                        return new Promise(tearDown);
+                }
+                return chain;
+            }, this._runTestCase(testCases[0]));
+
+            let lastAuditResult = await result;
+            auditReport.addResult(lastAuditResult);
+
+            // Make AuditReport read-only after all the AuditResults have been received.
+            auditReport.close();
+        }
+
+        this._reports.set(representedObject.id, auditReport);
+        this.dispatchEventToListeners(WI.AuditManager.Event.NewReportAdded, {auditReport});
+
+        return auditReport;
+    }
+
+    addTestSuite(auditTestSuiteConstructor)
+    {
+        if (this._testSuiteConstructors.indexOf(auditTestSuiteConstructor) >= 0)
+            throw new Error(`class ${auditTestSuiteConstructor.name} already exists.`);
+
+        let auditTestSuite = new auditTestSuiteConstructor;
+        this._testSuiteConstructors.push(auditTestSuiteConstructor);
+        this._testSuites.push(auditTestSuite);
+    }
+
+    reportForId(reportId)
+    {
+        return this._reports.get(reportId);
+    }
+
+    removeAllReports()
+    {
+        this._reports.clear();
+    }
+
+    // Private
+
+    async _runTestCase(testCase)
+    {
+        let didRaiseException = false;
+        let result;
+        this.dispatchEventToListeners(WI.AuditManager.Event.TestStarted, {test: testCase});
+        try {
+            result = await testCase.test.call(testCase, testCase.suite);
+        } catch (resultData) {
+            result = resultData;
+            didRaiseException = true;
+        }
+        this.dispatchEventToListeners(WI.AuditManager.Event.TestEnded, {test: testCase});
+        return new WI.AuditResult(testCase, {result}, didRaiseException);
+    }
+}
+
+WI.AuditManager.Event = {
+    TestStarted: Symbol("test-started"),
+    TestEnded: Symbol("test-ended")
+}
index d6ae8ed..010fc94 100644 (file)
     <script src="Models/WrappedPromise.js"></script>
     <script src="Models/XHRBreakpoint.js"></script>
 
+    <script src="Models/AuditReport.js"></script>
+    <script src="Models/AuditResult.js"></script>
+    <script src="Models/AuditTestCase.js"></script>
+    <script src="Models/AuditTestSuite.js"></script>
+
     <script src="Proxies/FormatterWorkerProxy.js"></script>
     <script src="Proxies/HeapSnapshotDiffProxy.js"></script>
     <script src="Proxies/HeapSnapshotEdgeProxy.js"></script>
     <script src="Controllers/CodeMirrorEditingController.js"></script>
 
     <script src="Controllers/AnalyzerManager.js"></script>
+    <script src="Controllers/AuditManager.js"></script>
     <script src="Controllers/ApplicationCacheManager.js"></script>
     <script src="Controllers/BasicBlockAnnotator.js"></script>
     <script src="Controllers/BranchManager.js"></script>
diff --git a/Source/WebInspectorUI/UserInterface/Models/AuditReport.js b/Source/WebInspectorUI/UserInterface/Models/AuditReport.js
new file mode 100644 (file)
index 0000000..6cb8305
--- /dev/null
@@ -0,0 +1,62 @@
+/*
+ * Copyright (C) 2018 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+ WI.AuditReport = class AuditReport
+{
+    constructor(representedTest)
+    {
+        console.assert(representedTest instanceof WI.AuditTestCase || representedTest instanceof WI.AuditTestSuite);
+        
+        this._results = [];
+        this._isWritable = true;
+        this._representedTestCases = (representedTest instanceof WI.AuditTestCase) ? [representedTest] : [...representedTest.testCases];
+        this._representedTestSuite = (representedTest instanceof WI.AuditTestSuite) ? representedTest : null;
+    }
+
+    // Public
+
+    get representedTestCases() { return this._representedTestCases.slice(); }
+    get representedTestSuite() { return this._representedTestSuite; }
+    get resultsData() { return this._results.slice(); }
+    get isWritable() { return this._isWritable; }
+
+    get failedCount() {
+        return this._results.slice().filter(result => result.failed).length;
+    }
+
+    addResult(auditResult)
+    {
+        if (!this._isWritable)
+            return;
+
+        console.assert(auditResult instanceof WI.AuditResult);
+        this._results.push(auditResult);
+    }
+
+    close()
+    {
+        this._isWritable = false;
+    }
+}
diff --git a/Source/WebInspectorUI/UserInterface/Models/AuditResult.js b/Source/WebInspectorUI/UserInterface/Models/AuditResult.js
new file mode 100644 (file)
index 0000000..ee9ba1b
--- /dev/null
@@ -0,0 +1,52 @@
+/*
+ * Copyright (C) 2018 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+ WI.AuditResult = class AuditResult
+{
+    constructor(testInstance, testResult, failed)
+    {
+        this._testResult = testResult;
+        this._failed = failed || false;
+        this._testName = testInstance.name;
+        this._errorDetails = testInstance.errorDetails;
+        this._logLevel = this._errorDetails.logLevel || WI.AuditResult.LogLevel.Passed;
+        this._errorTitle = this._errorDetails.title;
+        this._hint = this._errorDetails.hint;
+        this._documentation = this._errorDetails.documentation;
+    }
+
+    // Public
+
+    get testResult() { return this._testResult; }
+    get name() { return this._testName; }
+    get logLevel() { return this._logLevel; }
+    get failed() { return this._failed; }
+}
+
+WI.AuditResult.LogLevel = {
+    Error: "error",
+    Warning: "warning",
+    Passed: "passed"
+}
diff --git a/Source/WebInspectorUI/UserInterface/Models/AuditTestCase.js b/Source/WebInspectorUI/UserInterface/Models/AuditTestCase.js
new file mode 100644 (file)
index 0000000..4547e24
--- /dev/null
@@ -0,0 +1,62 @@
+/*
+ * Copyright (C) 2018 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+ WI.AuditTestCase = class AuditTestCase extends WI.Object
+{
+    constructor(suite, name, test, setup, tearDown, errorDetails = {})
+    {
+        console.assert(suite instanceof WI.AuditTestSuite);
+        console.assert(typeof(name) === "string");
+        
+        if (setup)
+            console.assert(setup instanceof Function);
+
+        if (tearDown)
+            console.assert(tearDown instanceof Function);
+
+        if (test[Symbol.toStringTag] !== "AsyncFunction")
+            throw new Error("Test functions must be async functions.");
+
+        super();
+        this._id = Symbol(name);
+        
+        this._suite = suite;
+        this._name = name;
+        this._test = test;
+        this._setup = setup;
+        this._tearDown = tearDown;
+        this._errorDetails = errorDetails;
+    }
+
+    // Public
+
+    get id() { return this._id; }
+    get name() { return this._name; }
+    get suite() { return this._suite; }
+    get test() { return this._test; }
+    get setup() { return this._setup; }
+    get tearDown() { return this._tearDown; }
+    get errorDetails() { return this._errorDetails; }
+}
diff --git a/Source/WebInspectorUI/UserInterface/Models/AuditTestSuite.js b/Source/WebInspectorUI/UserInterface/Models/AuditTestSuite.js
new file mode 100644 (file)
index 0000000..f073ef4
--- /dev/null
@@ -0,0 +1,70 @@
+/*
+ * Copyright (C) 2018 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+ WI.AuditTestSuite = class AuditTestSuite extends WI.Object
+{
+    constructor(identifier, name)
+    {
+        super();
+        this._id = Symbol(identifier);
+        this._name = name;
+        this._testCases = new Map;
+
+        this._buildTestCasesFromDescriptors();
+    }
+
+    static testCaseDescriptors() { throw WI.NotImplementedError.subclassMustOverride(); }
+
+    // Public
+
+    get id() { return this._id; }
+    get name() { return this._name; }
+    get testCases() {
+        return [...this._testCases.values()];
+    }
+
+    // Private
+
+    _buildTestCasesFromDescriptors()
+    {
+        for (let descriptor of this.constructor.testCaseDescriptors()) {
+            if (typeof(descriptor.name) !== "string" || !descriptor.name)
+                throw new Error("Test name must be a valid string.");
+            
+            let {name, test, setup, tearDown, errorDetails} = descriptor;
+
+            if (!test instanceof Function || test[Symbol.toStringTag] !== "AsyncFunction")
+                throw new Error("Test function must be an async function.");
+
+            let testCaseInstance = new WI.AuditTestCase(this, name, test, setup, tearDown, errorDetails);
+
+            this._testCases.set(testCaseInstance.id, testCaseInstance);
+        }
+    }
+}
+
+WI.AuditTestSuite.Event = {
+    NewAuditResultAvailable: Symbol("new-audit-result-available")
+}   
index af4e581..0defd44 100644 (file)
     <script src="Models/WrappedPromise.js"></script>
     <script src="Models/XHRBreakpoint.js"></script>
 
+    <script src="Models/AuditReport.js"></script>
+    <script src="Models/AuditResult.js"></script>
+    <script src="Models/AuditTestCase.js"></script>
+    <script src="Models/AuditTestSuite.js"></script>
+
     <script src="Proxies/FormatterWorkerProxy.js"></script>
     <script src="Proxies/HeapSnapshotDiffProxy.js"></script>
     <script src="Proxies/HeapSnapshotEdgeProxy.js"></script>
     <script src="Proxies/HeapSnapshotProxy.js"></script>
     <script src="Proxies/HeapSnapshotWorkerProxy.js"></script>
 
+    <script src="Controllers/AuditManager.js"></script>
     <script src="Controllers/BreakpointLogMessageLexer.js"></script>
     <script src="Controllers/CSSStyleManager.js"></script>
     <script src="Controllers/CanvasManager.js"></script>