Added sending notification feature when test group finishes.
authordewei_zhu@apple.com <dewei_zhu@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 8 Jun 2018 03:44:28 +0000 (03:44 +0000)
committerdewei_zhu@apple.com <dewei_zhu@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 8 Jun 2018 03:44:28 +0000 (03:44 +0000)
https://bugs.webkit.org/show_bug.cgi?id=184340

Reviewed by Ryosuke Niwa.

Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group
has a pending notification.
Added 'testgroup_notification_sent_at' to record the last notification sent time.
SQL queries to update existing database are:
    'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;'
    'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;'
Updated 'run-analysis' script to be able to send notification when test group finishes.
Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group.

* browser-tests/test-group-form-tests.js: Updated existing tests and added a new test.
* browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage.
* init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table.
* public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all
test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its
'testgroup_needs_notification' is true will be returned by this API.
* public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids.
* public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification'
while create an analysis_test_groups.
* public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'.
* public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'.
* public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'.
Extended this API to allow authentication both from CSRF token and slave.
* public/v3/components/custom-configuration-test-group-form.js:
(CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents
'testgroup_needs_notification' from API perspective.
* public/v3/components/customizable-test-group-form.js:
(CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents
'testgroup_needs_notification' from API perspective.
(CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list.
* public/v3/components/test-group-form.js:
(TestGroupForm): Added '_notifyOnCompletion' instance variable.
(TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox.
(TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents
'testgroup_needs_notification' from API perspective.
(TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list.
* public/v3/models/analysis-results.js: Export 'AnalysisResults'.
(AnalysisResults.fetch): Update API path to use absolute url.
(AnalysisResults):
* public/v3/models/analysis-task.js:
(AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as
'needsNotification' to send to server.
(AnalysisTask):
* public/v3/models/test-group.js:
(TestGroup): Added '_needsNotification' field.
(TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field.
(TestGroup.prototype.needsNotification): Returns '_needsNotification' field.
(TestGroup.prototype.author): Returns author information.
(TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true.
(TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched.
(TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as
'needsNotification' to send to server.
(TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as
'needsNotification' to send to server.
(TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as
'needsNotification' to send to server.
(TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'.
* public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around
(AnalysisTaskChartPane.prototype.didConstructShadowTree):
(AnalysisTaskResultsPane.prototype.didConstructShadowTree):
(AnalysisTaskTestGroupPane.prototype.didConstructShadowTree):
(AnalysisTaskPage.prototype.didConstructShadowTree):
(AnalysisTaskPage.prototype._retryCurrentTestGroup):
(AnalysisTaskPage.prototype.async._bisectCurrentTestGroup):
(AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const):
(AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList):
(AnalysisTaskPage.prototype._createCustomTestGroup):
* public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox.
(ChartPane.prototype.didConstructShadowTree):
(ChartPane.prototype.async._analyzeRange):
* public/v3/pages/create-analysis-task-page.js: Adapted API change.
(CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup):
* server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'.
* server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change.
Added new tests.
* server-tests/privileged-api-create-test-group-tests.js: Added new tests.
* server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API.
* server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true.
* server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change.
(async.createTestGroupWihPatch):
(createTestGroupWihOwnedCommit):
* tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups.
(AnalysisResultsNotifier):
(AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups):
(AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification.
(AnalysisResultsNotifier.prototype._constructMessageByRules):
(AnalysisResultsNotifier._matchesRule):
(AnalysisResultsNotifier._applyUpdate):
(AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group.
(AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task.
(AnalysisResultsNotifier._instantiateNotificationTemplate):
* tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result.
(TestGroupResultPage):
(TestGroupResultPage.prototype.async.setTestGroup):
(TestGroupResultPage._urlForAnalysisTask):
(TestGroupResultPage.prototype._URLForAnalysisTask):
(TestGroupResultPage.prototype.constructTables):
(TestGroupResultPage.prototype._constructTableForMetric):
(TestGroupResultPage.):
(TestGroupResultPage.prototype.get pageContent):
(TestGroupResultPage.prototype.get styleTemplate):
(BarGraph):
(BarGraph.prototype.setWidth):
(BarGraph.prototype._constructBarGraph):
(BarGraph.prototype.get pageContent):
(BarGraph.prototype.get styleTemplate):
* tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change.
(MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet):
(MeasurementSetAnalyzer):
* tools/js/v3-models.js:
* tools/run-analysis.js: Added the logic that sends notification for completed test groups.
(main):
(async.analysisLoop):
* unit-tests/analysis-task-tests.js:
* unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'.
* unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change.
* unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'.
* unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232612 268f45cc-cd09-0410-ab3c-d52691b4dbfc

35 files changed:
Websites/perf.webkit.org/ChangeLog
Websites/perf.webkit.org/browser-tests/index.html
Websites/perf.webkit.org/browser-tests/test-group-form-tests.js
Websites/perf.webkit.org/browser-tests/test-group-result-page-tests.js [new file with mode: 0644]
Websites/perf.webkit.org/init-database.sql
Websites/perf.webkit.org/public/api/test-groups.php
Websites/perf.webkit.org/public/include/build-requests-fetcher.php
Websites/perf.webkit.org/public/include/commit-sets-helpers.php
Websites/perf.webkit.org/public/privileged-api/create-analysis-task.php
Websites/perf.webkit.org/public/privileged-api/create-test-group.php
Websites/perf.webkit.org/public/privileged-api/update-test-group.php
Websites/perf.webkit.org/public/v3/components/custom-configuration-test-group-form.js
Websites/perf.webkit.org/public/v3/components/customizable-test-group-form.js
Websites/perf.webkit.org/public/v3/components/test-group-form.js
Websites/perf.webkit.org/public/v3/models/analysis-results.js
Websites/perf.webkit.org/public/v3/models/analysis-task.js
Websites/perf.webkit.org/public/v3/models/test-group.js
Websites/perf.webkit.org/public/v3/pages/analysis-task-page.js
Websites/perf.webkit.org/public/v3/pages/chart-pane.js
Websites/perf.webkit.org/public/v3/pages/create-analysis-task-page.js
Websites/perf.webkit.org/server-tests/api-test-groups.js [new file with mode: 0644]
Websites/perf.webkit.org/server-tests/privileged-api-create-analysis-task-tests.js
Websites/perf.webkit.org/server-tests/privileged-api-create-test-group-tests.js
Websites/perf.webkit.org/server-tests/privileged-api-update-test-group-tests.js [new file with mode: 0644]
Websites/perf.webkit.org/server-tests/resources/mock-data.js
Websites/perf.webkit.org/tools/js/analysis-results-notifier.js [new file with mode: 0644]
Websites/perf.webkit.org/tools/js/measurement-set-analyzer.js
Websites/perf.webkit.org/tools/js/test-group-result-page.js [new file with mode: 0644]
Websites/perf.webkit.org/tools/js/v3-models.js
Websites/perf.webkit.org/tools/run-analysis.js
Websites/perf.webkit.org/unit-tests/analysis-results-notifier-tests.js [new file with mode: 0644]
Websites/perf.webkit.org/unit-tests/analysis-task-tests.js
Websites/perf.webkit.org/unit-tests/measurement-set-analyzer-tests.js
Websites/perf.webkit.org/unit-tests/resources/mock-remote-api.js
Websites/perf.webkit.org/unit-tests/test-groups-tests.js

index 116b353..46d3bb4 100644 (file)
@@ -1,3 +1,128 @@
+2018-05-24  Dewei Zhu  <dewei_zhu@apple.com>
+
+        Added sending notification feature when test group finishes.
+        https://bugs.webkit.org/show_bug.cgi?id=184340
+
+        Reviewed by Ryosuke Niwa.
+
+        Added 'testgroup_needs_notification' filed to 'analysis_test_group' table to indicate whether a test group
+        has a pending notification.
+        Added 'testgroup_notification_sent_at' to record the last notification sent time.
+        SQL queries to update existing database are:
+            'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_needs_notification boolean NOT NULL DEFAULT FALSE;'
+            'ALTER TABLE analysis_test_groups ADD COLUMN testgroup_notification_sent_at timestamp DEFAULT NULL;'
+        Updated 'run-analysis' script to be able to send notification when test group finishes.
+        Added 'Notify on completion' checkbox while creating/retrying/bisecting a test group.
+
+        * browser-tests/test-group-form-tests.js: Updated existing tests and added a new test.
+        * browser-tests/test-group-result-page-tests.js: Added unit tests for TestGroupResultPage.
+        * init-database.sql: Added 'testgroup_needs_notification' filed to 'analysis_test_group' table.
+        * public/api/test-groups.php: Added '/api/test-groups/ready-for-notification' API to 'test-group' to show all
+        test groups that need to send notification. Only the ones with 'completed', 'failed' or 'cancelled' status and its
+        'testgroup_needs_notification' is true will be returned by this API.
+        * public/include/build-requests-fetcher.php: Added 'fetch_requests_for_groups' to return test groups with given ids.
+        * public/include/commit-sets-helpers.php: Updated the logic to support setting 'testgroup_needs_notification'
+        while create an analysis_test_groups.
+        * public/privileged-api/create-analysis-task.php: Updated the logic to support setting 'testgroup_needs_notification'.
+        * public/privileged-api/create-test-group.php: Updated the logic to support setting 'testgroup_needs_notification'.
+        * public/privileged-api/update-test-group.php: Updated the logic to support updating 'testgroup_needs_notification'.
+        Extended this API to allow authentication both from CSRF token and slave.
+        * public/v3/components/custom-configuration-test-group-form.js:
+        (CustomConfigurationTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents
+        'testgroup_needs_notification' from API perspective.
+        * public/v3/components/customizable-test-group-form.js:
+        (CustomizableTestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents
+        'testgroup_needs_notification' from API perspective.
+        (CustomizableTestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list.
+        * public/v3/components/test-group-form.js:
+        (TestGroupForm): Added '_notifyOnCompletion' instance variable.
+        (TestGroupForm.prototype.didConstructShadowTree): Added 'onchange' event for notify on completion checkbox.
+        (TestGroupForm.prototype.startTesting): Pass 'notifyOnCompletion' information which represents
+        'testgroup_needs_notification' from API perspective.
+        (TestGroupForm.cssTemplate): Added space between 'Notify on completion' checkbox and test group iteration selection list.
+        * public/v3/models/analysis-results.js: Export 'AnalysisResults'.
+        (AnalysisResults.fetch): Update API path to use absolute url.
+        (AnalysisResults):
+        * public/v3/models/analysis-task.js:
+        (AnalysisTask.async.create): Extend this function to take notifyOnCompletion as argument which will be used as
+        'needsNotification' to send to server.
+        (AnalysisTask):
+        * public/v3/models/test-group.js:
+        (TestGroup): Added '_needsNotification' field.
+        (TestGroup.prototype.updateSingleton): Added logic to update '_needsNotification' field.
+        (TestGroup.prototype.needsNotification): Returns '_needsNotification' field.
+        (TestGroup.prototype.author): Returns author information.
+        (TestGroup.prototype.async.didSendNotification): API that updates 'testgroup_needs_notification' to true.
+        (TestGroup.prototype.async.fetchTask): API to fetch the task when it has not been fetched.
+        (TestGroup.createWithTask): Updated this function to accept 'notifyOnCompletion' which will be used as
+        'needsNotification' to send to server.
+        (TestGroup.createWithCustomConfiguration): Updated this function to accept 'notifyOnCompletion' which will be used as
+        'needsNotification' to send to server.
+        (TestGroup.createAndRefetchTestGroups): Updated this function to accept 'notifyOnCompletion' which will be used as
+        'needsNotification' to send to server.
+        (TestGroup.fetchAllWithNotificationReady): New function that invokes '/api/test-groups/ready-for-notification'.
+        * public/v3/pages/analysis-task-page.js: Update logic to 'notifyOnCompletion' around
+        (AnalysisTaskChartPane.prototype.didConstructShadowTree):
+        (AnalysisTaskResultsPane.prototype.didConstructShadowTree):
+        (AnalysisTaskTestGroupPane.prototype.didConstructShadowTree):
+        (AnalysisTaskPage.prototype.didConstructShadowTree):
+        (AnalysisTaskPage.prototype._retryCurrentTestGroup):
+        (AnalysisTaskPage.prototype.async._bisectCurrentTestGroup):
+        (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList.set const):
+        (AnalysisTaskPage.prototype._createTestGroupAfterVerifyingCommitSetList):
+        (AnalysisTaskPage.prototype._createCustomTestGroup):
+        * public/v3/pages/chart-pane.js: Added 'Notify on completion' checkbox.
+        (ChartPane.prototype.didConstructShadowTree):
+        (ChartPane.prototype.async._analyzeRange):
+        * public/v3/pages/create-analysis-task-page.js: Adapted API change.
+        (CreateAnalysisTaskPage.prototype._createAnalysisTaskWithGroup):
+        * server-tests/api-test-groups.js: Added tests for '/api/test-groups/ready-for-notification'.
+        * server-tests/privileged-api-create-analysis-task-tests.js: Updated tests to adapt this change.
+        Added new tests.
+        * server-tests/privileged-api-create-test-group-tests.js: Added new tests.
+        * server-tests/privileged-api-update-test-group-tests.js: Added unit test for 'update-test-group' API.
+        * server-tests/resources/mock-data.js: addMockData should set 'testgroup_needs_notification' to be true.
+        * server-tests/tools-sync-buildbot-integration-tests.js: Updated tests to adapt this change.
+        (async.createTestGroupWihPatch):
+        (createTestGroupWihOwnedCommit):
+        * tools/js/analysis-results-notifier.js: Added notifier to send notification for completed test groups.
+        (AnalysisResultsNotifier):
+        (AnalysisResultsNotifier.prototype.async.sendNotificationsForTestGroups):
+        (AnalysisResultsNotifier.prototype._sendNotification): Invoke remote API to send notification.
+        (AnalysisResultsNotifier.prototype._constructMessageByRules):
+        (AnalysisResultsNotifier._matchesRule):
+        (AnalysisResultsNotifier._applyUpdate):
+        (AnalysisResultsNotifier.async._messageForTestGroup): Build html as message body for a test group.
+        (AnalysisResultsNotifier._URLForAnalysisTask): Returns URL for an analysis task.
+        (AnalysisResultsNotifier._instantiateNotificationTemplate):
+        * tools/js/test-group-result-page.js: Added 'TestGroupResultPage' and 'BarGraph' to show test group result.
+        (TestGroupResultPage):
+        (TestGroupResultPage.prototype.async.setTestGroup):
+        (TestGroupResultPage._urlForAnalysisTask):
+        (TestGroupResultPage.prototype._URLForAnalysisTask):
+        (TestGroupResultPage.prototype.constructTables):
+        (TestGroupResultPage.prototype._constructTableForMetric):
+        (TestGroupResultPage.):
+        (TestGroupResultPage.prototype.get pageContent):
+        (TestGroupResultPage.prototype.get styleTemplate):
+        (BarGraph):
+        (BarGraph.prototype.setWidth):
+        (BarGraph.prototype._constructBarGraph):
+        (BarGraph.prototype.get pageContent):
+        (BarGraph.prototype.get styleTemplate):
+        * tools/js/measurement-set-analyzer.js: Adapted 'AnalysisTask.create' change.
+        (MeasurementSetAnalyzer.prototype.async._analyzeMeasurementSet):
+        (MeasurementSetAnalyzer):
+        * tools/js/v3-models.js:
+        * tools/run-analysis.js: Added the logic that sends notification for completed test groups.
+        (main):
+        (async.analysisLoop):
+        * unit-tests/analysis-task-tests.js:
+        * unit-tests/analysis-results-notifier-tests.js: Added a unit test for 'AnalysisResultsNotifier' and 'NotificationService'.
+        * unit-tests/measurement-set-analyzer-tests.js: Updated unit tests per this change.
+        * unit-tests/test-groups-tests.js: Added unit tests for 'TestGroup.needsNotification'.
+        * unit-tests/resources/mock-remote-api.js: Only set 'privilegedAPI' when it exits.
+
 2018-06-07  Ryosuke Niwa  <rniwa@webkit.org>
 
         Add the basic support for writing components in node.js
index d26bdce..90f3c4d 100644 (file)
@@ -28,6 +28,7 @@ mocha.setup('bdd');
 <script src="commit-log-viewer-tests.js"></script>
 <script src="test-group-form-tests.js"></script>
 <script src="markup-page-tests.js"></script>
+<script src="test-group-result-page-tests.js"></script>
 <script>
 
 afterEach(() => {
index 7e20933..b4ab212 100644 (file)
@@ -18,7 +18,7 @@ describe('TestGroupFormTests', () => {
             testGroupForm.listenToAction('startTesting', (...args) => calls.push(args));
             expect(calls).to.eql({});
             testGroupForm.content('start-button').click();
-            expect(calls).to.eql([[4]]);
+            expect(calls).to.eql([[4, true]]);
         });
     });
 
@@ -29,12 +29,31 @@ describe('TestGroupFormTests', () => {
             testGroupForm.listenToAction('startTesting', (...args) => calls.push(args));
             expect(calls).to.eql({});
             testGroupForm.content('start-button').click();
-            expect(calls).to.eql([[4]]);
+            expect(calls).to.eql([[4, true]]);
             const countForm = testGroupForm.content('repetition-count');
             countForm.value = '6';
-            countForm.dispatchEvent(new Event('change')); 
+            countForm.dispatchEvent(new Event('change'));
             testGroupForm.content('start-button').click();
-            expect(calls).to.eql([[4], [6]]);
+            expect(calls).to.eql([[4, true], [6, true]]);
+        });
+    });
+
+    it('must update "notify on completion" when it is unchecked', () => {
+        const context = new BrowsingContext();
+        return createTestGroupFormWithContext(context).then((testGroupForm) => {
+            const calls = [];
+            testGroupForm.listenToAction('startTesting', (...args) => calls.push(args));
+            expect(calls).to.eql({});
+            testGroupForm.content('start-button').click();
+            expect(calls).to.eql([[4, true]]);
+            const countForm = testGroupForm.content('repetition-count');
+            countForm.value = '6';
+            countForm.dispatchEvent(new Event('change'));
+            const notifyOnCompletionCheckbox = testGroupForm.content('notify-on-completion-checkbox');
+            notifyOnCompletionCheckbox.checked = false;
+            notifyOnCompletionCheckbox.dispatchEvent(new Event('change'));
+            testGroupForm.content('start-button').click();
+            expect(calls).to.eql([[4, true], [6, false]]);
         });
     });
 
@@ -57,10 +76,10 @@ describe('TestGroupFormTests', () => {
                 testGroupForm.listenToAction('startTesting', (...args) => calls.push(args));
                 expect(calls).to.eql({});
                 testGroupForm.content().querySelector('button').click();
-                expect(calls).to.eql([[4]]);
+                expect(calls).to.eql([[4, true]]);
                 testGroupForm.setRepetitionCount(8);
                 testGroupForm.content().querySelector('button').click();
-                expect(calls).to.eql([[4], [8]]);
+                expect(calls).to.eql([[4, true], [8, true]]);
             });
         });
     });
diff --git a/Websites/perf.webkit.org/browser-tests/test-group-result-page-tests.js b/Websites/perf.webkit.org/browser-tests/test-group-result-page-tests.js
new file mode 100644 (file)
index 0000000..eea1e4b
--- /dev/null
@@ -0,0 +1,80 @@
+describe('TestGroupResultPage', () => {
+
+    async function importMarkupComponent(context)
+    {
+        return await context.importScripts(['lazily-evaluated-function.js', '../shared/common-component-base.js', '../../tools/js/markup-component.js', '../../tools/js/test-group-result-page.js',
+            'models/data-model.js', 'models/metric.js', '../shared/statistics.js',], 'TestGroupResultPage', 'Metric');
+    }
+
+    async function prepareTestGroupResultPage(context, resultA, resultB)
+    {
+        const [TestGroupResultPage, Metric] = await importMarkupComponent(context);
+
+        const mockAnalysisTask = {
+            metric: () => ({
+                makeFormatter: (sigFig, alwaysShowSign) => Metric.makeFormatter('MB', sigFig, alwaysShowSign),
+                aggregatorLabel: () => 'Arithmetic mean'
+            }),
+            name: () => 'mock-analysis-task'
+        };
+
+        const mockTestGroup = {
+            requestedCommitSets: () => ['A', 'B'],
+            test: () => ({test: () => 'speeodmeter-2', name: () => 'speedometer-2'}),
+            labelForCommitSet: (commitSet) => commitSet,
+            requestsForCommitSet: (commitSet) => ({'A': resultA, 'B': resultB}[commitSet]),
+            compareTestResults: (...args) => ({isStatisticallySignificant: true, changeType: 'worse'}),
+            name: () => 'mock-test-group',
+        };
+
+        const mockAnalysisResults = {
+            viewForMetric: (metric) => ({resultForRequest: (buildRequest) => (buildRequest === null ? null : {value: buildRequest})})
+        };
+
+        const page = new TestGroupResultPage('test');
+        page._testGroup = mockTestGroup;
+        page._analysisTask = mockAnalysisTask;
+        page._analysisResults = mockAnalysisResults;
+        page._analysisURL = 'http://localhost';
+
+        return page;
+    }
+
+    it('should render failed test group with empty bar', async () => {
+        const context = new BrowsingContext();
+        const page = await prepareTestGroupResultPage(context, [null, 3, 5], [2, 4, 6]);
+        await page.enqueueToRender();
+        const document = context.document;
+        document.open();
+        document.write(page.generateMarkup());
+        document.close();
+
+        expect(context.global.getComputedStyle(context.document.querySelector('.bar-graph-placeholder')).width).to.be('0px');
+    });
+
+    it('should render right ratio based on test group result', async () => {
+        const context = new BrowsingContext();
+        const resultA = [1, 3, 5];
+        const resultB = [2, 4, 6];
+        const page = await prepareTestGroupResultPage(context, resultA, resultB);
+        page.enqueueToRender();
+        const document = context.document;
+        document.open();
+        document.write(page.generateMarkup());
+        document.close();
+
+        const min = 0.5;
+        const max = 6.5;
+        const expectedPercentages = [...resultA, ...resultB].map((result) => (result - min) / (max - min));
+        const barNodes = context.document.querySelectorAll('.bar-graph-placeholder');
+        expect(barNodes.length).to.be(6);
+
+        const almostEqual = (a, b) => Math.abs(a - b) < 0.001;
+        let previousNodeWidth = parseFloat(context.global.getComputedStyle(barNodes[0]).width);
+        for (let i = 1; i < 6; ++ i) {
+            const currentNodeWidth = parseFloat(context.global.getComputedStyle(barNodes[i]).width);
+            expect(almostEqual(currentNodeWidth / previousNodeWidth, expectedPercentages[i] / expectedPercentages[i - 1])).to.be(true);
+            previousNodeWidth = currentNodeWidth;
+        }
+    });
+});
\ No newline at end of file
index c4742e7..913240a 100644 (file)
@@ -279,6 +279,8 @@ CREATE TABLE analysis_test_groups (
     testgroup_author varchar(256),
     testgroup_created_at timestamp NOT NULL DEFAULT (CURRENT_TIMESTAMP AT TIME ZONE 'UTC'),
     testgroup_hidden boolean NOT NULL DEFAULT FALSE,
+    testgroup_needs_notification boolean NOT NULL DEFAULT FALSE,
+    testgroup_notification_sent_at timestamp DEFAULT NULL,
     CONSTRAINT testgroup_name_must_be_unique_for_each_task UNIQUE(testgroup_task, testgroup_name));
 CREATE INDEX testgroup_task_index ON analysis_test_groups(testgroup_task);
 
index 42ddd30..e76a24a 100644 (file)
@@ -13,14 +13,7 @@ function main($path) {
 
     $build_requests_fetcher = new BuildRequestsFetcher($db);
 
-    if (count($path) > 0 && $path[0]) {
-        $group_id = intval($path[0]);
-        $group = $db->select_first_row('analysis_test_groups', 'testgroup', array('id' => $group_id));
-        if (!$group)
-            exit_with_error('GroupNotFound', array('id' => $group_id));
-        $test_groups = array($group);
-        $build_requests_fetcher->fetch_for_group($group['testgroup_task'], $group_id);
-    } else {
+    if (!count($path)) {
         $task_id = array_get($_GET, 'task');
         if (!$task_id)
             exit_with_error('TaskIdNotSpecified');
@@ -29,6 +22,28 @@ function main($path) {
         if (!is_array($test_groups))
             exit_with_error('FailedToFetchTestGroups');
         $build_requests_fetcher->fetch_for_task($task_id);
+    } elseif ($path[0] == 'ready-for-notification') {
+        $test_groups = $db->query_and_fetch_all("SELECT * FROM analysis_test_groups
+            WHERE EXISTS(SELECT 1 FROM build_requests
+                WHERE request_group = testgroup_id
+                    AND request_status IN ('pending', 'scheduled', 'running', 'canceled')) IS FALSE
+                    AND testgroup_needs_notification IS TRUE AND testgroup_hidden IS FALSE");
+
+        if (!count($test_groups)) {
+            exit_with_success(array('testGroups' => array(),
+                'buildRequests' => array(),
+                'commitSets' => array(),
+                'commits' => array(),
+                'uploadedFiles' => array()));
+        }
+        $build_requests_fetcher->fetch_requests_for_groups($test_groups);
+    } else {
+        $group_id = intval($path[0]);
+        $group = $db->select_first_row('analysis_test_groups', 'testgroup', array('id' => $group_id));
+        if (!$group)
+            exit_with_error('GroupNotFound', array('id' => $group_id));
+        $test_groups = array($group);
+        $build_requests_fetcher->fetch_for_group($group['testgroup_task'], $group_id);
     }
     if (!$build_requests_fetcher->has_results())
         exit_with_error('FailedToFetchBuildRequests');
@@ -63,8 +78,10 @@ function format_test_group($group_row) {
         'task' => $group_row['testgroup_task'],
         'name' => $group_row['testgroup_name'],
         'author' => $group_row['testgroup_author'],
-        'createdAt' => strtotime($group_row['testgroup_created_at']) * 1000,
+        'createdAt' => Database::to_js_time($group_row['testgroup_created_at']),
+        'notificationSentAt' => Database::to_js_time($group_row['testgroup_notification_sent_at']),
         'hidden' => Database::is_true($group_row['testgroup_hidden']),
+        'needsNotification' => Database::is_true($group_row['testgroup_needs_notification']),
         'buildRequests' => array(),
         'commitSets' => array(),
     );
index 27f0240..b388779 100644 (file)
@@ -30,6 +30,17 @@ class BuildRequestsFetcher {
             $row['task_id'] = $task_id;
     }
 
+    function fetch_requests_for_groups($test_groups) {
+        $test_group_id_list = array();
+        foreach($test_groups as $group)
+            array_push($test_group_id_list, intval($group['testgroup_id']));
+
+        $this->rows = $this->db->query_and_fetch_all('SELECT *, testgroup_task as task_id
+            FROM build_requests, analysis_test_groups
+            WHERE request_group = testgroup_id AND testgroup_id = ANY($1)
+            ORDER BY request_group, request_order', array('{' . implode(', ', $test_group_id_list) . '}'));
+    }
+
     function fetch_incomplete_requests_for_triggerable($triggerable_id) {
         $this->rows = $this->db->query_and_fetch_all('SELECT *, test_groups.testgroup_task as task_id FROM build_requests,
             (SELECT testgroup_id, testgroup_task, (case when testgroup_author is not null then 0 else 1 end) as author_order, testgroup_created_at
index a7adc4f..bec7d48 100644 (file)
@@ -4,12 +4,12 @@ require_once('repository-group-finder.php');
 require_once('commit-log-fetcher.php');
 
 # FIXME: Should create a helper class for below 3 helper functions to avoid passing long argument list.
-function create_test_group_and_build_requests($db, $commit_sets, $task_id, $name, $author, $triggerable_id, $platform_id, $test_id, $repetition_count) {
+function create_test_group_and_build_requests($db, $commit_sets, $task_id, $name, $author, $triggerable_id, $platform_id, $test_id, $repetition_count, $needs_notification) {
 
     list ($build_configuration_list, $test_configuration_list) = insert_commit_sets_and_construct_configuration_list($db, $commit_sets);
 
     $group_id = $db->insert_row('analysis_test_groups', 'testgroup',
-        array('task' => $task_id, 'name' => $name, 'author' => $author));
+        array('task' => $task_id, 'name' => $name, 'author' => $author, 'needs_notification' => $needs_notification));
 
     $build_count = count($build_configuration_list);
     $order = -$build_count;
index 940f80b..4804cc7 100644 (file)
@@ -10,6 +10,7 @@ function main() {
     $author = remote_user_name($data);
     $name = array_get($data, 'name');
     $repetition_count = array_get($data, 'repetitionCount');
+    $needs_notification = array_get($data, 'needsNotification', False);
     $test_group_name = array_get($data, 'testGroupName');
     $revision_set_list = array_get($data, 'revisionSets');
 
@@ -84,7 +85,7 @@ function main() {
         $triggerable_id = $triggerable['id'];
         $test_id = $triggerable['test'];
         $commit_sets = commit_sets_from_revision_sets($db, $triggerable_id, $revision_set_list);
-        create_test_group_and_build_requests($db, $commit_sets, $task_id, $test_group_name, $author, $triggerable_id, $config['config_platform'], $test_id, $repetition_count);
+        create_test_group_and_build_requests($db, $commit_sets, $task_id, $test_group_name, $author, $triggerable_id, $config['config_platform'], $test_id, $repetition_count, $needs_notification);
     }
 
     $db->commit_transaction();
index 56b4755..1023d4b 100644 (file)
@@ -18,6 +18,7 @@ function main()
     $task_id = array_get($arguments, 'task');
     $task_name = array_get($data, 'taskName');
     $repetition_count = $arguments['repetitionCount'];
+    $needs_notification = array_get($data, 'needsNotification', False);
     $platform_id = array_get($data, 'platform');
     $test_id = array_get($data, 'test');
     $revision_set_list = array_get($data, 'revisionSets');
@@ -85,7 +86,7 @@ function main()
     if ($task_name)
         $task_id = $db->insert_row('analysis_tasks', 'task', array('name' => $task_name, 'author' => $author));
 
-    $group_id = create_test_group_and_build_requests($db, $commit_sets, $task_id, $name, $author, $triggerable_id, $platform_id, $test_id, $repetition_count);
+    $group_id = create_test_group_and_build_requests($db, $commit_sets, $task_id, $name, $author, $triggerable_id, $platform_id, $test_id, $repetition_count, $needs_notification);
 
     $db->commit_transaction();
 
index db1ed38..ad1755f 100644 (file)
@@ -3,7 +3,8 @@
 require_once('../include/json-header.php');
 
 function main() {
-    $data = ensure_privileged_api_data_and_token();
+    $db = connect();
+    $data = ensure_privileged_api_data_and_token_or_slave($db);
 
     $test_group_id = array_get($data, 'group');
     if (!$test_group_id)
@@ -17,10 +18,20 @@ function main() {
     if (array_key_exists('hidden', $data))
         $values['hidden'] = Database::to_database_boolean($data['hidden']);
 
+    $has_needs_notification_field = array_key_exists('needsNotification', $data);
+    $has_notification_sent_at_field = array_key_exists('notificationSentAt', $data);
+
+    if ($has_needs_notification_field || $has_notification_sent_at_field) {
+        if (!!$has_notification_sent_at_field == !!$data['needsNotification'])
+            exit_with_error('NotificationSentAtFieldShouldOnlyBeSetWhenNeedsNotificationIsFalse');
+
+        $values['needs_notification'] = Database::to_database_boolean($data['needsNotification']);
+        if (!$data['needsNotification'])
+            $values['notification_sent_at'] = $data['notificationSentAt'];
+    }
+
     if (!$values)
         exit_with_error('NothingToUpdate');
-
-    $db = connect();
     $db->begin_transaction();
 
     if (!$db->update_row('analysis_test_groups', 'testgroup', array('id' => $test_group_id), $values)) {
index 58ebbaa..2a2b867 100644 (file)
@@ -38,7 +38,7 @@ class CustomConfigurationTestGroupForm extends TestGroupForm {
         const commitSets = configurator.commitSets();
         const platform = configurator.platform();
         const test = configurator.tests()[0]; // FIXME: Add the support for specifying multiple tests.
-        this.dispatchAction('startTesting', this._repetitionCount, testGroupName, commitSets, platform, test, taskName);
+        this.dispatchAction('startTesting', this._repetitionCount, testGroupName, commitSets, platform, test, taskName, this._notifyOnCompletion);
     }
 
     didConstructShadowTree()
index 53ef7f7..20441c1 100644 (file)
@@ -25,7 +25,7 @@ class CustomizableTestGroupForm extends TestGroupForm {
 
     startTesting()
     {
-        this.dispatchAction('startTesting', this._repetitionCount, this._name, this._computeCommitSetMap());
+        this.dispatchAction('startTesting', this._repetitionCount, this._name, this._computeCommitSetMap(), this._notifyOnCompletion);
     }
 
     didConstructShadowTree()
@@ -286,6 +286,10 @@ class CustomizableTestGroupForm extends TestGroupForm {
                 color: #333;
             }
 
+            #customize-link-container {
+                margin-left: 0.4rem;
+            }
+
             #custom-table:not(:empty) {
                 margin: 1rem 0;
             }
@@ -320,6 +324,10 @@ class CustomizableTestGroupForm extends TestGroupForm {
             #custom-table th {
                 text-align: center;
             }
+
+            #notify-on-completion-checkbox {
+                margin-left: 0.4rem;
+            }
             `;
     }
 
index c44b66f..a43edf3 100644 (file)
@@ -5,6 +5,7 @@ class TestGroupForm extends ComponentBase {
     {
         super(name || 'test-group-form');
         this._repetitionCount = 4;
+        this._notifyOnCompletion = true;
     }
 
     setRepetitionCount(count)
@@ -18,13 +19,15 @@ class TestGroupForm extends ComponentBase {
         const repetitionCountSelect = this.content('repetition-count');
         repetitionCountSelect.onchange = () => {
             this._repetitionCount = repetitionCountSelect.value;
-        }
+        };
+        const notifyOnCompletionCheckBox = this.content('notify-on-completion-checkbox');
+        notifyOnCompletionCheckBox.onchange = () => this._notifyOnCompletion = notifyOnCompletionCheckBox.checked;
         this.content('form').onsubmit = this.createEventHandler(() => this.startTesting());
     }
 
     startTesting()
     {
-        this.dispatchAction('startTesting', this._repetitionCount);
+        this.dispatchAction('startTesting', this._repetitionCount, this._notifyOnCompletion);
     }
 
     static htmlTemplate()
@@ -38,6 +41,11 @@ class TestGroupForm extends ComponentBase {
             :host {
                 display: block;
             }
+
+            #notify-on-completion-checkbox {
+                margin-left: 0.5rem;
+                width: 1rem;
+            }
         `;
     }
 
@@ -58,6 +66,7 @@ class TestGroupForm extends ComponentBase {
                 <option>10</option>
             </select>
             iterations per set
+            <input id="notify-on-completion-checkbox" type="checkbox" checked/>Notify on completion
         `;
     }
 
index d048bee..259553c 100644 (file)
@@ -84,7 +84,7 @@ class AnalysisResults {
     static fetch(taskId)
     {
         taskId = parseInt(taskId);
-        return RemoteAPI.getJSONWithStatus(`../api/measurement-set?analysisTask=${taskId}`).then(function (response) {
+        return RemoteAPI.getJSONWithStatus(`/api/measurement-set?analysisTask=${taskId}`).then(function (response) {
 
             Instrumentation.startMeasuringTime('AnalysisResults', 'fetch');
 
@@ -116,3 +116,8 @@ class AnalysisResultsView {
         return this._results.findResult(buildRequest.buildId(), this._metric.id());
     }
 }
+
+
+if (typeof module !== 'undefined') {
+    module.exports.AnalysisResults = AnalysisResults;
+}
\ No newline at end of file
index 5db4524..dbb7c2e 100644 (file)
@@ -303,7 +303,7 @@ class AnalysisTask extends LabeledObject {
         return results;
     }
 
-    static async create(name, startPoint, endPoint, testGroupName=null, repetitionCount=0)
+    static async create(name, startPoint, endPoint, testGroupName=null, repetitionCount=0, notifyOnCompletion=false)
     {
         const parameters = {name, startRun: startPoint.id, endRun: endPoint.id};
         if (testGroupName) {
@@ -311,6 +311,7 @@ class AnalysisTask extends LabeledObject {
             parameters['revisionSets'] = CommitSet.revisionSetsFromCommitSets([startPoint.commitSet(), endPoint.commitSet()]);
             parameters['repetitionCount'] = repetitionCount;
             parameters['testGroupName'] = testGroupName;
+            parameters['needsNotification'] = notifyOnCompletion;
         }
         const response = await PrivilegedAPI.sendRequest('create-analysis-task', parameters);
         return AnalysisTask.fetchById(response.taskId, true);
index 0317e05..fa41dda 100644 (file)
@@ -9,6 +9,7 @@ class TestGroup extends LabeledObject {
         this._authorName = object.author;
         this._createdAt = new Date(object.createdAt);
         this._isHidden = object.hidden;
+        this._needsNotification = object.needsNotification;
         this._buildRequests = [];
         this._orderBuildRequestsLazily = new LazilyEvaluatedFunction((...buildRequests) => {
             return buildRequests.sort((a, b) => a.order() - b.order());
@@ -30,12 +31,17 @@ class TestGroup extends LabeledObject {
         console.assert(this._platform == object.platform);
 
         this._isHidden = object.hidden;
+        this._needsNotification = object.needsNotification;
+        this._notificationSentAt = object.notificationSentAt ? new Date(object.notificationSentAt) : null;
     }
 
     task() { return AnalysisTask.findById(this._taskId); }
     createdAt() { return this._createdAt; }
     isHidden() { return this._isHidden; }
     buildRequests() { return this._buildRequests; }
+    needsNotification() { return this._needsNotification; }
+    notificationSentAt() { return this._notificationSentAt; }
+    author() { return this._authorName; }
     addBuildRequest(request)
     {
         this._buildRequests.push(request);
@@ -49,6 +55,13 @@ class TestGroup extends LabeledObject {
         return request ? request.test() : null;
     }
 
+    async fetchTask()
+    {
+        if (this.task())
+            return this.task();
+        return await AnalysisTask.fetchById(this._taskId);
+    }
+
     platform() { return this._platform; }
 
     _lastRequest()
@@ -185,11 +198,23 @@ class TestGroup extends LabeledObject {
         });
     }
 
-    static createWithTask(taskName, platform, test, groupName, repetitionCount, commitSets)
+    async didSendNotification()
+    {
+        const id = this.id();
+        await PrivilegedAPI.sendRequest('update-test-group', {
+            group: id,
+            needsNotification: false,
+            notificationSentAt: (new Date).toISOString()
+        });
+        const data = await TestGroup.cachedFetch(`/api/test-groups/${id}`, {}, true);
+        return TestGroup._createModelsFromFetchedTestGroups(data);
+    }
+
+    static createWithTask(taskName, platform, test, groupName, repetitionCount, commitSets, notifyOnCompletion)
     {
         console.assert(commitSets.length == 2);
         const revisionSets = CommitSet.revisionSetsFromCommitSets(commitSets);
-        const params = {taskName, name: groupName, platform: platform.id(), test: test.id(), repetitionCount, revisionSets};
+        const params = {taskName, name: groupName, platform: platform.id(), test: test.id(), repetitionCount, revisionSets, needsNotification: !!notifyOnCompletion};
         return PrivilegedAPI.sendRequest('create-test-group', params).then((data) => {
             return AnalysisTask.fetchById(data['taskId'], true);
         }).then((task) => {
@@ -197,17 +222,17 @@ class TestGroup extends LabeledObject {
         });
     }
 
-    static createWithCustomConfiguration(task, platform, test, groupName, repetitionCount, commitSets)
+    static createWithCustomConfiguration(task, platform, test, groupName, repetitionCount, commitSets, notifyOnCompletion)
     {
         console.assert(commitSets.length == 2);
         const revisionSets = CommitSet.revisionSetsFromCommitSets(commitSets);
-        const params = {task: task.id(), name: groupName, platform: platform.id(), test: test.id(), repetitionCount, revisionSets};
+        const params = {task: task.id(), name: groupName, platform: platform.id(), test: test.id(), repetitionCount, revisionSets, needsNotification: !!notifyOnCompletion};
         return PrivilegedAPI.sendRequest('create-test-group', params).then((data) => {
             return this.fetchForTask(data['taskId'], true);
         });
     }
 
-    static createAndRefetchTestGroups(task, name, repetitionCount, commitSets)
+    static createAndRefetchTestGroups(task, name, repetitionCount, commitSets, notifyOnCompletion)
     {
         console.assert(commitSets.length == 2);
         const revisionSets = CommitSet.revisionSetsFromCommitSets(commitSets);
@@ -216,6 +241,7 @@ class TestGroup extends LabeledObject {
             name: name,
             repetitionCount: repetitionCount,
             revisionSets: revisionSets,
+            needsNotification: !!notifyOnCompletion,
         }).then((data) => this.fetchForTask(data['taskId'], true));
     }
 
@@ -229,6 +255,11 @@ class TestGroup extends LabeledObject {
         return this.cachedFetch('/api/test-groups', {task: taskId}, ignoreCache).then(this._createModelsFromFetchedTestGroups.bind(this));
     }
 
+    static fetchAllWithNotificationReady()
+    {
+        return this.cachedFetch('/api/test-groups/ready-for-notification', null, true).then(this._createModelsFromFetchedTestGroups.bind(this));
+    }
+
     static _createModelsFromFetchedTestGroups(data)
     {
         var testGroups = data['testGroups'].map(function (row) {
index 5aaf068..14e3a72 100644 (file)
@@ -24,8 +24,8 @@ class AnalysisTaskChartPane extends ChartPaneBase {
 
     didConstructShadowTree()
     {
-        this.part('form').listenToAction('startTesting', (repetitionCount, name, commitSetMap) => {
-            this.dispatchAction('newTestGroup', name, repetitionCount, commitSetMap);
+        this.part('form').listenToAction('startTesting', (repetitionCount, name, commitSetMap, notifyOnCompletion) => {
+            this.dispatchAction('newTestGroup', name, repetitionCount, commitSetMap, notifyOnCompletion);
         });
     }
 
@@ -106,8 +106,8 @@ class AnalysisTaskResultsPane extends ComponentBase {
         repositoryPicker.addEventListener('change', () => this.enqueueToRender());
         this.part('commit-viewer').setShowRepositoryName(false);
 
-        this.part('form').listenToAction('startTesting', (repetitionCount, name, commitSetMap) => {
-            this.dispatchAction('newTestGroup', name, repetitionCount, commitSetMap);
+        this.part('form').listenToAction('startTesting', (repetitionCount, name, commitSetMap, notifyOnCompletion) => {
+            this.dispatchAction('newTestGroup', name, repetitionCount, commitSetMap, notifyOnCompletion);
         });
     }
 
@@ -267,14 +267,14 @@ class AnalysisTaskTestGroupPane extends ComponentBase {
     didConstructShadowTree()
     {
         this.content('hide-button').onclick = () => this.dispatchAction('toggleTestGroupVisibility', this._currentTestGroup);
-        this.part('retry-form').listenToAction('startTesting', (repetitionCount) => {
-            this.dispatchAction('retryTestGroup', this._currentTestGroup, repetitionCount);
+        this.part('retry-form').listenToAction('startTesting', (repetitionCount, notifyOnCompletion) => {
+            this.dispatchAction('retryTestGroup', this._currentTestGroup, repetitionCount, notifyOnCompletion);
         });
-        this.part('bisect-form').listenToAction('startTesting', (repetitionCount) => {
+        this.part('bisect-form').listenToAction('startTesting', (repetitionCount, notifyOnCompletion) => {
             const bisectingCommitSet = this._bisectingCommitSetByTestGroup.get(this._currentTestGroup);
             const [oneCommitSet, anotherCommitSet] = this._currentTestGroup.requestedCommitSets();
             const commitSets = [oneCommitSet, bisectingCommitSet, anotherCommitSet];
-            this.dispatchAction('bisectTestGroup', this._currentTestGroup, commitSets, repetitionCount);
+            this.dispatchAction('bisectTestGroup', this._currentTestGroup, commitSets, repetitionCount, notifyOnCompletion);
         });
     }
 
@@ -543,8 +543,8 @@ class AnalysisTaskPage extends PageWithHeading {
         groupPane.listenToAction('showHiddenTestGroups', () => this._showAllTestGroups());
         groupPane.listenToAction('renameTestGroup', (testGroup, newName) => this._updateTestGroupName(testGroup, newName));
         groupPane.listenToAction('toggleTestGroupVisibility', (testGroup) => this._hideCurrentTestGroup(testGroup));
-        groupPane.listenToAction('retryTestGroup', (testGroup, repetitionCount) => this._retryCurrentTestGroup(testGroup, repetitionCount));
-        groupPane.listenToAction('bisectTestGroup', (testGroup, commitSets, repetitionCount) => this._bisectCurrentTestGroup(testGroup, commitSets, repetitionCount));
+        groupPane.listenToAction('retryTestGroup', (testGroup, repetitionCount, notifyOnCompletion) => this._retryCurrentTestGroup(testGroup, repetitionCount, notifyOnCompletion));
+        groupPane.listenToAction('bisectTestGroup', (testGroup, commitSets, repetitionCount, notifyOnCompletion) => this._bisectCurrentTestGroup(testGroup, commitSets, repetitionCount, notifyOnCompletion));
 
         this.part('cause-list').listenToAction('addItem', (repository, revision) => {
             this._associateCommit('cause', repository, revision);
@@ -819,19 +819,19 @@ class AnalysisTaskPage extends PageWithHeading {
         });
     }
 
-    _retryCurrentTestGroup(testGroup, repetitionCount)
+    _retryCurrentTestGroup(testGroup, repetitionCount, notifyOnCompletion)
     {
         const existingNames = (this._testGroups || []).map((group) => group.name());
         const newName = CommitSet.createNameWithoutCollision(testGroup.name(), new Set(existingNames));
         const commitSetList = testGroup.requestedCommitSets();
         const platform = this._task.platform() || testGroup.platform();
-        return TestGroup.createWithCustomConfiguration(this._task, platform, testGroup.test(), newName, repetitionCount, commitSetList)
+        return TestGroup.createWithCustomConfiguration(this._task, platform, testGroup.test(), newName, repetitionCount, commitSetList, notifyOnCompletion)
             .then(this._didFetchTestGroups.bind(this), function (error) {
             alert('Failed to create a new test group: ' + error);
         });
     }
 
-    async _bisectCurrentTestGroup(testGroup, commitSets, repetitionCount)
+    async _bisectCurrentTestGroup(testGroup, commitSets, repetitionCount, notifyOnCompletion)
     {
         console.assert(testGroup.task());
         const existingTestGroupNames = new Set((this._testGroups || []).map((testGroup) => testGroup.name()));
@@ -841,7 +841,7 @@ class AnalysisTaskPage extends PageWithHeading {
             const currentCommitSet = commitSets[i];
             const testGroupName = CommitSet.createNameWithoutCollision(CommitSet.diff(previousCommitSet, currentCommitSet), existingTestGroupNames);
             try {
-                const testGroups = await TestGroup.createAndRefetchTestGroups(testGroup.task(), testGroupName, repetitionCount, [previousCommitSet, currentCommitSet]);
+                const testGroups = await TestGroup.createAndRefetchTestGroups(testGroup.task(), testGroupName, repetitionCount, [previousCommitSet, currentCommitSet], notifyOnCompletion);
                 await this._didFetchTestGroups(testGroups);
             } catch(error) {
                 alert('Failed to create a new test group: ' + error);
@@ -850,7 +850,7 @@ class AnalysisTaskPage extends PageWithHeading {
         }
     }
 
-    _createTestGroupAfterVerifyingCommitSetList(testGroupName, repetitionCount, commitSetMap)
+    _createTestGroupAfterVerifyingCommitSetList(testGroupName, repetitionCount, commitSetMap, notifyOnCompletion)
     {
         if (this._hasDuplicateTestGroupName(testGroupName)) {
             alert(`There is already a test group named "${testGroupName}"`);
@@ -876,13 +876,13 @@ class AnalysisTaskPage extends PageWithHeading {
         for (let label in commitSetMap)
             commitSets.push(commitSetMap[label]);
 
-        return TestGroup.createAndRefetchTestGroups(this._task, testGroupName, repetitionCount, commitSets)
+        return TestGroup.createAndRefetchTestGroups(this._task, testGroupName, repetitionCount, commitSets, notifyOnCompletion)
             .then(this._didFetchTestGroups.bind(this), function (error) {
             alert('Failed to create a new test group: ' + error);
         });
     }
 
-    _createCustomTestGroup(repetitionCount, testGroupName, commitSets, platform, test)
+    _createCustomTestGroup(repetitionCount, testGroupName, commitSets, platform, test, notifyOnCompletion)
     {
         console.assert(this._task.isCustom());
         if (this._hasDuplicateTestGroupName(testGroupName)) {
@@ -890,7 +890,7 @@ class AnalysisTaskPage extends PageWithHeading {
             return;
         }
 
-        TestGroup.createWithCustomConfiguration(this._task, platform, test, testGroupName, repetitionCount, commitSets)
+        TestGroup.createWithCustomConfiguration(this._task, platform, test, testGroupName, repetitionCount, commitSets, notifyOnCompletion)
             .then(this._didFetchTestGroups.bind(this), function (error) {
             alert('Failed to create a new test group: ' + error);
         });
index 8cd7aa0..d0af173 100644 (file)
@@ -119,7 +119,12 @@ class ChartPane extends ChartPaneBase {
         });
         const createWithTestGroupCheckbox = this.content('create-with-test-group');
         const repetitionCount = this.content('confirm-repetition');
-        createWithTestGroupCheckbox.onchange = () => repetitionCount.disabled = !createWithTestGroupCheckbox.checked;
+        const notifyOnCompletion = this.content('notify-on-completion');
+        createWithTestGroupCheckbox.onchange = () => {
+            const shouldDisable = !createWithTestGroupCheckbox.checked;
+            repetitionCount.disabled = shouldDisable;
+            notifyOnCompletion.disabled = shouldDisable;
+        }
     }
 
     serializeState()
@@ -237,10 +242,11 @@ class ChartPane extends ChartPaneBase {
         const name = this.content('task-name').value;
         const createWithTestGroup = this.content('create-with-test-group').checked;
         const repetitionCount = this.content('confirm-repetition').value;
+        const notifyOnCompletion = this.content('notify-on-completion').checked;
 
         try {
             const analysisTask = await (createWithTestGroup ?
-                AnalysisTask.create(name, startPoint, endPoint, 'Confirm', repetitionCount) : AnalysisTask.create(name, startPoint, endPoint));
+                AnalysisTask.create(name, startPoint, endPoint, 'Confirm', repetitionCount, notifyOnCompletion) : AnalysisTask.create(name, startPoint, endPoint));
             newWindow.location.href = router.url('analysis/task/' + analysisTask.id());
             this.fetchAnalysisTasks(true);
         } catch(error) {
@@ -583,6 +589,7 @@ class ChartPane extends ChartPaneBase {
                                     <option>10</option>
                                 </select>
                             <label>iterations</label>
+                            <label><input type="checkbox" id="notify-on-completion" checked> Notify on completion</label>
                         </li>
                     </form>
                     <ul class="chart-pane-filtering-options popover" style="display:none">
index a9f0890..301fd2a 100644 (file)
@@ -22,9 +22,9 @@ class CreateAnalysisTaskPage extends PageWithHeading {
         this.part('form').listenToAction('startTesting', this._createAnalysisTaskWithGroup.bind(this));
     }
 
-    _createAnalysisTaskWithGroup(repetitionCount, testGroupName, commitSets, platform, test, taskName)
+    _createAnalysisTaskWithGroup(repetitionCount, testGroupName, commitSets, platform, test, taskName, notifyOnCompletion)
     {
-        TestGroup.createWithTask(taskName, platform, test, testGroupName, repetitionCount, commitSets).then((task) => {
+        TestGroup.createWithTask(taskName, platform, test, testGroupName, repetitionCount, commitSets, notifyOnCompletion).then((task) => {
             const url = this.router().url(`analysis/task/${task.id()}`);
             location.href = this.router().url(`analysis/task/${task.id()}`);
         }, (error) => {
diff --git a/Websites/perf.webkit.org/server-tests/api-test-groups.js b/Websites/perf.webkit.org/server-tests/api-test-groups.js
new file mode 100644 (file)
index 0000000..a6c676a
--- /dev/null
@@ -0,0 +1,88 @@
+'use strict';
+
+const assert = require('assert');
+const MockData = require('./resources/mock-data.js');
+const TestServer = require('./resources/test-server.js');
+const prepareServerTest = require('./resources/common-operations.js').prepareServerTest;
+
+describe('/api/test-groups', function () {
+    prepareServerTest(this);
+
+    describe('/api/test-groups/ready-for-notification', () => {
+
+        it('should give an empty list if there is not existing test group at all', async () => {
+            const content = await TestServer.remoteAPI().getJSON('/api/test-groups/ready-for-notification');
+            assert.equal(content.status, 'OK');
+            assert.deepEqual(content.testGroups, []);
+            assert.deepEqual(content.buildRequests, []);
+            assert.deepEqual(content.commitSets, []);
+            assert.deepEqual(content.commits, []);
+            assert.deepEqual(content.uploadedFiles, []);
+        });
+
+        it('should not include a test group with "canceled" state in at least one build request', async () => {
+            await MockData.addMockData(TestServer.database(), ['completed', 'completed', 'completed', 'canceled']);
+            const content = await TestServer.remoteAPI().getJSON('/api/test-groups/ready-for-notification');
+            assert.equal(content.status, 'OK');
+            assert.deepEqual(content.testGroups, []);
+            assert.deepEqual(content.buildRequests, []);
+            assert.deepEqual(content.commitSets, []);
+            assert.deepEqual(content.commits, []);
+            assert.deepEqual(content.uploadedFiles, []);
+        });
+
+        it('should list all test groups with pending notification', async () => {
+            await MockData.addMockData(TestServer.database(), ['completed', 'completed', 'completed', 'completed']);
+            const content = await TestServer.remoteAPI().getJSON('/api/test-groups/ready-for-notification');
+            assert.equal(content.testGroups.length, 1);
+            const testGroup = content.testGroups[0];
+            assert.equal(testGroup.id, 600);
+            assert.equal(testGroup.task, 500);
+            assert.equal(testGroup.name, 'some test group');
+            assert.equal(testGroup.author, null);
+            assert.equal(testGroup.hidden, false);
+            assert.equal(testGroup.needsNotification, true);
+            assert.equal(testGroup.platform, 65);
+            assert.deepEqual(testGroup.buildRequests, ['700','701', '702', '703']);
+            assert.deepEqual(testGroup.commitSets, ['401', '402', '401', '402']);
+        });
+
+        it('should not list hidden test group', async () => {
+            const database = TestServer.database();
+            await MockData.addMockData(database, ['completed', 'completed', 'completed', 'completed']);
+            await database.query('UPDATE analysis_test_groups SET testgroup_hidden = TRUE WHERE testgroup_id = 600');
+            const content = await TestServer.remoteAPI().getJSON('/api/test-groups/ready-for-notification');
+            assert.equal(content.status, 'OK');
+            assert.deepEqual(content.testGroups, []);
+            assert.deepEqual(content.buildRequests, []);
+            assert.deepEqual(content.commitSets, []);
+            assert.deepEqual(content.commits, []);
+            assert.deepEqual(content.uploadedFiles, []);
+        });
+
+        it('should not list test groups without needs notification flag', async () => {
+            const database = TestServer.database();
+            await MockData.addMockData(database, ['completed', 'completed', 'completed', 'completed']);
+            await database.query('UPDATE analysis_test_groups SET testgroup_needs_notification = FALSE WHERE testgroup_id = 600');
+            const content = await TestServer.remoteAPI().getJSON('/api/test-groups/ready-for-notification');
+            assert.equal(content.status, 'OK');
+            assert.deepEqual(content.testGroups, []);
+            assert.deepEqual(content.buildRequests, []);
+            assert.deepEqual(content.commitSets, []);
+            assert.deepEqual(content.commits, []);
+            assert.deepEqual(content.uploadedFiles, []);
+        });
+
+        it('should not list a test group that has some incompleted build requests', async () => {
+            const database = TestServer.database();
+            await MockData.addMockData(database, ['completed', 'completed', 'completed', 'running']);
+            const content = await TestServer.remoteAPI().getJSON('/api/test-groups/ready-for-notification');
+            assert.equal(content.status, 'OK');
+            assert.deepEqual(content.testGroups, []);
+            assert.deepEqual(content.buildRequests, []);
+            assert.deepEqual(content.commitSets, []);
+            assert.deepEqual(content.commits, []);
+            assert.deepEqual(content.uploadedFiles, []);
+        });
+    });
+});
\ No newline at end of file
index c7a6d17..f2d01fa 100644 (file)
@@ -6,6 +6,7 @@ let MockData = require('./resources/mock-data.js');
 let TestServer = require('./resources/test-server.js');
 const addBuilderForReport = require('./resources/common-operations.js').addBuilderForReport;
 const prepareServerTest = require('./resources/common-operations.js').prepareServerTest;
+const assertThrows = require('./resources/common-operations.js').assertThrows;
 
 const reportWithRevision = [{
     "buildNumber": "124",
@@ -353,17 +354,10 @@ describe('/privileged-api/create-analysis-task with browser privileged api', fun
         const oneRevisionSet = {[webkitId]: {revision: '191622'}};
         const anotherRevisionSet = {[webkitId]: {revision: '191623'}};
 
-        let raiseException = false;
-
-        try {
-            await PrivilegedAPI.sendRequest('create-analysis-task', {name: 'confirm', repetitionCount: 1,
+        await assertThrows('TriggerableNotFoundForTask', () =>
+            PrivilegedAPI.sendRequest('create-analysis-task', {name: 'confirm', repetitionCount: 1,
                 revisionSets: [oneRevisionSet, anotherRevisionSet],
-                startRun: testRuns[0]['id'], endRun: testRuns[1]['id']});
-        } catch (error) {
-            assert.equal(error, 'TriggerableNotFoundForTask');
-            raiseException = true;
-        }
-        assert.ok(raiseException);
+                startRun: testRuns[0]['id'], endRun: testRuns[1]['id']}));
     });
 
     it('should create an analysis task with no test group when repetition count is 0', async () => {
@@ -456,6 +450,7 @@ describe('/privileged-api/create-analysis-task with browser privileged api', fun
         assert.equal(testGroups.length, 1);
         const testGroup = testGroups[0];
         assert.equal(testGroup.name(), 'Confirm');
+        assert.ok(!testGroup.needsNotification());
         const buildRequests = testGroup.buildRequests();
         assert.equal(buildRequests.length, 2);
 
@@ -513,17 +508,10 @@ describe('/privileged-api/create-analysis-task with node privileged api', functi
         const oneRevisionSet = {[webkitId]: {revision: '191622'}};
         const anotherRevisionSet = {[webkitId]: {revision: '191623'}};
 
-        let raiseException = false;
-
-        try {
-            await PrivilegedAPI.sendRequest('create-analysis-task', {name: 'confirm', repetitionCount: 1,
+        await assertThrows('SlaveNotFound', () =>
+            PrivilegedAPI.sendRequest('create-analysis-task', {name: 'confirm', repetitionCount: 1,
                 revisionSets: [oneRevisionSet, anotherRevisionSet],
-                startRun: testRuns[0]['id'], endRun: testRuns[1]['id']});
-        } catch (error) {
-            assert.equal(error, 'SlaveNotFound');
-            raiseException = true;
-        }
-        assert.ok(raiseException);
+                startRun: testRuns[0]['id'], endRun: testRuns[1]['id']}));
 
         const allAnalysisTasks = await db.selectRows('analysis_tasks');
         assert.ok(!allAnalysisTasks.length);
@@ -584,6 +572,87 @@ describe('/privileged-api/create-analysis-task with node privileged api', functi
         assert.equal(testGroups.length, 1);
         const testGroup = testGroups[0];
         assert.equal(testGroup.name(), 'Confirm');
+        assert.ok(!testGroup.needsNotification());
+        const buildRequests = testGroup.buildRequests();
+        assert.equal(buildRequests.length, 2);
+
+        assert.equal(buildRequests[0].triggerable().id(), triggerableId);
+        assert.equal(buildRequests[0].triggerable().id(), triggerableId);
+
+        assert.equal(buildRequests[0].testGroup(), testGroup);
+        assert.equal(buildRequests[1].testGroup(), testGroup);
+
+        assert.equal(buildRequests[0].platform(), task.platform());
+        assert.equal(buildRequests[1].platform(), task.platform());
+
+        assert.equal(buildRequests[0].analysisTaskId(), task.id());
+        assert.equal(buildRequests[1].analysisTaskId(), task.id());
+
+        assert.equal(buildRequests[0].test(), test1);
+        assert.equal(buildRequests[1].test(), test1);
+
+        assert.ok(!buildRequests[0].isBuild());
+        assert.ok(!buildRequests[1].isBuild());
+        assert.ok(buildRequests[0].isTest());
+        assert.ok(buildRequests[1].isTest());
+
+        const firstCommitSet = buildRequests[0].commitSet();
+        const secondCommitSet = buildRequests[1].commitSet();
+        const webkitRepository = Repository.findById(webkitId);
+        assert.equal(firstCommitSet.commitForRepository(webkitRepository).revision(), '191622');
+        assert.equal(secondCommitSet.commitForRepository(webkitRepository).revision(), '191623');
+    });
+
+    it('should create an analysis task with test group and respect the "needsNotification" flag in the http request', async () => {
+        const webkitId = 1;
+        const platformId = 1;
+        const test1Id = 2;
+        const triggerableId = 1234;
+
+        const db = TestServer.database();
+        await db.insert('tests', {id: 1, name: 'Suite'});
+        await db.insert('tests', {id: test1Id, name: 'test1', parent: 1});
+        await db.insert('repositories', {id: webkitId, name: 'WebKit'});
+        await db.insert('platforms', {id: platformId, name: 'some platform'});
+        await db.insert('build_triggerables', {id: 1234, name: 'test-triggerable'});
+        await db.insert('triggerable_repository_groups', {id: 2345, name: 'webkit-only', triggerable: triggerableId});
+        await db.insert('triggerable_repositories', {repository: webkitId, group: 2345});
+        await db.insert('triggerable_configurations', {test: test1Id, platform: platformId, triggerable: triggerableId});
+        await addBuilderForReport(reportWithRevision[0]);
+
+        await TestServer.remoteAPI().postJSON('/api/report/', reportWithRevision);
+        await TestServer.remoteAPI().postJSON('/api/report/', anotherReportWithRevision);
+        await Manifest.fetch();
+
+        let test1 = Test.findById(test1Id);
+        let somePlatform = Platform.findById(platformId);
+        const configRow = await db.selectFirstRow('test_configurations', {metric: test1.metrics()[0].id(), platform: somePlatform.id()});
+        const testRuns = await db.selectRows('test_runs', {config: configRow['id']});
+        assert.equal(testRuns.length, 2);
+
+        const oneRevisionSet = {[webkitId]: {revision: '191622'}};
+        const anotherRevisionSet = {[webkitId]: {revision: '191623'}};
+
+        const content = await PrivilegedAPI.sendRequest('create-analysis-task', {name: 'confirm', repetitionCount: 1,
+            testGroupName: 'Confirm', revisionSets: [oneRevisionSet, anotherRevisionSet],
+            startRun: testRuns[0]['id'], endRun: testRuns[1]['id'], needsNotification: true});
+
+        const task = await AnalysisTask.fetchById(content['taskId']);
+        assert.equal(task.name(), 'confirm');
+        assert(!task.hasResults());
+        assert(task.hasPendingRequests());
+        assert.deepEqual(task.bugs(), []);
+        assert.deepEqual(task.causes(), []);
+        assert.deepEqual(task.fixes(), []);
+        assert.equal(task.changeType(), null);
+        assert.equal(task.platform().label(), 'some platform');
+        assert.equal(task.metric().test().label(), 'test1');
+
+        const testGroups = await TestGroup.fetchForTask(task.id());
+        assert.equal(testGroups.length, 1);
+        const testGroup = testGroups[0];
+        assert.equal(testGroup.name(), 'Confirm');
+        assert.ok(testGroup.needsNotification());
         const buildRequests = testGroup.buildRequests();
         assert.equal(buildRequests.length, 2);
 
index 2f3b297..9aec5c3 100644 (file)
@@ -401,6 +401,7 @@ describe('/privileged-api/create-test-group', function () {
             const group = testGroups[0];
             assert.equal(group.id(), groupId);
             assert.equal(group.repetitionCount(), 1);
+            assert.ok(!group.needsNotification());
             const requests = group.buildRequests();
             assert.equal(requests.length, 4);
             assert(requests[0].isBuild());
@@ -447,6 +448,7 @@ describe('/privileged-api/create-test-group', function () {
             const group = testGroups[0];
             assert.equal(group.id(), groupId);
             assert.equal(group.repetitionCount(), 1);
+            assert.ok(!group.needsNotification());
             const requests = group.buildRequests();
             assert.equal(requests.length, 4);
             assert(requests[0].isBuild());
@@ -484,6 +486,7 @@ describe('/privileged-api/create-test-group', function () {
                 const group = testGroups[0];
                 assert.equal(group.id(), insertedGroupId);
                 assert.equal(group.repetitionCount(), 1);
+                assert.ok(!group.needsNotification());
                 const requests = group.buildRequests();
                 assert.equal(requests.length, 2);
 
@@ -522,6 +525,7 @@ describe('/privileged-api/create-test-group', function () {
                 const group = testGroups[0];
                 assert.equal(group.id(), insertedGroupId);
                 assert.equal(group.repetitionCount(), 1);
+                assert.ok(!group.needsNotification());
                 const requests = group.buildRequests();
                 assert.equal(requests.length, 2);
 
@@ -553,6 +557,7 @@ describe('/privileged-api/create-test-group', function () {
                 const group = testGroups[0];
                 assert.equal(group.id(), insertedGroupId);
                 assert.equal(group.repetitionCount(), 2);
+                assert.ok(!group.needsNotification());
                 const requests = group.buildRequests();
                 assert.equal(requests.length, 4);
                 const webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
@@ -599,6 +604,7 @@ describe('/privileged-api/create-test-group', function () {
                 const group = testGroups[0];
                 assert.equal(group.id(), insertedGroupId);
                 assert.equal(group.repetitionCount(), 2);
+                assert.ok(!group.needsNotification());
                 const requests = group.buildRequests();
                 assert.equal(requests.length, 4);
 
@@ -640,6 +646,7 @@ describe('/privileged-api/create-test-group', function () {
                 const group = testGroups[0];
                 assert.equal(group.id(), insertedGroupId);
                 assert.equal(group.repetitionCount(), 2);
+                assert.ok(!group.needsNotification());
                 const requests = group.buildRequests();
                 assert.equal(requests.length, 4);
 
@@ -688,6 +695,7 @@ describe('/privileged-api/create-test-group', function () {
                 const group = testGroups[0];
                 assert.equal(group.id(), insertedGroupId);
                 assert.equal(group.repetitionCount(), 2);
+                assert.ok(!group.needsNotification());
                 const requests = group.buildRequests();
                 assert.equal(requests.length, 4);
 
@@ -736,6 +744,7 @@ describe('/privileged-api/create-test-group', function () {
             const group = testGroups[0];
             assert.equal(group.id(), insertedGroupId);
             assert.equal(group.repetitionCount(), 2);
+            assert.ok(!group.needsNotification());
             assert.equal(group.test(), Test.findById(MockData.someTestId()));
             assert.equal(group.platform(), Platform.findById(MockData.somePlatformId()));
             const requests = group.buildRequests();
@@ -800,6 +809,7 @@ describe('/privileged-api/create-test-group', function () {
             const group = testGroups[0];
             assert.equal(group.id(), insertedGroupId);
             assert.equal(group.repetitionCount(), 2);
+            assert.ok(!group.needsNotification());
             assert.equal(group.test(), Test.findById(MockData.someTestId()));
             assert.equal(group.platform(), Platform.findById(MockData.somePlatformId()));
             const requests = group.buildRequests();
@@ -874,6 +884,7 @@ describe('/privileged-api/create-test-group', function () {
             const group = testGroups[0];
             assert.equal(group.id(), insertedGroupId);
             assert.equal(group.repetitionCount(), 2);
+            assert.ok(!group.needsNotification());
             assert.equal(group.test(), Test.findById(MockData.someTestId()));
             assert.equal(group.platform(), Platform.findById(MockData.somePlatformId()));
             const requests = group.buildRequests();
@@ -956,6 +967,7 @@ describe('/privileged-api/create-test-group', function () {
             const group = testGroups[0];
             assert.equal(group.id(), insertedGroupId);
             assert.equal(group.repetitionCount(), 2);
+            assert.ok(!group.needsNotification());
             assert.equal(group.test(), Test.findById(MockData.someTestId()));
             assert.equal(group.platform(), Platform.findById(MockData.somePlatformId()));
             const requests = group.buildRequests();
@@ -1050,6 +1062,7 @@ describe('/privileged-api/create-test-group', function () {
             assert.equal(group.id(), insertedGroupId);
             assert.equal(group.repetitionCount(), 2);
             assert.equal(group.test(), Test.findById(MockData.someTestId()));
+            assert.ok(!group.needsNotification());
             assert.equal(group.platform(), Platform.findById(MockData.somePlatformId()));
             const requests = group.buildRequests();
             assert.equal(requests.length, 6);
@@ -1145,6 +1158,7 @@ describe('/privileged-api/create-test-group', function () {
             const group = testGroups[0];
             assert.equal(group.id(), insertedGroupId);
             assert.equal(group.repetitionCount(), 2);
+            assert.ok(!group.needsNotification());
             assert.equal(group.test(), Test.findById(MockData.someTestId()));
             assert.equal(group.platform(), Platform.findById(MockData.somePlatformId()));
             const requests = group.buildRequests();
@@ -1207,38 +1221,62 @@ describe('/privileged-api/create-test-group', function () {
         });
     });
 
-    it('should create a test group with an analysis task', () => {
-        let insertedGroupId;
-        let webkit;
-        return addTriggerableAndCreateTask('some task').then(() => {
-            webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
-            const revisionSets = [{[webkit.id()]: {revision: '191622'}}, {[webkit.id()]: {revision: '191623'}}];
-            return PrivilegedAPI.sendRequest('create-test-group',
-                {name: 'test', taskName: 'other task', platform: MockData.somePlatformId(), test: MockData.someTestId(), revisionSets});
-        }).then((result) => {
-            insertedGroupId = result['testGroupId'];
-            return Promise.all([AnalysisTask.fetchById(result['taskId']), TestGroup.fetchForTask(result['taskId'], true)]);
-        }).then((result) => {
-            const [analysisTask, testGroups] = result;
-
-            assert.equal(analysisTask.name(), 'other task');
-
-            assert.equal(testGroups.length, 1);
-            const group = testGroups[0];
-            assert.equal(group.id(), insertedGroupId);
-            assert.equal(group.repetitionCount(), 1);
-            const requests = group.buildRequests();
-            assert.equal(requests.length, 2);
+    it('should create a test group with an analysis task with needs-notification flag set', async () => {
+        await addTriggerableAndCreateTask('some task');
+        const webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
+        const revisionSets = [{[webkit.id()]: {revision: '191622'}}, {[webkit.id()]: {revision: '191623'}}];
+        let result = await PrivilegedAPI.sendRequest('create-test-group',
+            {name: 'test', taskName: 'other task', platform: MockData.somePlatformId(), test: MockData.someTestId(), needsNotification: true, revisionSets});
+        const insertedGroupId = result['testGroupId'];
+
+        const [analysisTask, testGroups] = await Promise.all([AnalysisTask.fetchById(result['taskId']), TestGroup.fetchForTask(result['taskId'], true)]);
+        assert.equal(analysisTask.name(), 'other task');
+
+        assert.equal(testGroups.length, 1);
+        const group = testGroups[0];
+        assert.equal(group.id(), insertedGroupId);
+        assert.equal(group.repetitionCount(), 1);
+        assert.ok(group.needsNotification());
+        const requests = group.buildRequests();
+        assert.equal(requests.length, 2);
+
+        const set0 = requests[0].commitSet();
+        const set1 = requests[1].commitSet();
+        assert.deepEqual(set0.repositories(), [webkit]);
+        assert.deepEqual(set0.customRoots(), []);
+        assert.deepEqual(set1.repositories(), [webkit]);
+        assert.deepEqual(set1.customRoots(), []);
+        assert.equal(set0.revisionForRepository(webkit), '191622');
+        assert.equal(set1.revisionForRepository(webkit), '191623');
+    });
 
-            const set0 = requests[0].commitSet();
-            const set1 = requests[1].commitSet();
-            assert.deepEqual(set0.repositories(), [webkit]);
-            assert.deepEqual(set0.customRoots(), []);
-            assert.deepEqual(set1.repositories(), [webkit]);
-            assert.deepEqual(set1.customRoots(), []);
-            assert.equal(set0.revisionForRepository(webkit), '191622');
-            assert.equal(set1.revisionForRepository(webkit), '191623');
-        });
+    it('should be able to create a test group with needs-notification flag unset', async () => {
+        await addTriggerableAndCreateTask('some task');
+        const webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
+        const revisionSets = [{[webkit.id()]: {revision: '191622'}}, {[webkit.id()]: {revision: '191623'}}];
+        let result = await PrivilegedAPI.sendRequest('create-test-group',
+            {name: 'test', taskName: 'other task', platform: MockData.somePlatformId(), test: MockData.someTestId(), needsNotification: false, revisionSets});
+        const insertedGroupId = result['testGroupId'];
+
+        const [analysisTask, testGroups] = await Promise.all([AnalysisTask.fetchById(result['taskId']), TestGroup.fetchForTask(result['taskId'], true)]);
+        assert.equal(analysisTask.name(), 'other task');
+
+        assert.equal(testGroups.length, 1);
+        const group = testGroups[0];
+        assert.equal(group.id(), insertedGroupId);
+        assert.equal(group.repetitionCount(), 1);
+        assert.ok(!group.needsNotification());
+        const requests = group.buildRequests();
+        assert.equal(requests.length, 2);
+
+        const set0 = requests[0].commitSet();
+        const set1 = requests[1].commitSet();
+        assert.deepEqual(set0.repositories(), [webkit]);
+        assert.deepEqual(set0.customRoots(), []);
+        assert.deepEqual(set1.repositories(), [webkit]);
+        assert.deepEqual(set1.customRoots(), []);
+        assert.equal(set0.revisionForRepository(webkit), '191622');
+        assert.equal(set1.revisionForRepository(webkit), '191623');
     });
 
     it('should create a custom test group for an existing custom analysis task', () => {
diff --git a/Websites/perf.webkit.org/server-tests/privileged-api-update-test-group-tests.js b/Websites/perf.webkit.org/server-tests/privileged-api-update-test-group-tests.js
new file mode 100644 (file)
index 0000000..c7be48a
--- /dev/null
@@ -0,0 +1,207 @@
+'use strict';
+
+const assert = require('assert');
+
+const MockData = require('./resources/mock-data.js');
+const TestServer = require('./resources/test-server.js');
+const addSlaveForReport = require('./resources/common-operations.js').addSlaveForReport;
+const prepareServerTest = require('./resources/common-operations.js').prepareServerTest;
+const assertThrows = require('./resources/common-operations.js').assertThrows;
+
+async function createAnalysisTask(name, webkitRevisions = ["191622", "191623"])
+{
+    const reportWithRevision = [{
+        "buildNumber": "124",
+        "buildTime": "2015-10-27T15:34:51",
+        "revisions": {
+            "WebKit": {
+                "revision": webkitRevisions[0],
+                "timestamp": '2015-10-27T11:36:56.878473Z',
+            },
+            "macOS": {
+                "revision": "15A284",
+            }
+        },
+        "builderName": "someBuilder",
+        "slaveName": "someSlave",
+        "slavePassword": "somePassword",
+        "platform": "some platform",
+        "tests": {
+            "some test": {
+                "metrics": {
+                    "Time": ["Arithmetic"],
+                },
+                "tests": {
+                    "test1": {
+                        "metrics": {"Time": { "current": [11] }},
+                    }
+                }
+            },
+        }}];
+
+    const anotherReportWithRevision = [{
+        "buildNumber": "125",
+        "buildTime": "2015-10-27T17:27:41",
+        "revisions": {
+            "WebKit": {
+                "revision": webkitRevisions[1],
+                "timestamp": '2015-10-27T16:38:10.768995Z',
+            },
+            "macOS": {
+                "revision": "15A284",
+            }
+        },
+        "builderName": "someBuilder",
+        "slaveName": "someSlave",
+        "slavePassword": "somePassword",
+        "platform": "some platform",
+        "tests": {
+            "some test": {
+                "metrics": {
+                    "Time": ["Arithmetic"],
+                },
+                "tests": {
+                    "test1": {
+                        "metrics": {"Time": { "current": [12] }},
+                    }
+                }
+            },
+        }}];
+
+    const db = TestServer.database();
+    const remote = TestServer.remoteAPI();
+    await addSlaveForReport(reportWithRevision[0]);
+    await remote.postJSON('/api/report/', reportWithRevision);
+    await remote.postJSON('/api/report/', anotherReportWithRevision);
+    await Manifest.fetch();
+    const test = Test.findByPath(['some test', 'test1']);
+    const platform = Platform.findByName('some platform');
+    const configRow = await db.selectFirstRow('test_configurations', {metric: test.metrics()[0].id(), platform: platform.id()});
+    const testRuns = await db.selectRows('test_runs', {config: configRow['id']});
+
+    assert.equal(testRuns.length, 2);
+    const content = await PrivilegedAPI.sendRequest('create-analysis-task', {
+        name: name,
+        startRun: testRuns[0]['id'],
+        endRun: testRuns[1]['id'],
+        needsNotification: true,
+    });
+    return content['taskId'];
+}
+
+async function addTriggerableAndCreateTask(name, webkitRevisions)
+{
+    const report = {
+        'slaveName': 'anotherSlave',
+        'slavePassword': 'anotherPassword',
+        'triggerable': 'build-webkit',
+        'configurations': [
+            {test: MockData.someTestId(), platform: MockData.somePlatformId()},
+            {test: MockData.someTestId(), platform: MockData.otherPlatformId()},
+        ],
+        'repositoryGroups': [
+            {name: 'os-only', acceptsRoot: true, repositories: [
+                {repository: MockData.macosRepositoryId(), acceptsPatch: false},
+            ]},
+            {name: 'webkit-only', acceptsRoot: true, repositories: [
+                {repository: MockData.webkitRepositoryId(), acceptsPatch: true},
+            ]},
+            {name: 'system-and-webkit', acceptsRoot: true, repositories: [
+                {repository: MockData.macosRepositoryId(), acceptsPatch: false},
+                {repository: MockData.webkitRepositoryId(), acceptsPatch: true}
+            ]},
+            {name: 'system-webkit-sjc', acceptsRoot: true, repositories: [
+                {repository: MockData.macosRepositoryId(), acceptsPatch: false},
+                {repository: MockData.jscRepositoryId(), acceptsPatch: false},
+                {repository: MockData.webkitRepositoryId(), acceptsPatch: true}
+            ]},
+        ]
+    };
+    await MockData.addMockData(TestServer.database());
+    await addSlaveForReport(report);
+    await TestServer.remoteAPI().postJSON('/api/update-triggerable/', report);
+    await createAnalysisTask(name, webkitRevisions);
+}
+
+describe('/privileged-api/update-test-group', function(){
+    prepareServerTest(this, 'node');
+    beforeEach(() => {
+        PrivilegedAPI.configure('test', 'password');
+    });
+
+    it('should return "SlaveNotFound" if invalid slave name and password combination is provided', async () => {
+        await addTriggerableAndCreateTask('some task');
+        PrivilegedAPI.configure('test', 'wrongpassword');
+
+        await assertThrows('SlaveNotFound', () => PrivilegedAPI.sendRequest('update-test-group', {}));
+    });
+
+    it('should return "TestGroupNotSpecified" if test group is not specified', async () => {
+        await addTriggerableAndCreateTask('some task');
+
+        await assertThrows('TestGroupNotSpecified', () => PrivilegedAPI.sendRequest('update-test-group', {}));
+    });
+
+    it('should return "NotificationSentAtFieldShouldOnlyBeSetWhenNeedsNotificationIsFalse" if "needsNotification" is false but "notificationSentAt" is not set', async () => {
+        await addTriggerableAndCreateTask('some task');
+
+        await assertThrows('NotificationSentAtFieldShouldOnlyBeSetWhenNeedsNotificationIsFalse', () =>
+            PrivilegedAPI.sendRequest('update-test-group', {group: 1, needsNotification: false}));
+    });
+
+    it('should return "NotificationSentAtFieldShouldOnlyBeSetWhenNeedsNotificationIsFalse" if "needsNotification" is true but "notificationSentAt" is set', async () => {
+        await addTriggerableAndCreateTask('some task');
+
+        await assertThrows('NotificationSentAtFieldShouldOnlyBeSetWhenNeedsNotificationIsFalse', () =>
+            PrivilegedAPI.sendRequest('update-test-group', {group: 1, needsNotification: true, notificationSentAt: (new Date).toISOString()}));
+    });
+
+    it('should be able to update "needs_notification" field to false', async () => {
+        await addTriggerableAndCreateTask('some task');
+        const webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
+        const revisionSets = [{[webkit.id()]: {revision: '191622'}}, {[webkit.id()]: {revision: '191623'}}];
+        let result = await PrivilegedAPI.sendRequest('create-test-group',
+            {name: 'test', taskName: 'other task', platform: MockData.somePlatformId(), test: MockData.someTestId(), needsNotification: true, revisionSets});
+        const insertedGroupId = result['testGroupId'];
+
+        const testGroups = await TestGroup.fetchForTask(result['taskId'], true);
+        assert.equal(testGroups.length, 1);
+        const group = testGroups[0];
+        assert.equal(group.id(), insertedGroupId);
+        assert.equal(group.repetitionCount(), 1);
+        assert.equal(group.needsNotification(), true);
+        assert.ok(!group.notificationSentAt());
+
+        const notificationSentAt = new Date;
+        await PrivilegedAPI.sendRequest('update-test-group', {group: insertedGroupId, needsNotification: false, notificationSentAt: notificationSentAt.toISOString()});
+
+        const updatedGroups = await TestGroup.fetchForTask(result['taskId'], true);
+        assert.equal(updatedGroups.length, 1);
+        assert.equal(updatedGroups[0].needsNotification(), false);
+        assert.equal(updatedGroups[0].notificationSentAt().toISOString(), notificationSentAt.toISOString());
+    });
+
+    it('should be able to update "needs_notification" field to true', async () => {
+        await addTriggerableAndCreateTask('some task');
+        const webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
+        const revisionSets = [{[webkit.id()]: {revision: '191622'}}, {[webkit.id()]: {revision: '191623'}}];
+        let result = await PrivilegedAPI.sendRequest('create-test-group',
+            {name: 'test', taskName: 'other task', platform: MockData.somePlatformId(), test: MockData.someTestId(), needsNotification: false, revisionSets});
+        const insertedGroupId = result['testGroupId'];
+
+        const testGroups = await TestGroup.fetchForTask(result['taskId'], true);
+        assert.equal(testGroups.length, 1);
+        const group = testGroups[0];
+        assert.equal(group.id(), insertedGroupId);
+        assert.equal(group.repetitionCount(), 1);
+        assert.equal(group.needsNotification(), false);
+        assert.ok(!group.notificationSentAt());
+
+        await PrivilegedAPI.sendRequest('update-test-group', {group: insertedGroupId, needsNotification: true});
+
+        const updatedGroups = await TestGroup.fetchForTask(result['taskId'], true);
+        assert.equal(updatedGroups.length, 1);
+        assert.equal(updatedGroups[0].needsNotification(), true);
+        assert.ok(!group.notificationSentAt());
+    });
+});
\ No newline at end of file
index 94a393f..64520a5 100644 (file)
@@ -79,7 +79,7 @@ MockData = {
             db.insert('analysis_tasks', {id: 500, platform: 65, metric: 300, name: 'some task',
                 start_run: 801, start_run_time: '2015-10-27T12:05:27.1Z',
                 end_run: 801, end_run_time: '2015-10-27T12:05:27.1Z'}),
-            db.insert('analysis_test_groups', {id: 600, task: 500, name: 'some test group'}),
+            db.insert('analysis_test_groups', {id: 600, task: 500, name: 'some test group', needs_notification: true}),
             db.insert('build_requests', {id: 700, status: statusList[0], triggerable: 1000, repository_group: 2001, platform: 65, test: 200, group: 600, order: 0, commit_set: 401}),
             db.insert('build_requests', {id: 701, status: statusList[1], triggerable: 1000, repository_group: 2001, platform: 65, test: 200, group: 600, order: 1, commit_set: 402}),
             db.insert('build_requests', {id: 702, status: statusList[2], triggerable: 1000, repository_group: 2001, platform: 65, test: 200, group: 600, order: 2, commit_set: 401}),
diff --git a/Websites/perf.webkit.org/tools/js/analysis-results-notifier.js b/Websites/perf.webkit.org/tools/js/analysis-results-notifier.js
new file mode 100644 (file)
index 0000000..759dc26
--- /dev/null
@@ -0,0 +1,139 @@
+global.CommonComponentBase = require('../../public/shared/common-component-base').CommonComponentBase;
+global.MarkupPage = require('./markup-component.js').MarkupPage;
+global.MarkupComponentBase = require('./markup-component.js').MarkupComponentBase;
+
+const fs = require('fs');
+const path = require('path');
+const os = require('os');
+const TestGroupResultPage = require('./test-group-result-page.js').TestGroupResultPage;
+
+class AnalysisResultsNotifier {
+    constructor(messageTemplate, finalizeScript, messageConstructionRules, notificationServerRemoteAPI, notificationServicePath, Subprocess)
+    {
+        this._messageTemplate = messageTemplate;
+        this._notificationServerRemoteAPI = notificationServerRemoteAPI;
+        this._notificationServicePath = notificationServicePath;
+        this._subprocess = Subprocess;
+        this._finalizeScript = finalizeScript;
+        AnalysisResultsNotifier._validateRules(messageConstructionRules);
+        this._messageConstructionRules = messageConstructionRules;
+    }
+
+    static _validateRules(rules)
+    {
+        function isNonemptyArrayOfStrings(object) {
+            if (!object)
+                return false;
+            if (!Array.isArray(object))
+                return false;
+            return object.every((entry) => entry instanceof String || typeof(entry) === 'string');
+        }
+        for (const rule of rules)
+            console.assert(isNonemptyArrayOfStrings(rule.platforms) || isNonemptyArrayOfStrings(rule.tests), 'Either tests or platforms should be an array of strings');
+    }
+
+    async sendNotificationsForTestGroups(testGroups)
+    {
+        for (const testGroup of testGroups) {
+            await testGroup.fetchTask();
+            const title = `"${testGroup.task().name()}" - "${testGroup.name()}" has finished`;
+            const message = await AnalysisResultsNotifier._messageForTestGroup(testGroup, title);
+            let content = AnalysisResultsNotifier._instantiateNotificationTemplate(this._messageTemplate, title, message);
+            content = this._applyRules(testGroup.platform().name(), testGroup.test().path()[0].name(), content);
+            const testGroupInfo = {author: testGroup.author()};
+
+            const tempDir = fs.mkdtempSync(os.tmpdir());
+            const tempFilePath = path.join(tempDir, 'temp-content.json');
+            fs.writeFileSync(tempFilePath, JSON.stringify({content, testGroupInfo}));
+            content = JSON.parse(await this._subprocess.execute([...this._finalizeScript, tempFilePath]));
+            fs.unlinkSync(tempFilePath);
+            fs.rmdirSync(tempDir);
+
+            await this._sendNotification(content);
+            await testGroup.didSendNotification();
+        }
+    }
+
+    static async _messageForTestGroup(testGroup, title)
+    {
+        const page = new TestGroupResultPage(title);
+        await page.setTestGroup(testGroup);
+
+        return page.generateMarkup();
+    }
+
+    static _instantiateNotificationTemplate(template, title, message)
+    {
+        const instance = {};
+        for (const name in template) {
+            const value = template[name];
+            if (typeof(value) === 'string')
+                instance[name] = value.replace(/\$title/g, title).replace(/\$message/g, message);
+            else if (typeof(template[name]) === 'object')
+                instance[name] = this._instantiateNotificationTemplate(value, title, message);
+            else
+                instance[name] = value;
+        }
+        return instance;
+    }
+
+    _applyRules(platformName, testName, message)
+    {
+        for (const rule of this._messageConstructionRules) {
+            if (AnalysisResultsNotifier._matchesRule(platformName, testName, rule))
+                message = AnalysisResultsNotifier._applyUpdate(message, rule.parameters);
+        }
+        return message;
+    }
+
+    static _matchesRule(platform, test, rule)
+    {
+        if (rule.tests && !rule.tests.includes(test))
+            return false;
+
+        if (rule.platforms && !rule.platforms.includes(platform))
+            return false;
+
+        return true;
+    }
+
+    static _applyUpdate(message, update)
+    {
+        const messageType = typeof message;
+        const updateType = typeof update;
+        const supportedPrimitiveTypes = ["string", "number", "boolean"];
+        const unsupportedPrimitiveTypes = ["symbol", "function", "undefined"];
+        console.assert(!unsupportedPrimitiveTypes.includes(messageType) && !unsupportedPrimitiveTypes.includes(updateType));
+
+        if (supportedPrimitiveTypes.includes(messageType) || supportedPrimitiveTypes.includes(updateType))
+            return [message, update];
+
+        for (let [key, value] of Object.entries(update)) {
+            let mergedValue = null;
+            let valueToMerge = message[key];
+
+            if (!(key in message))
+                mergedValue = value;
+            else if (Array.isArray(value) || Array.isArray(valueToMerge)) {
+                if (!Array.isArray(value))
+                    value = [value];
+                if (!Array.isArray(valueToMerge))
+                    valueToMerge = [valueToMerge];
+                mergedValue = [...valueToMerge, ...value];
+            } else
+                mergedValue = this._applyUpdate(valueToMerge, value);
+
+            message[key] = mergedValue;
+        }
+        return message;
+    }
+
+    _sendNotification(content)
+    {
+        return this._notificationServerRemoteAPI.postJSON(this._notificationServicePath, content);
+    }
+}
+
+
+if (typeof module !== 'undefined')
+    module.exports.AnalysisResultsNotifier = AnalysisResultsNotifier;
\ No newline at end of file
index 2db047a..c93b90b 100644 (file)
@@ -104,7 +104,7 @@ class MeasurementSetAnalyzer {
 
         // FIXME: The iteration count should be smarter than hard-coding.
         const analysisTask = await AnalysisTask.create(summary, rangeWithMostSignificantChange.startPoint,
-            rangeWithMostSignificantChange.endPoint, 'Confirm', 4);
+            rangeWithMostSignificantChange.endPoint, 'Confirm', 4, true);
 
         this._logger.info(`Created analysis task with id "${analysisTask.id()}" to confirm: "${summary}".`);
     }
diff --git a/Websites/perf.webkit.org/tools/js/test-group-result-page.js b/Websites/perf.webkit.org/tools/js/test-group-result-page.js
new file mode 100644 (file)
index 0000000..285a3e2
--- /dev/null
@@ -0,0 +1,246 @@
+class TestGroupResultPage extends MarkupPage {
+    constructor(title)
+    {
+        super(title);
+        this._testGroup = null;
+        this._analysisResults = null;
+        this._analysisURL = null;
+        this._analysisTask = null;
+        this._constructTablesLazily = new LazilyEvaluatedFunction(this.constructTables.bind(this));
+    }
+
+    async setTestGroup(testGroup)
+    {
+        this._testGroup = testGroup;
+        this._analysisTask = await testGroup.fetchTask();
+        this._analysisResults = await AnalysisResults.fetch(this._analysisTask.id());
+        this._analysisURL = TestGroupResultPage._urlForAnalysisTask(this._analysisTask);
+        this.enqueueToRender();
+    }
+
+    static _urlForAnalysisTask(analysisTask)
+    {
+        return global.RemoteAPI.url(`/v3/#/analysis/task/${analysisTask.id()}`);
+    }
+
+    _URLForAnalysisTask(testGroup, analysisResultsView)
+    {
+        const resultsByCommitSet = new Map;
+        let maxValue = -Infinity;
+        let minValue = Infinity;
+        for (const commitSet of testGroup.requestedCommitSets())
+        {
+            const buildRequestsForCommitSet = testGroup.requestsForCommitSet(commitSet);
+            const results = buildRequestsForCommitSet.map((buildRequest) => analysisResultsView.resultForRequest(buildRequest));
+            resultsByCommitSet.set(commitSet, results);
+            for (const result of results) {
+                if (!result)
+                    continue;
+                maxValue = Math.max(maxValue, result.value);
+                minValue = Math.min(minValue, result.value);
+            }
+        }
+        const diff = maxValue - minValue;
+        minValue -= diff * 0.1;
+        maxValue += diff * 0.1;
+
+        return {resultsByCommitSet, widthForValue: (value) => (value - minValue) / (maxValue - minValue) * 100};
+    }
+
+    constructTables(testGroup, analysisResults, analysisURL, analysisTask)
+    {
+        const requestedCommitSets = testGroup.requestedCommitSets();
+        console.assert(requestedCommitSets.length, 2);
+
+        const metrics = analysisTask.metric() ? [analysisTask.metric()] : testGroup.test().metrics();
+
+        const tablesWithSummary = metrics.map((metric) => this._constructTableForMetric(metric, testGroup, analysisResults, requestedCommitSets));
+        const description = this.createElement('h1', [this.createElement('em', testGroup.name()), ' - ', this.createElement('em', this.createLink(analysisTask.name(), analysisURL))]);
+
+        return [description, tablesWithSummary];
+    }
+
+    _constructTableForMetric(metric, testGroup, analysisResults, requestedCommitSets)
+    {
+        const formatter = metric.makeFormatter(4);
+        const deltaFormatter = metric.makeFormatter(2, false);
+        const formatValue = (value, interval) => {
+            const delta = interval ? (interval[1] - interval[0]) / 2 : null;
+            const resultParts = [value == null || isNaN(value) ? '-' : formatter(value)];
+            if (delta != null && !isNaN(delta))
+                resultParts.push(` \u00b1 ${deltaFormatter(delta)}`);
+            return resultParts;
+        };
+
+        const analysisResultsView = analysisResults.viewForMetric(metric);
+        const {resultsByCommitSet, widthForValue} = this._URLForAnalysisTask(testGroup, analysisResultsView);
+
+        const tableBodies = [];
+        for (const commitSet of requestedCommitSets) {
+            let firstRow = true;
+            const tableRows = [];
+            const results = resultsByCommitSet.get(commitSet);
+            const values = results.filter((result) => !!result).map((result) => result.value);
+            const averageColumnContents = formatValue(Statistics.mean(values), Statistics.confidenceInterval(values));
+            const label = testGroup.labelForCommitSet(commitSet);
+
+            for (const result of results) {
+                const cellValue = result ? formatValue(result.value, result.interval).join('') : 'Failed';
+                const barWidth = result ? widthForValue(result.value) : 0;
+                tableRows.push(this._constructTableRow(cellValue, barWidth, firstRow, results.length, label, averageColumnContents));
+                firstRow = false;
+            }
+            tableBodies.push(this.createElement('tbody', tableRows));
+        }
+
+        const beforeValues = resultsByCommitSet.get(requestedCommitSets[0]).filter((result) => !!result).map((result) => result.value);
+        const afterValues = resultsByCommitSet.get(requestedCommitSets[1]).filter((result) => !!result).map((result) => result.value);
+        const comparison = testGroup.compareTestResults(metric, beforeValues, afterValues);
+        const changeStyleClass =  `${comparison.isStatisticallySignificant ? comparison.changeType : 'insignificant'}-result`;
+        const caption = this.createElement('caption', [`${testGroup.test().name()} - ${metric.aggregatorLabel()}: `,
+            this.createElement('em', {class: changeStyleClass}, comparison.fullLabel)]);
+
+        return this.createElement('table', {class: 'result-table'}, [caption, tableBodies]);
+    }
+
+    _constructTableRow(cellValue, barWidth, firstRow, tableHeadRowSpan, labelForCommitSet, averageColumnContents)
+    {
+        const barGraph = new BarGraph;
+        barGraph.setWidth(barWidth);
+        const cellContent = [barGraph, cellValue];
+
+        if (firstRow) {
+            return this.createElement('tr', [
+                this.createElement('th', {class: 'first-row', rowspan: tableHeadRowSpan},
+                    [labelForCommitSet + ': ', averageColumnContents.map((content => this.createElement('span', {class: 'no-wrap'}, content)))]),
+                this.createElement('td', {class: 'result-cell first-row'}, cellContent),
+            ])
+        }
+        else
+            return this.createElement('tr', this.createElement('td', {class: 'result-cell'}, cellContent));
+    }
+
+    render()
+    {
+        super.render();
+        this.renderReplace(this.content(), this._constructTablesLazily.evaluate(this._testGroup, this._analysisResults, this._analysisURL, this._analysisTask));
+    }
+
+    static get pageContent()
+    {
+        return [];
+    }
+
+    static get styleTemplate()
+    {
+        return {
+            'body': {
+                'font-family': 'sans-serif',
+            },
+            'h1': {
+                'font-size': '1.3rem',
+                'font-weight': 'normal',
+            },
+            'em': {
+                'font-weight': 'bold',
+                'font-style': 'normal',
+            },
+            'caption': {
+                'font-size': '1.3rem',
+                'margin': '1rem 0',
+                'text-align': 'left',
+                'white-space': 'nowrap',
+            },
+            'td': {
+                'padding': '0.2rem',
+            },
+            '.first-row': {
+                'border-top': 'solid 1px #ccc',
+            },
+            '.no-wrap': {
+                'white-space': 'nowrap',
+            },
+            'th': {
+                'padding': '0.2rem',
+            },
+            '.result-table': {
+                'margin-top': '1rem',
+                'text-align': 'center',
+                'border-collapse': 'collapse',
+            },
+            '.result-cell': {
+                'min-width': '20rem',
+                'position': 'relative',
+            },
+            '.worse-result': {
+                'color': '#c33',
+            },
+            '.better-result': {
+                'color': '#33c',
+            },
+            '.insignificant-result': {
+                'color': '#666',
+            },
+        }
+    }
+}
+
+class BarGraph extends MarkupComponentBase {
+    constructor()
+    {
+        super('bar-graph');
+        this._constructBarGraphLazily = new LazilyEvaluatedFunction(this._constructBarGraph.bind(this));
+    }
+
+    setWidth(width)
+    {
+        this._width = width;
+        this.enqueueToRender();
+    }
+
+    render()
+    {
+        super.render();
+        this.renderReplace(this.content(), this._constructBarGraphLazily.evaluate(this._width));
+    }
+
+    _constructBarGraph(width)
+    {
+        const barGraphPlaceholder = this.createElement('div',{class: 'bar-graph-placeholder'});
+        if (width)
+            barGraphPlaceholder.style.width = width + '%';
+        return barGraphPlaceholder;
+    }
+
+    static get contentTemplate()
+    {
+        return [];
+    }
+
+    static get styleTemplate()
+    {
+        return {
+            ':host': {
+                'position': 'absolute',
+                'left': 0,
+                'top': 0,
+                'width': 'calc(100% - 2px)',
+                'height': 'calc(100% - 2px)',
+                'padding': '1px',
+                'z-index': -1,
+            },
+            '.bar-graph-placeholder': {
+                'background-color': '#ccc',
+                'height': '100%',
+                'width': '0rem',
+            }
+        };
+    }
+}
+
+MarkupComponentBase.defineElement('test-group-result-page', TestGroupResultPage);
+MarkupComponentBase.defineElement('bar-graph', BarGraph);
+
+
+if (typeof module !== 'undefined')
+    module.exports.TestGroupResultPage = TestGroupResultPage;
\ No newline at end of file
index e1b249c..f208e84 100644 (file)
@@ -11,6 +11,7 @@ importFromV3('models/data-model.js', 'DataModelObject');
 importFromV3('models/data-model.js', 'LabeledObject');
 
 importFromV3('models/analysis-task.js', 'AnalysisTask');
+importFromV3('models/analysis-results.js', 'AnalysisResults');
 importFromV3('models/bug.js', 'Bug');
 importFromV3('models/bug-tracker.js', 'BugTracker');
 importFromV3('models/build-request.js', 'BuildRequest');
index 081a0d1..0f7d3f0 100644 (file)
@@ -3,7 +3,9 @@
 const fs = require('fs');
 const parseArguments = require('./js/parse-arguments.js').parseArguments;
 const RemoteAPI = require('./js/remote.js').RemoteAPI;
-const MeasurementSetAnalyzer = require('./js/measurement-set-analyzer').MeasurementSetAnalyzer;
+const MeasurementSetAnalyzer = require('./js/measurement-set-analyzer.js').MeasurementSetAnalyzer;
+const AnalysisResultsNotifier = require('./js/analysis-results-notifier.js').AnalysisResultsNotifier;
+const Subprocess = require('./js/subprocess.js').Subprocess;
 require('./js/v3-models.js');
 global.PrivilegedAPI = require('./js/privileged-api.js').PrivilegedAPI;
 
@@ -11,6 +13,7 @@ function main(argv)
 {
     const options = parseArguments(argv, [
         {name: '--server-config-json', required: true},
+        {name: '--notification-config-json', required: true},
         {name: '--analysis-range-in-days', type: parseFloat, default: 10},
         {name: '--seconds-to-sleep', type: parseFloat, default: 1200},
     ]);
@@ -26,6 +29,7 @@ async function analysisLoop(options)
     let secondsToSleep;
     try {
         const serverConfig = JSON.parse(fs.readFileSync(options['--server-config-json'], 'utf-8'));
+        const notificationConfig = JSON.parse(fs.readFileSync(options['--notification-config-json'], 'utf-8'));
         const analysisRangeInDays = options['--analysis-range-in-days'];
         secondsToSleep = options['--seconds-to-sleep'];
         global.RemoteAPI = new RemoteAPI(serverConfig.server);
@@ -40,6 +44,15 @@ async function analysisLoop(options)
 
         console.log(`Start analyzing last ${analysisRangeInDays} days measurement sets.`);
         await analyzer.analyzeOnce();
+
+        const testGroups = await TestGroup.fetchAllWithNotificationReady();
+
+        const notificationRemoveAPI = new RemoteAPI(notificationConfig.notificationServerConfig);
+        const notificationMessageConfig = notificationConfig.notificationMessageConfig;
+        const notifier = new AnalysisResultsNotifier(notificationMessageConfig.messageTemplate, notificationMessageConfig.finalizeScript,
+            notificationMessageConfig.messageConstructionRules, notificationRemoveAPI, notificationConfig.notificationServerConfig.path, new Subprocess);
+
+        await notifier.sendNotificationsForTestGroups(testGroups);
     } catch(error) {
         console.error(`Failed analyze measurement sets due to ${error}`);
     }
diff --git a/Websites/perf.webkit.org/unit-tests/analysis-results-notifier-tests.js b/Websites/perf.webkit.org/unit-tests/analysis-results-notifier-tests.js
new file mode 100644 (file)
index 0000000..104b571
--- /dev/null
@@ -0,0 +1,138 @@
+'use strict';
+
+global.CommonComponentBase = require('../public/shared/common-component-base.js').CommonComponentBase;
+global.MarkupPage = require('../tools/js/markup-component.js').MarkupPage;
+global.MarkupComponentBase = require('../tools/js/markup-component.js').MarkupComponentBase;
+
+const assert = require('assert');
+const MockRemoteAPI = require('./resources/mock-remote-api.js').MockRemoteAPI;
+const AnalysisResultsNotifier = require('../tools/js/analysis-results-notifier.js').AnalysisResultsNotifier;
+const assertThrows = require('../server-tests/resources/common-operations.js').assertThrows;
+
+describe('AnalysisResultsNotifier', () => {
+    describe('_matchesRule', () => {
+
+        const trunkMacBook = 'Trunk MacBook';
+        const trunkMacBookPro = 'Trunk MacBook Pro';
+        const trunkMacBookAir = 'Trunk MacBook Air';
+        const speedometer = 'speedometer';
+        const speedometer2 = 'speedometer-2';
+        const jetStream = 'JetStream';
+
+        it('should return a group of matching function based on configuration', () => {
+            const rule = {platforms: [trunkMacBook, trunkMacBookPro], tests: [speedometer, speedometer2]};
+            assert.ok(AnalysisResultsNotifier._matchesRule(trunkMacBook, speedometer2, rule));
+            assert.ok(AnalysisResultsNotifier._matchesRule(trunkMacBookPro, speedometer, rule));
+            assert.ok(AnalysisResultsNotifier._matchesRule(trunkMacBookPro, speedometer2, rule));
+            assert.ok(!AnalysisResultsNotifier._matchesRule(trunkMacBook, jetStream, rule));
+            assert.ok(!AnalysisResultsNotifier._matchesRule(trunkMacBookAir, speedometer, rule));
+            assert.ok(!AnalysisResultsNotifier._matchesRule(trunkMacBookAir, jetStream, rule));
+
+        });
+
+        it('should match rule only contains tests correctly', () => {
+            const rule = {tests: [jetStream]};
+            assert.ok(!AnalysisResultsNotifier._matchesRule(trunkMacBook, speedometer, rule));
+            assert.ok(!AnalysisResultsNotifier._matchesRule(trunkMacBook, speedometer2, rule));
+            assert.ok(!AnalysisResultsNotifier._matchesRule(trunkMacBookPro, speedometer, rule));
+            assert.ok(!AnalysisResultsNotifier._matchesRule(trunkMacBookPro, speedometer2, rule));
+            assert.ok(AnalysisResultsNotifier._matchesRule(trunkMacBook, jetStream, rule));
+            assert.ok(AnalysisResultsNotifier._matchesRule(trunkMacBookPro, jetStream, rule));
+            assert.ok(AnalysisResultsNotifier._matchesRule(trunkMacBookAir, jetStream, rule));
+        });
+
+        it('should match rule only contains platforms correctly', () => {
+            const rule = {platforms: [trunkMacBook]};
+            assert.ok(AnalysisResultsNotifier._matchesRule(trunkMacBook, speedometer, rule));
+            assert.ok(AnalysisResultsNotifier._matchesRule(trunkMacBook, speedometer2, rule));
+            assert.ok(AnalysisResultsNotifier._matchesRule(trunkMacBook, jetStream, rule));
+            assert.ok(!AnalysisResultsNotifier._matchesRule(trunkMacBookPro, speedometer, rule));
+            assert.ok(!AnalysisResultsNotifier._matchesRule(trunkMacBookPro, speedometer2, rule));
+            assert.ok(!AnalysisResultsNotifier._matchesRule(trunkMacBookPro, jetStream, rule));
+            assert.ok(!AnalysisResultsNotifier._matchesRule(trunkMacBookAir, jetStream, rule));
+
+        });
+    });
+
+    describe('_validateRules', () => {
+        it('should fail the validation for empty rule', () => {
+            assert.throws(() => AnalysisResultsNotifier._validateRules([{}]), 'AssertionError [ERR_ASSERTION]', 'Either tests or platforms should be an array of strings');
+        });
+
+        it('should fail the validation for platforms of a rule is not an array of string', () => {
+            assert.throws(() => AnalysisResultsNotifier._validateRules([{tests: [{}]}]), 'AssertionError [ERR_ASSERTION]', 'Either tests or platforms should be an array of strings');
+        });
+
+        it('should pass the validation for if a rule only has tests specified', () => {
+            AnalysisResultsNotifier._validateRules([{tests: ['speedometer']}]);
+        });
+
+        it('should pass the validation for if a rule only has platforms specified', () => {
+            AnalysisResultsNotifier._validateRules([{platforms: ['Trunk MacBook']}]);
+        });
+
+        it('should pass the validation for if a rule has both tests and platforms specified', () => {
+            AnalysisResultsNotifier._validateRules([{tests: ['speedometer'], platforms: ['Trunk MacBook']}]);
+        });
+    });
+
+    describe('_applyUpdate', () => {
+        it('should set value directly if key does not exist in another object', () => {
+            const merged = AnalysisResultsNotifier._applyUpdate({a: 1}, {b: 2});
+            assert.deepEqual(merged, {a: 1, b: 2});
+        });
+
+        it('should concatenate arrays if both are array', () => {
+            const merged = AnalysisResultsNotifier._applyUpdate({a: [1]}, {a: [2]});
+            assert.deepEqual(merged, {a: [1, 2]});
+        });
+
+        it('should add to the array if only one side is array', () => {
+            const merged = AnalysisResultsNotifier._applyUpdate({a: [1], b: 1}, {a: 2, b: [2]});
+            assert.deepEqual(merged, {a: [1, 2], b: [1, 2]});
+        });
+
+        it('should merge recursively', () => {
+            const merged = AnalysisResultsNotifier._applyUpdate({a: {b: 1}}, {a: {b: [2]}});
+            assert.deepEqual(merged, {a: {b: [1, 2]}});
+        });
+
+        it('should merge values into array if one of value with same key is one of follow primitive types: number, string and boolean', () => {
+            const merged = AnalysisResultsNotifier._applyUpdate({a: 1, b: "a", c: 1, d: true, e: {}}, {a: 2, b: "b", c: "b", d: 2, e: 1});
+            assert.deepEqual(merged, {a: [1, 2], b: ["a", "b"], c: [1, "b"], d: [true, 2], e: [{}, 1]});
+        });
+    });
+
+    describe('_sendNotification', () => {
+        beforeEach(() => {
+            MockRemoteAPI.reset('http://send-notification.webkit.org/');
+        });
+
+        it('send a notification successfully', async () => {
+            const notifier = new AnalysisResultsNotifier(null, null, [{tests: ['a']}], MockRemoteAPI, 'send-notification');
+            const promise = notifier._sendNotification('test');
+            assert.equal(MockRemoteAPI.requests.length, 1);
+            const request = MockRemoteAPI.requests[0];
+            assert.equal(request.data, 'test');
+            assert.equal(request.url, 'send-notification');
+            assert.equal(request.method, 'POST');
+
+            request.resolve('ok');
+            const result = await promise;
+            assert.equal(result, 'ok');
+        });
+
+        it('should fail if notification request gets rejected', async () => {
+            const notifier = new AnalysisResultsNotifier(null, null, [{tests: ['a']}], MockRemoteAPI, 'send-notification');
+            const promise = notifier._sendNotification('test');
+            assert.equal(MockRemoteAPI.requests.length, 1);
+            const request = MockRemoteAPI.requests[0];
+            assert.equal(request.data, 'test');
+            assert.equal(request.url, 'send-notification');
+            assert.equal(request.method, 'POST');
+
+            request.reject('unavailable');
+            assertThrows('unavailable', () => promise);
+        });
+    });
+});
\ No newline at end of file
index 008df3f..d845c91 100644 (file)
@@ -304,7 +304,7 @@ describe('AnalysisTask', () => {
 
         it('should create analysis task with confirming repetition count specified', async () => {
             const [startPoint, endPoint] = mockStartAndEndPoints();
-            AnalysisTask.create('confirm', startPoint, endPoint, 'Confirm', 4);
+            AnalysisTask.create('confirm', startPoint, endPoint, 'Confirm', 4, true);
             assert.equal(requests.length, 1);
             assert.equal(requests[0].url, '/privileged-api/generate-csrf-token');
             requests[0].resolve({
@@ -315,7 +315,29 @@ describe('AnalysisTask', () => {
             await MockRemoteAPI.waitForRequest();
             assert.equal(requests[1].url, '/privileged-api/create-analysis-task');
             assert.equal(requests.length, 2);
-            assert.deepEqual(requests[1].data, {name: 'confirm', repetitionCount: 4,
+            assert.deepEqual(requests[1].data, {name: 'confirm', repetitionCount: 4, needsNotification: true,
+                startRun: 1, endRun: 2, testGroupName: 'Confirm', token: 'abc', revisionSets: [
+                    {'11': {revision: 'webkit-revision-1', ownerRevision: null, patch: null},
+                        '22': {revision: 'ios-revision-1', ownerRevision: null, patch: null}},
+                    {'11': {revision: 'webkit-revision-2', ownerRevision: null, patch: null},
+                        '22': { revision: 'ios-revision-2', ownerRevision: null, patch: null}}]}
+            );
+        });
+
+        it('should create analysis task and test groups with "needsNotification" set to false if specified in creation', async () => {
+            const [startPoint, endPoint] = mockStartAndEndPoints();
+            AnalysisTask.create('confirm', startPoint, endPoint, 'Confirm', 4, false);
+            assert.equal(requests.length, 1);
+            assert.equal(requests[0].url, '/privileged-api/generate-csrf-token');
+            requests[0].resolve({
+                token: 'abc',
+                expiration: Date.now() + 3600 * 1000,
+            });
+
+            await MockRemoteAPI.waitForRequest();
+            assert.equal(requests[1].url, '/privileged-api/create-analysis-task');
+            assert.equal(requests.length, 2);
+            assert.deepEqual(requests[1].data, {name: 'confirm', repetitionCount: 4, needsNotification: false,
                 startRun: 1, endRun: 2, testGroupName: 'Confirm', token: 'abc', revisionSets: [
                     {'11': {revision: 'webkit-revision-1', ownerRevision: null, patch: null},
                         '22': {revision: 'ios-revision-1', ownerRevision: null, patch: null}},
@@ -326,7 +348,7 @@ describe('AnalysisTask', () => {
 
         it('should sync the new analysis task status once it is created', async () => {
             const [startPoint, endPoint] = mockStartAndEndPoints();
-            const creatingPromise = AnalysisTask.create('confirm', startPoint, endPoint, 'Confirm', 4);
+            const creatingPromise = AnalysisTask.create('confirm', startPoint, endPoint, 'Confirm', 4, true);
             assert.equal(requests.length, 1);
             assert.equal(requests[0].url, '/privileged-api/generate-csrf-token');
             requests[0].resolve({
@@ -337,7 +359,7 @@ describe('AnalysisTask', () => {
             await MockRemoteAPI.waitForRequest();
             assert.equal(requests[1].url, '/privileged-api/create-analysis-task');
             assert.equal(requests.length, 2);
-            assert.deepEqual(requests[1].data, {name: 'confirm', repetitionCount: 4,
+            assert.deepEqual(requests[1].data, {name: 'confirm', repetitionCount: 4, needsNotification: true,
                 startRun: 1, endRun: 2, testGroupName: 'Confirm', token: 'abc', revisionSets: [
                     {'11': {revision: 'webkit-revision-1', ownerRevision: null, patch: null},
                         '22': {revision: 'ios-revision-1', ownerRevision: null, patch: null}},
@@ -387,7 +409,7 @@ describe('AnalysisTask', () => {
 
         it('should return an rejected promise when analysis task creation failed', async () => {
             const [startPoint, endPoint] = mockStartAndEndPoints();
-            const creatingPromise = AnalysisTask.create('confirm', startPoint, endPoint, 'Confirm', 4);
+            const creatingPromise = AnalysisTask.create('confirm', startPoint, endPoint, 'Confirm', 4, true);
             assert.equal(requests.length, 1);
             assert.equal(requests[0].url, '/privileged-api/generate-csrf-token');
             requests[0].resolve({
@@ -398,7 +420,7 @@ describe('AnalysisTask', () => {
             await MockRemoteAPI.waitForRequest();
             assert.equal(requests[1].url, '/privileged-api/create-analysis-task');
             assert.equal(requests.length, 2);
-            assert.deepEqual(requests[1].data, {name: 'confirm', repetitionCount: 4,
+            assert.deepEqual(requests[1].data, {name: 'confirm', repetitionCount: 4, needsNotification: true,
                 startRun: 1, endRun: 2, testGroupName: 'Confirm', token: 'abc', revisionSets: [
                     {'11': {revision: 'webkit-revision-1', ownerRevision: null, patch: null},
                         '22': {revision: 'ios-revision-1', ownerRevision: null, patch: null}},
@@ -432,10 +454,10 @@ describe('AnalysisTask', () => {
 
         it('should create analysis task with confirming repetition count specified', () => {
             const [startPoint, endPoint] = mockStartAndEndPoints();
-            AnalysisTask.create('confirm', startPoint, endPoint, 'Confirm', 4);
+            AnalysisTask.create('confirm', startPoint, endPoint, 'Confirm', 4, true);
             assert.equal(requests[0].url, '/privileged-api/create-analysis-task');
             assert.equal(requests.length, 1);
-            assert.deepEqual(requests[0].data, {name: 'confirm', repetitionCount: 4,
+            assert.deepEqual(requests[0].data, {name: 'confirm', repetitionCount: 4, needsNotification: true,
                 startRun: 1, endRun: 2, slaveName: 'worker', slavePassword: 'password',
                 testGroupName: 'Confirm', revisionSets: [
                     {'11': {revision: 'webkit-revision-1', ownerRevision: null, patch: null},
index 6a89f35..aa50081 100644 (file)
@@ -189,6 +189,7 @@ describe('MeasurementSetAnalyzer', () => {
                 endRun: 6443,
                 repetitionCount: 4,
                 testGroupName: 'Confirm',
+                needsNotification: true,
                 revisionSets: [{'11': {revision: 35, ownerRevision: null, patch: null}},
                     {'11': {revision: 44, ownerRevision: null, patch: null}}]
             });
@@ -242,6 +243,7 @@ describe('MeasurementSetAnalyzer', () => {
                 endRun: 6443,
                 repetitionCount: 4,
                 testGroupName: 'Confirm',
+                needsNotification: true,
                 revisionSets: [{'11': {revision: 35, ownerRevision: null, patch: null}},
                     {'11': {revision: 44, ownerRevision: null, patch: null}}]
             });
@@ -386,6 +388,7 @@ describe('MeasurementSetAnalyzer', () => {
                 endRun: 6448,
                 repetitionCount: 4,
                 testGroupName: 'Confirm',
+                needsNotification: true,
                 revisionSets: [{'11': {revision: 40, ownerRevision: null, patch: null}},
                     {'11': {revision: 49, ownerRevision: null, patch: null}}]
             });
@@ -472,6 +475,7 @@ describe('MeasurementSetAnalyzer', () => {
                 endRun: 6407,
                 repetitionCount: 4,
                 testGroupName: 'Confirm',
+                needsNotification: true,
                 revisionSets: [{'11': {revision: 3, ownerRevision: null, patch: null}},
                     {'11': {revision: 8, ownerRevision: null, patch: null}}]
             });
index be43fee..2f451e4 100644 (file)
@@ -69,14 +69,14 @@ var MockRemoteAPI = {
             global.RemoteAPI = MockRemoteAPI;
             originalPrivilegedAPI = global.PrivilegedAPI;
             global.PrivilegedAPI = privilegedAPI;
-            if (privilegedAPI._token)
+            if (privilegedAPI && privilegedAPI._token)
                 privilegedAPI._token = null;
         });
 
         afterEach(() => {
             global.RemoteAPI = originalRemoteAPI;
             global.PrivilegedAPI = originalPrivilegedAPI;
-            if (privilegedAPI._token)
+            if (privilegedAPI && privilegedAPI._token)
                 privilegedAPI._token = null;
         });
 
index de59652..fb89162 100644 (file)
@@ -3,10 +3,11 @@
 const assert = require('assert');
 require('../tools/js/v3-models.js');
 const BrowserPrivilegedAPI = require('../public/v3/privileged-api.js').PrivilegedAPI;
+const NodePrivilegedAPI = require('../tools/js/privileged-api.js').PrivilegedAPI;
 const MockModels = require('./resources/mock-v3-models.js').MockModels;
 const MockRemoteAPI = require('./resources/mock-remote-api.js').MockRemoteAPI;
 
-function sampleTestGroup() {
+function sampleTestGroup(needsNotification=true) {
     return {
         "testGroups": [{
             "id": "2128",
@@ -16,6 +17,7 @@ function sampleTestGroup() {
             "author": "rniwa",
             "createdAt": 1458688514000,
             "hidden": false,
+            "needsNotification": needsNotification,
             "buildRequests": ["16985", "16986", "16987", "16988", "16989", "16990", "16991", "16992"],
             "commitSets": ["4255", "4256"],
         }],
@@ -137,6 +139,41 @@ describe('TestGroup', function () {
         });
     });
 
+    describe('needsNotification', () => {
+        const requests = MockRemoteAPI.inject('https://perf.webkit.org', NodePrivilegedAPI);
+        beforeEach(() => {
+            PrivilegedAPI.configure('test', 'password');
+        });
+
+        it('should update notified author flag', async () => {
+            const fetchPromise = TestGroup.fetchForTask(1376);
+            requests[0].resolve(sampleTestGroup());
+            let testGroups = await fetchPromise;
+            assert(testGroups.length, 1);
+            let testGroup = testGroups[0];
+            assert.equal(testGroup.needsNotification(), true);
+
+            const updatePromise = testGroup.didSendNotification();
+            assert.equal(requests.length, 2);
+            assert.equal(requests[1].method, 'POST');
+            assert.equal(requests[1].url, '/privileged-api/update-test-group');
+            delete requests[1].data.notificationSentAt;
+            assert.deepEqual(requests[1].data, {group: '2128', needsNotification: false, slaveName: 'test', slavePassword: 'password'});
+            requests[1].resolve();
+
+            await MockRemoteAPI.waitForRequest();
+            assert.equal(requests.length, 3);
+            assert.equal(requests[2].method, 'GET');
+            assert.equal(requests[2].url, '/api/test-groups/2128');
+            const updatedTestGroup = sampleTestGroup(false);
+            requests[2].resolve(updatedTestGroup);
+
+            testGroups = await updatePromise;
+            testGroup = testGroups[0];
+            assert.equal(testGroup.needsNotification(), false);
+        });
+    });
+
     describe('_createModelsFromFetchedTestGroups', function () {
         it('should create test groups', function () {
             var groups = TestGroup._createModelsFromFetchedTestGroups(sampleTestGroup());
@@ -147,6 +184,7 @@ describe('TestGroup', function () {
             assert.equal(group.id(), 2128);
             assert.ok(group.createdAt() instanceof Date);
             assert.equal(group.isHidden(), false);
+            assert.equal(group.needsNotification(), true);
             assert.equal(+group.createdAt(), 1458688514000);
             assert.equal(group.repetitionCount(), sampleTestGroup()['buildRequests'].length / 2);
             assert.ok(group.hasPending());
@@ -361,7 +399,7 @@ describe('TestGroup', function () {
             set2.setRevisionForRepository(MockModels.webkit, '191623');
             set2.setRevisionForRepository(MockModels.sharedRepository, '80229');
 
-            const promise = TestGroup.createWithTask('some-task', MockModels.somePlatform, MockModels.someTest, 'some-group', 4, [set1, set2]);
+            const promise = TestGroup.createWithTask('some-task', MockModels.somePlatform, MockModels.someTest, 'some-group', 4, [set1, set2], true);
             assert.equal(requests.length, 2);
             assert.equal(requests[1].url, '/privileged-api/generate-csrf-token');
             requests[1].resolve({
@@ -371,6 +409,10 @@ describe('TestGroup', function () {
             await MockRemoteAPI.waitForRequest();
             assert.equal(requests.length, 3);
             assert.equal(requests[2].method, 'POST');
+            assert.deepEqual(requests[2].data, {taskName: 'some-task', name: 'some-group', platform: 65, test: 1,
+                repetitionCount: 4, revisionSets: [{'11': {ownerRevision: null, patch: null, revision: "191622"},
+                    '16': {ownerRevision: null, patch: null, revision: "80229"}}, {'11': {ownerRevision: null, patch: null, revision: "191623"},
+                    '16': {ownerRevision: null, patch: null, revision: "80229"}}], needsNotification: true, token: 'abc'});
             assert.equal(requests[2].url, '/privileged-api/create-test-group');
             requests[2].resolve({
                 taskId: 123,