Analysis task pages are unusable
authorrniwa@webkit.org <rniwa@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 19 Feb 2015 02:41:45 +0000 (02:41 +0000)
committerrniwa@webkit.org <rniwa@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 19 Feb 2015 02:41:45 +0000 (02:41 +0000)
https://bugs.webkit.org/show_bug.cgi?id=141786

Reviewed by Andreas Kling.

This patch makes following improvements to analysis task pages:
1. Making the main chart interactive. This change required the use of App.Pane as well as moving the code to
compute the data for the details pane from PaneController.
2. Moving the form to add a new test group to the top of test groups instead of the bottom of them.
3. Grouping the build requests in each test group by root sets instead of the order by which they were ran.
This change required the creation of App.TestGroupPane as well as its methods.
4. Show a box plot for each root set configuration as well as each build request. This change required
App.BoxPlotComponent.
5. Show revisions of each repository (e.g. WebKit) for each root set and build request.

* public/api/build-requests.php:
(main): Update per the rename of BuildRequestsFetcher::root_sets to BuildRequestsFetcher::root_sets_by_id.

* public/api/test-groups.php:
(main): Include root sets and roots in the response.
(format_test_group):

* public/include/build-requests-fetcher.php:
(BuildRequestsFetcher::root_sets_by_id): Renamed from root_sets.
(BuildRequestsFetcher::root_sets): Added.
(BuildRequestsFetcher::roots): Added.
(BuildRequestsFetcher::fetch_roots_for_set): Takes a boolean argument $resolve_ids. This flag is only set to
true in /api/build-requests/ (as done prior to this patch) to use repository names as identifiers since
tools/sync-with-buildbot.py can't convert repository names to their ids.

* public/v2/analysis.js:
(App.Root): Added.
(App.RootSet): Added.
(App.RootSet.revisionForRepository): Added.
(App.TestGroup.rootSets): Deleted the code to compute root set ids from build requests now that the JSON
response at /api/test-groups will include them.
(App.BuildRequest): Ditto. Also deleted 'configLetter' property, which has been moved to a proxy created by
_createConfigurationSummary.
(App.BuildRequest.statusLabel): Use 'Completed' as the human readable label for 'completed' status.
(App.BuildRequest.aggregateStatuses): Added. Generates a human readable status for a set of build requests.

* public/v2/app.css: Updated style rules for analysis task pages.

* public/v2/app.js:
(App.Pane): This class is now used in analysis task pages to make the main chart interactive.
(App.Pane._updateDetails): Moved from App.PaneController.

(App.PaneController._updateCanAnalyze): Updated the code per the move of selectedPoints.

(App.AnalysisTaskController): Added 'details'.
(App.AnalysisTaskController._taskUpdated):
(App.AnalysisTaskController.paneDomain):Renamed from _fetchedRuns.
(App.AnalysisTaskController.updateTestGroupPanes): Added. Creates App.TestGroupPane for each test group.
(App.AnalysisTaskController.actions.toggleShowRequestList): Added.

(App.TestGroupPane): Added.
(App.TestGroupPane._populate): Added. Group build requests by root sets and create a summary for each group.
(App.TestGroupPane._computeRepositoryList): Added. Returns a sorted list of repositories which is the union
of all repositories appearing in root sets and builds associated with A/B testing results.
(App.TestGroupPane._groupRequestsByConfigurations): Added. Groups build requests by root sets.
(App.TestGroupPane._createConfigurationSummary): Added. Creates a summary for a group of build requests that
use the same root set. We start by wrapping "raw" build requests in a proxy with formatted values,
build numbers, etc... obtained from the fetched chart data. The list of revisions shown in the group summary
is a union of revisions in the root set and the first build request in the group. We null-out revision info
for a build request if it is identical to the one in the summary. The range of values is expanded as needed
by the values in the group as well as 95% percentile confidence interval.

(App.BoxPlotComponent): Added. Controls a box plot shown for each test group summary and build request.
(App.BoxPlotComponent.didInsertElement): Added. Inserts a SVG element as well as two indicator rects to show
the mean and the confidence interval.
(App.BoxPlotComponent._updateBars): Added. Updates the dimensions of the indicator rects.
(App.BoxPlotComponent.valueChanged): Added. Computes the relative dimensions of the indicator rects and
calls _updateBars to update the rects.

* public/v2/chart-pane.css: Added some style rules to be used in the details pane in analysis task pages.

* public/v2/data.js:
(Measurement.prototype.formattedRevisions):
(Measurement.formatRevisionRange): Renamed from Measurement.prototype._formatRevisionRange so that it can be
called in _createConfigurationSummary.

* public/v2/index.html: Updated the templates for analysis task pages. Moved the form to create a new test
group above all test groups, and replaced the list of data points by "details" pane used in the charts page.
Also made the fetching of chartData no longer block showing of test groups.

* public/v2/interactive-chart.js:
(App.InteractiveChartComponent._updateDomain): Added an early exit to fix a newly revealed race condition.
(App.InteractiveChartComponent._domainChanged): Ditto.
(App.InteractiveChartComponent._updateSelectionToolbar): Made it respect 'zoomable' boolean property.

* public/v2/js/statistics.js:
(Statistics.min): Added.
(Statistics.max): Added.

* public/v2/manifest.js:
(App.Manifest.fetchRunsWithPlatformAndMetric): Added formatWithDeltaAndUnit to be used in _createConfigurationSummary.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@180333 268f45cc-cd09-0410-ab3c-d52691b4dbfc

13 files changed:
Websites/perf.webkit.org/ChangeLog
Websites/perf.webkit.org/public/api/build-requests.php
Websites/perf.webkit.org/public/api/test-groups.php
Websites/perf.webkit.org/public/include/build-requests-fetcher.php
Websites/perf.webkit.org/public/v2/analysis.js
Websites/perf.webkit.org/public/v2/app.css
Websites/perf.webkit.org/public/v2/app.js
Websites/perf.webkit.org/public/v2/chart-pane.css
Websites/perf.webkit.org/public/v2/data.js
Websites/perf.webkit.org/public/v2/index.html
Websites/perf.webkit.org/public/v2/interactive-chart.js
Websites/perf.webkit.org/public/v2/js/statistics.js
Websites/perf.webkit.org/public/v2/manifest.js

index c5d0d71..9e1c6c7 100644 (file)
@@ -1,3 +1,102 @@
+2015-02-18  Ryosuke Niwa  <rniwa@webkit.org>
+
+        Analysis task pages are unusable
+        https://bugs.webkit.org/show_bug.cgi?id=141786
+
+        Reviewed by Andreas Kling.
+
+        This patch makes following improvements to analysis task pages:
+        1. Making the main chart interactive. This change required the use of App.Pane as well as moving the code to
+        compute the data for the details pane from PaneController.
+        2. Moving the form to add a new test group to the top of test groups instead of the bottom of them.
+        3. Grouping the build requests in each test group by root sets instead of the order by which they were ran.
+        This change required the creation of App.TestGroupPane as well as its methods.
+        4. Show a box plot for each root set configuration as well as each build request. This change required
+        App.BoxPlotComponent.
+        5. Show revisions of each repository (e.g. WebKit) for each root set and build request.
+
+        * public/api/build-requests.php:
+        (main): Update per the rename of BuildRequestsFetcher::root_sets to BuildRequestsFetcher::root_sets_by_id.
+
+        * public/api/test-groups.php:
+        (main): Include root sets and roots in the response.
+        (format_test_group):
+
+        * public/include/build-requests-fetcher.php:
+        (BuildRequestsFetcher::root_sets_by_id): Renamed from root_sets.
+        (BuildRequestsFetcher::root_sets): Added.
+        (BuildRequestsFetcher::roots): Added.
+        (BuildRequestsFetcher::fetch_roots_for_set): Takes a boolean argument $resolve_ids. This flag is only set to
+        true in /api/build-requests/ (as done prior to this patch) to use repository names as identifiers since
+        tools/sync-with-buildbot.py can't convert repository names to their ids.
+
+        * public/v2/analysis.js:
+        (App.Root): Added.
+        (App.RootSet): Added.
+        (App.RootSet.revisionForRepository): Added.
+        (App.TestGroup.rootSets): Deleted the code to compute root set ids from build requests now that the JSON
+        response at /api/test-groups will include them.
+        (App.BuildRequest): Ditto. Also deleted 'configLetter' property, which has been moved to a proxy created by
+        _createConfigurationSummary.
+        (App.BuildRequest.statusLabel): Use 'Completed' as the human readable label for 'completed' status.
+        (App.BuildRequest.aggregateStatuses): Added. Generates a human readable status for a set of build requests.
+
+        * public/v2/app.css: Updated style rules for analysis task pages.
+
+        * public/v2/app.js:
+        (App.Pane): This class is now used in analysis task pages to make the main chart interactive.
+        (App.Pane._updateDetails): Moved from App.PaneController.
+
+        (App.PaneController._updateCanAnalyze): Updated the code per the move of selectedPoints.
+
+        (App.AnalysisTaskController): Added 'details'.
+        (App.AnalysisTaskController._taskUpdated):
+        (App.AnalysisTaskController.paneDomain):Renamed from _fetchedRuns.
+        (App.AnalysisTaskController.updateTestGroupPanes): Added. Creates App.TestGroupPane for each test group.
+        (App.AnalysisTaskController.actions.toggleShowRequestList): Added.
+
+        (App.TestGroupPane): Added.
+        (App.TestGroupPane._populate): Added. Group build requests by root sets and create a summary for each group.
+        (App.TestGroupPane._computeRepositoryList): Added. Returns a sorted list of repositories which is the union
+        of all repositories appearing in root sets and builds associated with A/B testing results.
+        (App.TestGroupPane._groupRequestsByConfigurations): Added. Groups build requests by root sets.
+        (App.TestGroupPane._createConfigurationSummary): Added. Creates a summary for a group of build requests that
+        use the same root set. We start by wrapping "raw" build requests in a proxy with formatted values,
+        build numbers, etc... obtained from the fetched chart data. The list of revisions shown in the group summary
+        is a union of revisions in the root set and the first build request in the group. We null-out revision info
+        for a build request if it is identical to the one in the summary. The range of values is expanded as needed
+        by the values in the group as well as 95% percentile confidence interval.
+
+        (App.BoxPlotComponent): Added. Controls a box plot shown for each test group summary and build request.
+        (App.BoxPlotComponent.didInsertElement): Added. Inserts a SVG element as well as two indicator rects to show
+        the mean and the confidence interval.
+        (App.BoxPlotComponent._updateBars): Added. Updates the dimensions of the indicator rects.
+        (App.BoxPlotComponent.valueChanged): Added. Computes the relative dimensions of the indicator rects and
+        calls _updateBars to update the rects.
+
+        * public/v2/chart-pane.css: Added some style rules to be used in the details pane in analysis task pages.
+
+        * public/v2/data.js:
+        (Measurement.prototype.formattedRevisions):
+        (Measurement.formatRevisionRange): Renamed from Measurement.prototype._formatRevisionRange so that it can be
+        called in _createConfigurationSummary.
+
+        * public/v2/index.html: Updated the templates for analysis task pages. Moved the form to create a new test
+        group above all test groups, and replaced the list of data points by "details" pane used in the charts page.
+        Also made the fetching of chartData no longer block showing of test groups.
+
+        * public/v2/interactive-chart.js:
+        (App.InteractiveChartComponent._updateDomain): Added an early exit to fix a newly revealed race condition.
+        (App.InteractiveChartComponent._domainChanged): Ditto.
+        (App.InteractiveChartComponent._updateSelectionToolbar): Made it respect 'zoomable' boolean property.
+
+        * public/v2/js/statistics.js:
+        (Statistics.min): Added.
+        (Statistics.max): Added.
+
+        * public/v2/manifest.js:
+        (App.Manifest.fetchRunsWithPlatformAndMetric): Added formatWithDeltaAndUnit to be used in _createConfigurationSummary.
+
 2015-02-14  Ryosuke Niwa  <rniwa@webkit.org>
 
         Build URL on new perf dashboard doesn't resolve $builderName
index 71a9373..bcad292 100644 (file)
@@ -45,7 +45,7 @@ function main($path, $post_data) {
 
     exit_with_success(array(
         'buildRequests' => $requests_fetcher->results_with_resolved_ids(),
-        'rootSets' => $requests_fetcher->root_sets(),
+        'rootSets' => $requests_fetcher->root_sets_by_id(),
         'updates' => $updates,
     ));
 }
index 088ff62..f822202 100644 (file)
@@ -39,10 +39,16 @@ function main($path) {
         $group_by_id[$group['id']] = &$group;
 
     $build_requests = $build_requests_fetcher->results();
-    foreach ($build_requests as $request)
-        array_push($group_by_id[$request['testGroup']]['buildRequests'], $request['id']);
+    foreach ($build_requests as $request) {
+        $request_group = &$group_by_id[$request['testGroup']];
+        array_push($request_group['buildRequests'], $request['id']);
+        array_push($request_group['rootSets'], $request['rootSet']);
+    }
 
-    exit_with_success(array('testGroups' => $test_groups, 'buildRequests' => $build_requests));
+    exit_with_success(array('testGroups' => $test_groups,
+        'buildRequests' => $build_requests,
+        'rootSets' => $build_requests_fetcher->root_sets(),
+        'roots' => $build_requests_fetcher->roots()));
 }
 
 function format_test_group($group_row) {
@@ -53,6 +59,7 @@ function format_test_group($group_row) {
         'author' => $group_row['testgroup_author'],
         'createdAt' => strtotime($group_row['testgroup_created_at']) * 1000,
         'buildRequests' => array(),
+        'rootSets' => array(),
     );
 }
 
index ce52cba..5dd9751 100644 (file)
@@ -6,6 +6,8 @@ class BuildRequestsFetcher {
     function __construct($db) {
         $this->db = $db;
         $this->rows = null;
+        $this->root_sets = array();
+        $this->roots = array();
         $this->root_sets_by_id = array();
     }
 
@@ -50,7 +52,7 @@ class BuildRequestsFetcher {
             $root_set_id = $row['request_root_set'];
 
             if (!array_key_exists($root_set_id, $this->root_sets_by_id))
-                $this->root_sets_by_id[$root_set_id] = $this->fetch_roots_for_set($root_set_id);
+                $this->root_sets_by_id[$root_set_id] = $this->fetch_roots_for_set($root_set_id, $resolve_ids);
 
             array_push($requests, array(
                 'id' => $row['request_id'],
@@ -69,18 +71,34 @@ class BuildRequestsFetcher {
         return $requests;
     }
 
-    function root_sets() {
+    function root_sets_by_id() {
         return $this->root_sets_by_id;
     }
 
-    private function fetch_roots_for_set($root_set_id) {
+    function root_sets() {
+        return $this->root_sets;
+    }
+
+    function roots() {
+        return $this->roots;
+    }
+
+    private function fetch_roots_for_set($root_set_id, $resolve_ids) {
         $root_rows = $this->db->query_and_fetch_all('SELECT *
             FROM roots, commits LEFT OUTER JOIN repositories ON commit_repository = repository_id
             WHERE root_commit = commit_id AND root_set = $1', array($root_set_id));
 
         $roots = array();
-        foreach ($root_rows as $row)
-            $roots[$row['repository_name']] = $row['commit_revision'];
+        $root_ids = array();
+        foreach ($root_rows as $row) {
+            $repository = $row['repository_id'];
+            $revision = $row['commit_revision'];
+            $root_id = $root_set_id . '-' . $repository;
+            array_push($root_ids, $root_id);
+            array_push($this->roots, array('id' => $root_id, 'repository' => $repository, 'revision' => $revision));
+            $roots[$resolve_ids ? $row['repository_name'] : $row['repository_id']] = $revision;
+        }
+        array_push($this->root_sets, array('id' => $root_set_id, 'roots' => $root_ids));
 
         return $roots;
     }
index 82e045f..98bc74a 100644 (file)
@@ -65,21 +65,28 @@ App.BugAdapter = DS.RESTAdapter.extend({
     }
 });
 
+App.Root = App.Model.extend({
+    repository: DS.belongsTo('repository'),
+    revision: DS.attr('string'),
+});
+
+App.RootSet = App.Model.extend({
+    roots: DS.hasMany('roots'),
+    revisionForRepository: function (repository)
+    {
+        var root = this.get('roots').findBy('repository', repository);
+        if (!root)
+            return null;
+        return root.get('revision');
+    }
+});
+
 App.TestGroup = App.NameLabelModel.extend({
     task: DS.belongsTo('analysisTask'),
     author: DS.attr('string'),
     createdAt: DS.attr('date'),
     buildRequests: DS.hasMany('buildRequests'),
-    rootSets: function ()
-    {
-        var rootSetIds = [];
-        this.get('buildRequests').forEach(function (request) {
-            var rootSet = request.get('rootSet');
-            if (!rootSetIds.contains(rootSet))
-                rootSetIds.push(rootSet);
-        });
-        return rootSetIds;
-    }.property('buildRequests'),
+    rootSets: DS.hasMany('rootSets'),
     _fetchChartData: function ()
     {
         var task = this.get('task');
@@ -144,13 +151,7 @@ App.BuildRequest = App.Model.extend({
     {
         return this.get('order') + 1;
     }.property('order'),
-    rootSet: DS.attr('number'),
-    configLetter: function ()
-    {
-        var rootSets = this.get('testGroup').get('rootSets');
-        var index = rootSets.indexOf(this.get('rootSet'));
-        return String.fromCharCode('A'.charCodeAt(0) + index);
-    }.property('testGroup', 'testGroup.rootSets'),
+    rootSet: DS.belongsTo('rootSet'),
     status: DS.attr('string'),
     statusLabel: function ()
     {
@@ -164,24 +165,33 @@ App.BuildRequest = App.Model.extend({
         case 'failed':
             return 'Failed';
         case 'completed':
-            return 'Finished';
+            return 'Completed';
         }
     }.property('status'),
     url: DS.attr('string'),
     build: DS.attr('number'),
-    _fetchMean: function ()
-    {
-        var testGroup = this.get('testGroup');
-        if (!testGroup)
-            return;
-        var chartData = testGroup.get('chartData');
-        if (!chartData)
-            return;
-
-        var point = chartData.current.findPointByBuild(this.get('build'));
-        if (!point)
-            return;
-        this.set('mean', chartData.formatter(point.value) + (chartData.unit ? ' ' + chartData.unit : ''));
-        this.set('buildNumber', point.measurement.buildNumber());
-    }.observes('build', 'testGroup', 'testGroup.chartData').on('init'),
 });
+
+App.BuildRequest.aggregateStatuses = function (requests)
+{
+    var completeCount = 0;
+    var failureCount = 0;
+    requests.forEach(function (request) {
+        switch (request.get('status')) {
+        case 'failed':
+            failureCount++;
+            break;
+        case 'completed':
+            completeCount++;
+            break;
+        }
+    });
+    if (completeCount == requests.length)
+        return 'Done';
+    if (failureCount == requests.length)
+        return 'All failed';
+    var status = completeCount + ' out of ' + requests.length + ' completed';
+    if (failureCount)
+        status += ', ' + failureCount + ' failed';
+    return status;
+}
index fa3e08a..b7eca0a 100755 (executable)
@@ -451,6 +451,49 @@ table.dashboard tbody td .failure {
     border-top: solid 1px #ddd;
 }
 
+.analysis-group .results .summary td {
+    vertical-align: top;
+}
+
+.analysis-group .results thead td {
+    text-align: center;
+}
+
+.analysis-group .results .config-letter,
+.analysis-group .results .summary {
+    cursor: pointer;
+}
+
+.analysis-group .results .request .config-letter {
+    border-color: transparent;
+}
+
+.analysis-group .results .hideRequests .request {
+    display: none;
+}
+
+.box-plot {
+    display: inline-block;
+    width: 100px;
+    height: 0.6rem;
+    border: solid 1px #ddd;
+    padding: 1px;
+    vertical-align: middle;
+}
+
+.box-plot .percentage {
+    fill: #ccc;
+}
+
+.box-plot .delta {
+    fill: #333;
+    opacity: 0.5;
+}
+
+.box-plot svg {
+    display: block;
+}
+
 #analysis-task-title {
     font-weight: normal;
     font-size: 1.2rem;
@@ -470,15 +513,27 @@ table.dashboard tbody td .failure {
     border: 1px solid #bbb;
     border-radius: 0.5rem;
     box-shadow: rgba(0, 0, 0, 0.03) 1px 1px 0px 0px;
-
-    padding: 0.5rem 1rem;
     margin-bottom: 1.5rem;
 }
 
-.analysis-group caption {
+.analysis-group > * {
+    margin: 0.5rem;
+}
+
+.analysis-group > h1 {
     font-size: 1.1rem;
+    font-weight: normal;
     text-align: left;
     margin-bottom: 0.5rem;
+    border-bottom: 1px solid #bbb;
+    margin: 0;
+    padding: 0.2rem 0.5rem;
+}
+
+.analysis-group h1 > input {
+    font-size: 1rem;
+    min-width: 20rem;
+    margin: 0.2rem 0;
 }
 
 .analysis-bugs th {
index d3e36f9..50c5c18 100755 (executable)
@@ -297,6 +297,8 @@ App.Pane = Ember.Object.extend({
     metricId: null,
     metric: null,
     selectedItem: null,
+    selectedPoints: null,
+    hoveredOrSelectedItem: null,
     showFullYAxis: false,
     searchCommit: function (repository, keyword) {
         var self = this;
@@ -538,6 +540,56 @@ App.Pane = Ember.Object.extend({
         if (JSON.stringify(config) != JSON.stringify(this.get(configName)))
             this.set(configName, config);
     },
+    _updateDetails: function ()
+    {
+        var selectedPoints = this.get('selectedPoints');
+        var currentPoint = this.get('hoveredOrSelectedItem');
+        if (!selectedPoints && !currentPoint) {
+            this.set('details', null);
+            return;
+        }
+
+        var currentMeasurement;
+        var previousPoint;
+        if (!selectedPoints)
+            previousPoint = currentPoint.series.previousPoint(currentPoint);
+        else {
+            currentPoint = selectedPoints[selectedPoints.length - 1];
+            previousPoint = selectedPoints[0];
+        }
+        var currentMeasurement = currentPoint.measurement;
+        var oldMeasurement = previousPoint ? previousPoint.measurement : null;
+
+        var formattedRevisions = currentMeasurement.formattedRevisions(oldMeasurement);
+        var revisions = App.Manifest.get('repositories')
+            .filter(function (repository) { return formattedRevisions[repository.get('id')]; })
+            .map(function (repository) {
+            var revision = Ember.Object.create(formattedRevisions[repository.get('id')]);
+            revision['url'] = revision.previousRevision
+                ? repository.urlForRevisionRange(revision.previousRevision, revision.currentRevision)
+                : repository.urlForRevision(revision.currentRevision);
+            revision['name'] = repository.get('name');
+            revision['repository'] = repository;
+            return revision; 
+        });
+
+        var buildNumber = null;
+        var buildURL = null;
+        if (!selectedPoints) {
+            buildNumber = currentMeasurement.buildNumber();
+            var builder = App.Manifest.builder(currentMeasurement.builderId());
+            if (builder)
+                buildURL = builder.urlFromBuildNumber(buildNumber);
+        }
+
+        this.set('details', Ember.Object.create({
+            status: this.computeStatus(currentPoint, previousPoint),
+            buildNumber: buildNumber,
+            buildURL: buildURL,
+            buildTime: currentMeasurement.formattedBuildTime(),
+            revisions: revisions,
+        }));
+    }.observes('hoveredOrSelectedItem', 'selectedPoints'),
 });
 
 App.encodePrettifiedJSON = function (plain)
@@ -900,65 +952,13 @@ App.PaneController = Ember.ObjectController.extend({
         this.set('mainPlotDomain', newSelection || this.get('overviewDomain'));
         this.set('overviewSelection', newSelection);
     }.observes('parentController.sharedZoom').on('init'),
-    _updateDetails: function ()
-    {
-        var selectedPoints = this.get('selectedPoints');
-        var currentPoint = this.get('currentItem');
-        if (!selectedPoints && !currentPoint) {
-            this.set('details', null);
-            return;
-        }
-
-        var currentMeasurement;
-        var previousPoint;
-        if (!selectedPoints)
-            previousPoint = currentPoint.series.previousPoint(currentPoint);
-        else {
-            currentPoint = selectedPoints[selectedPoints.length - 1];
-            previousPoint = selectedPoints[0];
-        }
-        var currentMeasurement = currentPoint.measurement;
-        var oldMeasurement = previousPoint ? previousPoint.measurement : null;
-
-        var formattedRevisions = currentMeasurement.formattedRevisions(oldMeasurement);
-        var revisions = App.Manifest.get('repositories')
-            .filter(function (repository) { return formattedRevisions[repository.get('id')]; })
-            .map(function (repository) {
-            var revision = Ember.Object.create(formattedRevisions[repository.get('id')]);
-            revision['url'] = revision.previousRevision
-                ? repository.urlForRevisionRange(revision.previousRevision, revision.currentRevision)
-                : repository.urlForRevision(revision.currentRevision);
-            revision['name'] = repository.get('name');
-            revision['repository'] = repository;
-            return revision; 
-        });
-
-        var buildNumber = null;
-        var buildURL = null;
-        if (!selectedPoints) {
-            buildNumber = currentMeasurement.buildNumber();
-            var builder = App.Manifest.builder(currentMeasurement.builderId());
-            if (builder)
-                buildURL = builder.urlFromBuildNumber(buildNumber);
-        }
-
-        this.set('details', Ember.Object.create({
-            status: this.get('model').computeStatus(currentPoint, previousPoint),
-            buildNumber: buildNumber,
-            buildURL: buildURL,
-            buildTime: currentMeasurement.formattedBuildTime(),
-            revisions: revisions,
-        }));
-        this._updateCanAnalyze();
-    }.observes('currentItem', 'selectedPoints'),
     _updateCanAnalyze: function ()
     {
-        var points = this.get('selectedPoints');
+        var points = this.get('model').get('selectedPoints');
         this.set('cannotAnalyze', !this.get('newAnalysisTaskName') || !points || points.length < 2);
-    }.observes('newAnalysisTaskName'),
+    }.observes('newAnalysisTaskName', 'model.selectedPoints'),
 });
 
-
 App.AnalysisRoute = Ember.Route.extend({
     model: function () {
         return this.store.findAll('analysisTask').then(function (tasks) {
@@ -978,7 +978,7 @@ App.AnalysisTaskController = Ember.Controller.extend({
     label: Ember.computed.alias('model.name'),
     platform: Ember.computed.alias('model.platform'),
     metric: Ember.computed.alias('model.metric'),
-    testGroups: Ember.computed.alias('model.testGroups'),
+    details: Ember.computed.alias('pane.details'),
     testSets: [],
     roots: [],
     bugTrackers: [],
@@ -989,10 +989,12 @@ App.AnalysisTaskController = Ember.Controller.extend({
         if (!model)
             return;
 
-        var platformId = model.get('platform').get('id');
-        var metricId = model.get('metric').get('id');
         App.Manifest.fetch(this.store).then(this._fetchedManifest.bind(this));
-        App.Manifest.fetchRunsWithPlatformAndMetric(this.store, platformId, metricId).then(this._fetchedRuns.bind(this));
+        this.set('pane', App.Pane.create({
+            store: this.store,
+            platformId: model.get('platform').get('id'),
+            metricId: model.get('metric').get('id'),
+        }));
     }.observes('model').on('init'),
     _fetchedManifest: function ()
     {
@@ -1010,17 +1012,24 @@ App.AnalysisTaskController = Ember.Controller.extend({
             });
         }));
     },
-    _fetchedRuns: function (result)
+    paneDomain: function ()
     {
-        var chartData = result.data;
+        var pane = this.get('pane');
+        if (!pane)
+            return;
+
+        var chartData = pane.get('chartData');
+        if (!chartData)
+            return null;
+
         var currentTimeSeries = chartData.current;
         if (!currentTimeSeries)
-            return; // FIXME: Report an error.
+            return null; // FIXME: Report an error.
 
         var start = currentTimeSeries.findPointByMeasurementId(this.get('model').get('startRun'));
         var end = currentTimeSeries.findPointByMeasurementId(this.get('model').get('endRun'));
         if (!start || !end)
-            return; // FIXME: Report an error.
+            return null; // FIXME: Report an error.
 
         var highlightedItems = {};
         highlightedItems[start.measurement.id()] = true;
@@ -1031,16 +1040,16 @@ App.AnalysisTaskController = Ember.Controller.extend({
                 id: point.measurement.id(),
                 measurement: point.measurement,
                 label: 'Point ' + (index + 1),
-                value: chartData.formatter(point.value) + (chartData.unit ? ' ' + chartData.unit : ''),
+                value: chartData.formatWithUnit(point.value),
             };
         });
 
         var margin = (end.time - start.time) * 0.1;
-        this.set('chartData', chartData);
-        this.set('chartDomain', [start.time - margin, +end.time + margin]);
         this.set('highlightedItems', highlightedItems);
         this.set('analysisPoints', formatedPoints);
-    },
+
+        return [start.time - margin, +end.time + margin];
+    }.property('pane.chartData', 'model', 'model'),
     testSets: function ()
     {
         var analysisPoints = this.get('analysisPoints');
@@ -1115,6 +1124,16 @@ App.AnalysisTaskController = Ember.Controller.extend({
             }));
         });
     }.observes('analysisPoints'),
+    updateTestGroupPanes: function ()
+    {
+        var model = this.get('model');
+        if (!model)
+            return;
+        var self = this;
+        model.get('testGroups').then(function (groups) {
+            self.set('testGroupPanes', groups.map(function (group) { return App.TestGroupPane.create({content: group}); }));
+        });
+    }.observes('model'),
     actions: {
         associateBug: function (bugTracker, bugNumber)
         {
@@ -1136,5 +1155,190 @@ App.AnalysisTaskController = Ember.Controller.extend({
                 
             });
         },
+        toggleShowRequestList: function (configuration)
+        {
+            configuration.toggleProperty('showRequestList');
+        }
     },
 });
+
+App.TestGroupPane = Ember.ObjectProxy.extend({
+    _populate: function ()
+    {
+        var buildRequests = this.get('buildRequests');
+        var chartData = this.get('chartData');
+        if (!buildRequests || !chartData)
+            return [];
+
+        var repositories = this._computeRepositoryList();
+        this.set('repositories', repositories);
+
+        var requestsByRooSet = this._groupRequestsByConfigurations(buildRequests);
+
+        var configurations = [];
+        var index = 0;
+        var range = {min: Infinity, max: -Infinity};
+        for (var rootSetId in requestsByRooSet) {
+            var configLetter = String.fromCharCode('A'.charCodeAt(0) + index++);
+            configurations.push(this._createConfigurationSummary(requestsByRooSet[rootSetId], configLetter, range));
+        }
+
+        var margin = 0.1 * (range.max - range.min);
+        range.max += margin;
+        range.min -= margin;
+
+        this.set('configurations', configurations);
+    }.observes('chartData', 'buildRequests'),
+    _computeRepositoryList: function ()
+    {
+        var specifiedRepositories = new Ember.Set();
+        (this.get('rootSets') || []).forEach(function (rootSet) {
+            (rootSet.get('roots') || []).forEach(function (root) {
+                specifiedRepositories.add(root.get('repository'));
+            });
+        });
+        var reportedRepositories = new Ember.Set();
+        var chartData = this.get('chartData');
+        (this.get('buildRequests') || []).forEach(function (request) {
+            var point = chartData.current.findPointByBuild(request.get('build'));
+            if (!point)
+                return;
+
+            var revisionByRepositoryId = point.measurement.formattedRevisions();
+            for (var repositoryId in revisionByRepositoryId) {
+                var repository = App.Manifest.repository(repositoryId);
+                if (!specifiedRepositories.contains(repository))
+                    reportedRepositories.add(repository);
+            }
+        });
+        return specifiedRepositories.sortBy('name').concat(reportedRepositories.sortBy('name'));
+    },
+    _groupRequestsByConfigurations: function (requests, repositoryList)
+    {
+        var rootSetIdToRequests = {};
+        var testGroup = this;
+        requests.forEach(function (request) {
+            var rootSetId = request.get('rootSet').get('id');
+            if (!rootSetIdToRequests[rootSetId])
+                rootSetIdToRequests[rootSetId] = [];
+            rootSetIdToRequests[rootSetId].push(request);
+        });
+        return rootSetIdToRequests;
+    },
+    _createConfigurationSummary: function (buildRequests, configLetter, range)
+    {
+        var repositories = this.get('repositories');
+        var chartData = this.get('chartData');
+        var requests = buildRequests.map(function (originalRequest) {
+            var point = chartData.current.findPointByBuild(originalRequest.get('build'));
+            var revisionByRepositoryId = point ? point.measurement.formattedRevisions() : {};
+            return Ember.ObjectProxy.create({
+                content: originalRequest,
+                revisions: repositories.map(function (repository, index) {
+                    return (revisionByRepositoryId[repository.get('id')] || {label:null}).label;
+                }),
+                value: point ? point.value : null,
+                valueRange: range,
+                formattedValue: point ? chartData.formatWithUnit(point.value) : null,
+                buildNumber: point ? point.measurement.buildNumber() : null,
+            });
+        });
+
+        var rootSet = requests ? requests[0].get('rootSet') : null;
+        var summaryRevisions = repositories.map(function (repository, index) {
+            var revision = rootSet ? rootSet.revisionForRepository(repository) : null;
+            if (!revision)
+                return requests[0].get('revisions')[index];
+            return Measurement.formatRevisionRange(revision).label;
+        });
+
+        requests.forEach(function (request) {
+            var revisions = request.get('revisions');
+            repositories.forEach(function (repository, index) {
+                if (revisions[index] == summaryRevisions[index])
+                    revisions[index] = null;
+            });
+        });
+
+        var valuesInConfig = requests.mapBy('value').filter(function (value) { return typeof(value) === 'number' && !isNaN(value); });
+        var sum = Statistics.sum(valuesInConfig);
+        var ciDelta = Statistics.confidenceIntervalDelta(0.95, valuesInConfig.length, sum, Statistics.squareSum(valuesInConfig));
+        var mean = sum / valuesInConfig.length;
+
+        range.min = Math.min(range.min, Statistics.min(valuesInConfig));
+        range.max = Math.max(range.max, Statistics.max(valuesInConfig));
+        if (ciDelta && !isNaN(ciDelta)) {
+            range.min = Math.min(range.min, mean - ciDelta);
+            range.max = Math.max(range.max, mean + ciDelta);
+        }
+
+        var summary = Ember.Object.create({
+            isAverage: true,
+            configLetter: configLetter,
+            revisions: summaryRevisions,
+            formattedValue: isNaN(mean) ? null : chartData.formatWithDeltaAndUnit(mean, ciDelta),
+            value: mean,
+            confidenceIntervalDelta: ciDelta,
+            valueRange: range,
+            statusLabel: App.BuildRequest.aggregateStatuses(requests),
+        });
+
+        return Ember.Object.create({summary: summary, items: requests});
+    },
+});
+
+App.BoxPlotComponent = Ember.Component.extend({
+    classNames: ['box-plot'],
+    range: null,
+    value: null,
+    delta: null,
+    didInsertElement: function ()
+    {
+        var element = this.get('element');
+        var svg = d3.select(element).append('svg')
+            .attr('viewBox', '0 0 100 20')
+            .attr('preserveAspectRatio', 'none')
+            .style({width: '100%', height: '100%'});
+
+        this._percentageRect = svg
+            .append('rect')
+            .attr('x', 0)
+            .attr('y', 0)
+            .attr('width', 0)
+            .attr('height', 20)
+            .attr('class', 'percentage');
+
+        this._deltaRect = svg
+            .append('rect')
+            .attr('x', 0)
+            .attr('y', 5)
+            .attr('width', 0)
+            .attr('height', 10)
+            .attr('class', 'delta')
+            .attr('opacity', 0.5)
+        this._updateBars();
+    },
+    _updateBars: function ()
+    {
+        if (!this._percentageRect || typeof(this._percentage) !== 'number' || isNaN(this._percentage))
+            return;
+
+        this._percentageRect.attr('width', this._percentage);
+        if (typeof(this._delta) === 'number' && !isNaN(this._delta)) {
+            this._deltaRect.attr('x', this._percentage - this._delta);
+            this._deltaRect.attr('width', this._delta * 2);
+        }
+    },
+    valueChanged: function ()
+    {
+        var range = this.get('range');
+        var value = this.get('value');
+        if (!range || !value)
+            return;
+        var scalingFactor = 100 / (range.max - range.min);
+        var percentage = (value - range.min) * scalingFactor;
+        this._percentage = percentage;
+        this._delta = this.get('delta') * scalingFactor;
+        this._updateBars();
+    }.observes('value', 'range').on('init'),
+});
index 3f97063..131d567 100755 (executable)
     border-left: solid 1px #bbb;
 }
 
+.analysis-chart-pane {
+    height: 15rem;
+}
+
 .analysis-chart-pane .details {
     overflow: scroll;
 }
     height: 13rem;
     overflow: scroll;
 }
+.analysis-chart-pane .details-table-container {
+    position: static;
+    height: 15rem;
+}
 
 .chart-pane .details-table,
 .chart-pane .commits-viewer {
index dd86d95..94d0fb2 100755 (executable)
@@ -169,14 +169,14 @@ Measurement.prototype.formattedRevisions = function (previousMeasurement)
     for (var repositoryId in revisions) {
         var currentRevision = revisions[repositoryId][0];
         var previousRevision = previousRevisions ? previousRevisions[repositoryId][0] : null;
-        var formatttedRevision = this._formatRevisionRange(previousRevision, currentRevision);
+        var formatttedRevision = Measurement.formatRevisionRange(currentRevision, previousRevision);
         formattedRevisions[repositoryId] = formatttedRevision;
     }
 
     return formattedRevisions;
 }
 
-Measurement.prototype._formatRevisionRange = function (previousRevision, currentRevision)
+Measurement.formatRevisionRange = function (currentRevision, previousRevision)
 {
     var revisionChanged = false;
     if (previousRevision == currentRevision)
index 30f4078..bdfc99b 100755 (executable)
                             domain=mainPlotDomain
                             interactive=true
                             chartPointRadius=2
-                            currentItem=currentItem
+                            currentItem=hoveredOrSelectedItem
                             currentTime=sharedTime
                             selectedItem=selectedItem
                             highlightedItems=highlightedItems
                             rangeRoute="analysisTask"
                             selection=timeRange
                             selectedPoints=selectedPoints
-                            markedPoints=markedPoints
                             showFullYAxis=showFullYAxis
+                            zoomable=true
                             zoom="zoomed"}}
                     {{else}}
                         {{#if failure}}
                                 selection=overviewSelection}}
                         {{/if}}
                         </div>
-                        {{#if details}}
-                            {{partial "chart-details"}}
-                        {{/if}}
+                        {{partial "chart-details"}}
                     </div>
                 </div>
 
     </script>
 
     <script type="text/x-handlebars" data-template-name="chart-details">
+    {{#if details}}
     <div class="details-table-container">
         <table class="details-table">
             <tbody class="bugs">
             {{/each}}
         </div>
     </div>
+    {{/if}}
     </script>
 
     <script type="text/x-handlebars" data-template-name="components/commits-viewer">
             <h3 id="analysis-task-testname">{{metric.fullName}} - {{platform.label}}</h3>
         {{/if}}
 
-        {{#if chartData}}
-            <section class="analysis-chart-pane chart-pane">
+        {{#if pane}}
+            <section class="analysis-chart-pane chart-pane" tabindex="0">
                 <div class="svg-container">
                     {{interactive-chart
-                        chartData=chartData
-                        enableSelection=false
+                        chartData=pane.chartData
+                        ranges=pane.analyticRanges
+                        domain=paneDomain
+                        interactive=true
                         chartPointRadius=2
-                        domain=chartDomain
-                        highlightedItems=highlightedItems}}
+                        currentItem=pane.hoveredOrSelectedItem
+                        selectedPoints=pane.selectedPoints
+                        selection=timeRange
+                        highlightedItems=highlightedItems
+                        rangeRoute="analysisTask"}}
                 </div>
                 <div class="details">
-                    <table class="analysis-bugs">
-                        <tbody>
-                            {{#each bugTrackers}}
-                                <tr>
-                                    <th>{{label}}</th>
-                                    <td>
-                                        <form {{action "associateBug" this editedBugNumber on="submit"}}>
-                                            {{input type=text value=editedBugNumber}}
-                                        </form>
-                                    </td>
-                                </tr>
-                            {{/each}}
-                        </tbody>
-                    </table>
-                    <table>
-                        <tbody>
-                            {{#each analysisPoints}}
-                                <tr><td>{{label}}</td><td>{{value}}</td></tr>
-                            {{/each}}
-                        </tbody>
-                    </table>
+                    {{partial "chart-details"}}
                 </div>
             </section>
-            {{#each testGroups}}
-                <section class="analysis-group">
-                    <table>
-                        <caption>{{name}}</caption>
-                        <thead>
-                            <tr>
-                                <td>Order</td>
-                                <td>Configuration</td>
-                                <td>Status</td>
-                                <td>Build</td>
-                                <td>{{../metric.fullName}}</td>
+        {{/if}}
+
+        {{partial "testGroupForm"}}
+
+        {{#each testGroupPanes}}
+            {{partial "testGroup"}}
+        {{/each}}
+    </script>
+
+    <script type="text/x-handlebars" data-template-name="testGroup">
+        <section class="analysis-group">
+            <h1>{{name}}</h1>
+            <table class="results">
+                <thead>
+                    <tr>
+                        <td colspan="2">Configuration</td>
+                        {{#each repositories}}
+                            <td>{{name}}</td>
+                        {{/each}}
+                        <td>Results</td>
+                        <td>Status</td>
+                    </tr>
+                </thead>
+                {{#each configurations}}
+                    <tbody {{bind-attr class="showRequestList::hideRequests"}}>
+                        <tr class="summary" {{action toggleShowRequestList this}}>
+                            <td class="config-letter" colspan="2">{{summary.configLetter}}</td>
+                            {{#with summary}}
+                                {{partial "testGroupRow"}}
+                            {{/with}}
+                        </tr>
+                        {{#each items}}
+                            <tr class="request">
+                                {{#with ../this}}
+                                    <td class="config-letter" {{action toggleShowRequestList this}}></td>
+                                {{/with}}
+                                <td>Run {{orderLabel}}</td>
+                                {{partial "testGroupRow"}}
                             </tr>
-                        </thead>
-                        <tbody>
-                            {{#each buildRequests}}
-                                <tr>
-                                    <td>{{orderLabel}}</td>
-                                    <td>{{configLetter}}</td>
-                                    <td><a {{bind-attr href=url}}>{{statusLabel}}</a></td>
-                                    <td>{{buildNumber}}</td>
-                                    <td>{{mean}}</td>
-                                </tr>
-                            {{/each}}
-                        </tbody>
-                    </table>
-                </section>
-            {{/each}}
+                        {{/each}}
+                    </tbody>
+                {{/each}}
+            </table>
+        </section>
+    </script>
+
+    <script type="text/x-handlebars" data-template-name="testGroupRow">
+        {{#each revisions}}
+            <td>{{this}}</td>
+        {{/each}}
+        <td>
+            {{#if value}}
+                {{box-plot range=valueRange value=value delta=confidenceIntervalDelta}}
+            {{/if}}
+            {{formattedValue}}
+        </td>
+        <td>
+            {{#if buildNumber}}
+                 {{statusLabel}} / <a {{bind-attr href=url}}>Build {{buildNumber}}</a>
+            {{else}}
+                <a {{bind-attr href=url}}>{{statusLabel}}</a>
+            {{/if}}
+        </td>
+    </script>
 
-            {{#if roots}}
-            <form method="POST" {{action "createTestGroup" newTestGroupName repetitionCount on="submit"}} class="analysis-group">
+    <script type="text/x-handlebars" data-template-name="testGroupForm">
+    {{#if roots}}
+        <form method="POST" {{action "createTestGroup" newTestGroupName repetitionCount on="submit"}}>
+            <section class="analysis-group">
+                <h1>{{input name="name" value=newTestGroupName placeholder="Test group name" required=true type="text"}}</h1>
                 <table>
-                    <caption>{{input name="name" value=newTestGroupName placeholder="Test group name" required=true type="text"}}</caption>
                     <thead>
                         <tr>
                             <th>Root</th>
                 </table>
 
                 <button type="submit">Start A/B testing</button>
-            </form>
-            {{/if}}
-        {{/if}}
+            </section>
+        </form>
+    {{/if}}
     </script>
 
 </head>
index 68c44cd..0567680 100644 (file)
@@ -197,6 +197,8 @@ App.InteractiveChartComponent = Ember.Component.extend({
     _updateDomain: function ()
     {
         var xDomain = this.get('domain');
+        if (!xDomain || !this._currentTimeSeriesData)
+            return null;
         var intrinsicXDomain = this._computeXAxisDomain(this._currentTimeSeriesData);
         if (!xDomain)
             xDomain = intrinsicXDomain;
@@ -373,6 +375,8 @@ App.InteractiveChartComponent = Ember.Component.extend({
     {
         var selection = this._currentSelection() || this.get('sharedSelection');
         var newXDomain = this._updateDomain();
+        if (!newXDomain)
+            return;
 
         if (selection && newXDomain && selection[0] <= newXDomain[0] && newXDomain[1] <= selection[1])
             selection = null; // Otherwise the user has no way of clearing the selection.
@@ -753,7 +757,7 @@ App.InteractiveChartComponent = Ember.Component.extend({
     },
     _updateSelectionToolbar: function ()
     {
-        if (!this.get('interactive'))
+        if (!this.get('zoomable'))
             return;
 
         var selection = this._currentSelection();
index 10ffd3f..c32eb65 100755 (executable)
@@ -1,5 +1,13 @@
 var Statistics = new (function () {
 
+    this.min = function (values) {
+        return Math.min.apply(Math, values);
+    }
+
+    this.max = function (values) {
+        return Math.max.apply(Math, values);
+    }
+
     this.sum = function (values) {
         return values.length ? values.reduce(function (a, b) { return a + b; }) : 0;
     }
index 822e8cf..7d93aef 100755 (executable)
@@ -302,6 +302,8 @@ App.Manifest = Ember.Controller.extend({
             var smallerIsBetter = unit != 'fps' && unit != '/s'; // Assume smaller is better for unit-less metrics.
 
             var useSI = unit == 'bytes';
+            var unitSuffix = unit ? ' ' + unit : '';
+            var deltaFormatterWithoutSign = useSI ? d3.format('.2s') : d3.format('.2g');
             return {
                 platform: platform,
                 metric: metric,
@@ -310,6 +312,11 @@ App.Manifest = Ember.Controller.extend({
                     baseline: runs.baseline ? runs.baseline.timeSeriesByCommitTime() : null,
                     target: runs.target ? runs.target.timeSeriesByCommitTime() : null,
                     unit: unit,
+                    formatWithUnit: function (value) { return this.formatter(value) + unitSuffix; },
+                    formatWithDeltaAndUnit: function (value, delta)
+                    {
+                        return this.formatter(value) + (delta && !isNaN(delta) ? ' \u00b1 ' + deltaFormatterWithoutSign(delta) : '') + unitSuffix;
+                    },
                     formatter: useSI ? d3.format('.4s') : d3.format('.4g'),
                     deltaFormatter: useSI ? d3.format('+.2s') : d3.format('+.2g'),
                     smallerIsBetter: smallerIsBetter,