Perf dashboard should have a way of marking outliers
authorrniwa@webkit.org <rniwa@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 7 Apr 2015 21:42:37 +0000 (21:42 +0000)
committerrniwa@webkit.org <rniwa@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 7 Apr 2015 21:42:37 +0000 (21:42 +0000)
https://bugs.webkit.org/show_bug.cgi?id=143466

Reviewed by Chris Dumez.

Added UI to mark a data point as an outlier as well as a button to toggle the visibility of outliers.
Added a new privileged API /privileged-api/update-run-status to store this boolean flag.

* init-database.sql: Added run_marked_outlier column to test_runs table.

* public/admin/tests.php:

* public/api/runs.php:
(main): Only emit Cache-Control and Expires headers in v1 UI.
(RunsGenerator::format_run): Emit markedOutlier.

* public/include/admin-header.php:

* public/include/db.php:
(Database::is_true): Made it static.

* public/include/manifest.php:
(Manifest::platforms):

* public/index.html: Call into /api/runs/ with ?cache=true.

* public/privileged-api/update-run-status.php: Added.
(main): Updates the newly added column in test_runs table.

* public/v2/app.js:
(App.Pane._fetch):
(App.Pane.refetchRuns): Extracted from App.Pane._fetch.
(App.Pane._didFetchRuns): Renamed from _updateChartData.
(App.Pane._setNewChartData): Added. Pick the right time series based based on the value of showOutlier.
Cloning chartData is necessary when toggling the outlier visibility or using statistics tools because
the interactive chart component only observes changes to chartData and not individual properties of it.
(App.Pane._highlightPointsMarkedAsOutlier): Added. Highlight points marked as outliers.
(App.Pane._movingAverageOrEnvelopeStrategyDidChange): Call to _setNewChartData replaced the code to
clone chartData here.

(App.PaneController.actions.toggleShowOutlier): Toggle the visibility of points marked as outliers by
invoking App.Pane._setNewChartData.
(App.PaneController._detailsChanged): Don't hide the analysis pane when details changed since keep
opening the pane for marking points as outliers would be annoying.
(App.PaneController._updateCanAnalyze): Update 'cannotMarkOutlier' as well as 'cannotAnalyze'.
(App.PaneController.selectedMeasurement): Added.
(App.PaneController.showOutlierTitle): Added.
(App.PaneController._selectedItemIsMarkedOutlierDidChange): Added. Call out to setMarkedOutlier to
mark the selected point as an outlier via the newly added privileged API.

* public/v2/chart-pane.css: Updated styles.

* public/v2/data.js:
(PrivilegedAPI._post): Report the semantic errors.
(Measurement.prototype.markedOutlier): Added.
(Measurement.prototype.setMarkedOutlier): Added. Uses PrivilegedAPI to update the database.
(RunsData.prototype.timeSeriesByCommitTime): Added a new argument, includeOutliers, to indicate
whether the time series should include measurements marked as outliers or not.
(RunsData.prototype.timeSeriesByBuildTime): Ditto.
(RunsData.prototype._timeSeriesByTimeInternal): Extracted from timeSeriesByCommitTime and
timeSeriesByBuildTime to share code. Now ignores measurements marked as outliers if needed.

* public/v2/index.html: Added an icon for showing and hiding outliers. Also added a checkbox to
mark individual points as outliers.

* public/v2/interactive-chart.js:
(App.InteractiveChartComponent._selectClosestPointToMouseAsCurrentItem): Re-enable the distance
heuristics that takes vertical closeness into account. This heuristics is more useful when marking
some points as outliers. This heuristics was disabled because the behavior was unpredictable but
with the arrow key navigation support, this is no longer an issue.

* public/v2/manifest.js:
(App.Manifest._formatFetchedData): Added showOutlier to the chart data. This function dynamically
updates the time series in this chart data in order to include or exclude outliers.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@182496 268f45cc-cd09-0410-ab3c-d52691b4dbfc

15 files changed:
Websites/perf.webkit.org/ChangeLog
Websites/perf.webkit.org/init-database.sql
Websites/perf.webkit.org/public/admin/tests.php
Websites/perf.webkit.org/public/api/runs.php
Websites/perf.webkit.org/public/include/admin-header.php
Websites/perf.webkit.org/public/include/db.php
Websites/perf.webkit.org/public/include/manifest.php
Websites/perf.webkit.org/public/index.html
Websites/perf.webkit.org/public/privileged-api/update-run-status.php [new file with mode: 0644]
Websites/perf.webkit.org/public/v2/app.js
Websites/perf.webkit.org/public/v2/chart-pane.css
Websites/perf.webkit.org/public/v2/data.js
Websites/perf.webkit.org/public/v2/index.html
Websites/perf.webkit.org/public/v2/interactive-chart.js
Websites/perf.webkit.org/public/v2/manifest.js

index 66d9434..6936a1e 100644 (file)
@@ -1,3 +1,80 @@
+2015-04-07  Ryosuke Niwa  <rniwa@webkit.org>
+
+        Perf dashboard should have a way of marking outliers
+        https://bugs.webkit.org/show_bug.cgi?id=143466
+
+        Reviewed by Chris Dumez.
+
+        Added UI to mark a data point as an outlier as well as a button to toggle the visibility of outliers.
+        Added a new privileged API /privileged-api/update-run-status to store this boolean flag.
+
+        * init-database.sql: Added run_marked_outlier column to test_runs table.
+
+        * public/admin/tests.php:
+
+        * public/api/runs.php:
+        (main): Only emit Cache-Control and Expires headers in v1 UI.
+        (RunsGenerator::format_run): Emit markedOutlier.
+
+        * public/include/admin-header.php:
+
+        * public/include/db.php:
+        (Database::is_true): Made it static.
+
+        * public/include/manifest.php:
+        (Manifest::platforms):
+
+        * public/index.html: Call into /api/runs/ with ?cache=true.
+
+        * public/privileged-api/update-run-status.php: Added.
+        (main): Updates the newly added column in test_runs table.
+
+        * public/v2/app.js:
+        (App.Pane._fetch):
+        (App.Pane.refetchRuns): Extracted from App.Pane._fetch.
+        (App.Pane._didFetchRuns): Renamed from _updateChartData.
+        (App.Pane._setNewChartData): Added. Pick the right time series based based on the value of showOutlier.
+        Cloning chartData is necessary when toggling the outlier visibility or using statistics tools because
+        the interactive chart component only observes changes to chartData and not individual properties of it.
+        (App.Pane._highlightPointsMarkedAsOutlier): Added. Highlight points marked as outliers.
+        (App.Pane._movingAverageOrEnvelopeStrategyDidChange): Call to _setNewChartData replaced the code to
+        clone chartData here.
+
+        (App.PaneController.actions.toggleShowOutlier): Toggle the visibility of points marked as outliers by
+        invoking App.Pane._setNewChartData.
+        (App.PaneController._detailsChanged): Don't hide the analysis pane when details changed since keep
+        opening the pane for marking points as outliers would be annoying.
+        (App.PaneController._updateCanAnalyze): Update 'cannotMarkOutlier' as well as 'cannotAnalyze'.
+        (App.PaneController.selectedMeasurement): Added.
+        (App.PaneController.showOutlierTitle): Added.
+        (App.PaneController._selectedItemIsMarkedOutlierDidChange): Added. Call out to setMarkedOutlier to
+        mark the selected point as an outlier via the newly added privileged API.
+
+        * public/v2/chart-pane.css: Updated styles.
+
+        * public/v2/data.js:
+        (PrivilegedAPI._post): Report the semantic errors.
+        (Measurement.prototype.markedOutlier): Added.
+        (Measurement.prototype.setMarkedOutlier): Added. Uses PrivilegedAPI to update the database.
+        (RunsData.prototype.timeSeriesByCommitTime): Added a new argument, includeOutliers, to indicate
+        whether the time series should include measurements marked as outliers or not.
+        (RunsData.prototype.timeSeriesByBuildTime): Ditto.
+        (RunsData.prototype._timeSeriesByTimeInternal): Extracted from timeSeriesByCommitTime and
+        timeSeriesByBuildTime to share code. Now ignores measurements marked as outliers if needed.
+
+        * public/v2/index.html: Added an icon for showing and hiding outliers. Also added a checkbox to
+        mark individual points as outliers.
+
+        * public/v2/interactive-chart.js:
+        (App.InteractiveChartComponent._selectClosestPointToMouseAsCurrentItem): Re-enable the distance
+        heuristics that takes vertical closeness into account. This heuristics is more useful when marking
+        some points as outliers. This heuristics was disabled because the behavior was unpredictable but
+        with the arrow key navigation support, this is no longer an issue.
+
+        * public/v2/manifest.js:
+        (App.Manifest._formatFetchedData): Added showOutlier to the chart data. This function dynamically
+        updates the time series in this chart data in order to include or exclude outliers.
+
 2015-04-03  Ryosuke Niwa  <rniwa@webkit.org>
 
         Perf dashboard should be able to trigger A/B testing jobs for iOS
index 1fc3cd0..7e859c0 100644 (file)
@@ -135,6 +135,7 @@ CREATE TABLE test_runs (
     run_mean_cache double precision,
     run_sum_cache double precision,
     run_square_sum_cache double precision,
+    run_marked_outlier boolean,
     CONSTRAINT test_config_build_must_be_unique UNIQUE(run_config, run_build));
 CREATE INDEX run_config_index ON test_runs(run_config);
 CREATE INDEX run_build_index ON test_runs(run_build);
index adbd601..3cd7e66 100644 (file)
@@ -196,9 +196,9 @@ EOF;
                         if (!$configurations)
                             continue;
                         echo "<label><input type=\"checkbox\" name=\"metric_platforms[]\" value=\"{$platform['platform_id']}\"";
-                        if ($db->is_true($configurations[0]['config_is_in_dashboard']))
+                        if (Database::is_true($configurations[0]['config_is_in_dashboard']))
                             echo ' checked';
-                        else if ($db->is_true($platform['platform_hidden']))
+                        else if (Database::is_true($platform['platform_hidden']))
                             echo 'disabled';
                         echo ">$platform_name</label>";
                     }
index 6bf7fcf..9681a37 100644 (file)
@@ -43,10 +43,10 @@ function main($path) {
         exit_with_error('ConfigurationNotFound');
 
     $test_group_id = array_get($_GET, 'testGroup');
+    $should_cache = array_get($_GET, 'cache');
     if ($test_group_id)
         $test_group_id = intval($test_group_id);
-    else {
-        // FIXME: We should support revalication as well as caching results in the server side.
+    else if ($should_cache) { // Only v1 UI needs caching.
         $maxage = config('jsonCacheMaxAge');
         header('Expires: ' . gmdate('D, d M Y H:i:s', time() + $maxage) . ' GMT');
         header("Cache-Control: maxage=$maxage");
@@ -102,6 +102,7 @@ class RunsGenerator {
             'iterationCount' => intval($run['run_iteration_count_cache']),
             'sum' => floatval($run['run_sum_cache']),
             'squareSum' => floatval($run['run_square_sum_cache']),
+            'markedOutlier' => Database::is_true($run['run_marked_outlier']),
             'revisions' => self::parse_revisions_array($run['revisions']),
             'build' => $run['build_id'],
             'buildTime' => Database::to_js_time($run['build_time']),
index 4d1254e..697ecca 100644 (file)
@@ -130,7 +130,7 @@ END;
             $show_update_button = $show_update_button_if_needed;
             break;
         case 'boolean':
-            $checkedness = $this->db->is_true($value) ? ' checked' : '';
+            $checkedness = Database::is_true($value) ? ' checked' : '';
             echo <<< END
 <input type="checkbox" name="$name"$checkedness>
 END;
index 35b5e29..23ab2ca 100644 (file)
@@ -60,7 +60,7 @@ class Database
         $this->connection = false;
     }
 
-    function is_true($value) {
+    static function is_true($value) {
         return $value == 't';
     }
 
index d36763b..a1b16dd 100644 (file)
@@ -79,7 +79,7 @@ class ManifestGenerator {
         $platform_metrics = array();
         if ($config_table) {
             foreach ($config_table as $config_row) {
-                if ($is_dashboard && !$this->db->is_true($config_row['config_is_in_dashboard']))
+                if ($is_dashboard && !Database::is_true($config_row['config_is_in_dashboard']))
                     continue;
 
                 $new_last_modified = array_get($config_row, 'config_runs_last_modified', 0);
@@ -104,7 +104,7 @@ class ManifestGenerator {
         $platforms = array();
         if ($platform_table) {
             foreach ($platform_table as $platform_row) {
-                if ($this->db->is_true($platform_row['platform_hidden']))
+                if (Database::is_true($platform_row['platform_hidden']))
                     continue;
                 $id = $platform_row['platform_id'];
                 if (array_key_exists($id, $platform_metrics)) {
index e4e2207..6e69333 100644 (file)
@@ -840,7 +840,7 @@ function fetchTest(repositories, builders, filename, platform, metric, callback)
         return runs;
     }
 
-    $.getJSON('api/runs/' + filename, function (response) {
+    $.getJSON('api/runs/' + filename + '?cache=true', function (response) {
         var data = response.configurations;
         callback(createRunAndResults(data.current), createRunAndResults(data.baseline), createRunAndResults(data.target));
     });
diff --git a/Websites/perf.webkit.org/public/privileged-api/update-run-status.php b/Websites/perf.webkit.org/public/privileged-api/update-run-status.php
new file mode 100644 (file)
index 0000000..000cfdc
--- /dev/null
@@ -0,0 +1,30 @@
+<?php
+
+require_once('../include/json-header.php');
+
+function main() {
+    $data = ensure_privileged_api_data_and_token();
+
+    $run_id = array_get($data, 'run');
+    if (!$run_id)
+        exit_with_error('MissingRunId');
+
+    $db = connect();
+    $run = $db->select_first_row('test_runs', 'run', array('id' => $run_id));
+    if (!$run)
+        exit_with_error('InvalidRun', array('run' => $run_id));
+
+    $marked_outlier = array_get($data, 'markedOutlier');
+
+    $db->begin_transaction();
+    $db->update_row('test_runs', 'run', array('id' => $run_id), array(
+        'id' => $run_id,
+        'marked_outlier' => $marked_outlier ? 't' : 'f'));
+    $db->commit_transaction();
+
+    exit_with_success();
+}
+
+main();
+
+?>
index 72d21b2..a711f8b 100755 (executable)
@@ -349,28 +349,56 @@ App.Pane = Ember.Object.extend({
         else if (!this._isValidId(metricId))
             this.set('failure', metricId ? 'Invalid metric id:' + metricId : 'Metric id was not specified');
         else {
-            var store = this.get('store');
-            var updateChartData = this._updateChartData.bind(this);
-            var handleErrors = this._handleFetchErrors.bind(this, platformId, metricId);
+            var self = this;
             var useCache = true;
-            App.Manifest.fetchRunsWithPlatformAndMetric(store, platformId, metricId, null, useCache).then(function (result) {
-                    updateChartData(result);
-                    if (!result.shouldRefetch)
-                        return;
-
-                    useCache = false;
-                    App.Manifest.fetchRunsWithPlatformAndMetric(store, platformId, metricId, null, useCache)
-                        .then(updateChartData, handleErrors);
-                }, handleErrors);
+            App.Manifest.fetchRunsWithPlatformAndMetric(this.get('store'), platformId, metricId, null, useCache)
+                .then(function (result) {
+                    self._didFetchRuns(result);
+                    if (result.shouldRefetch)
+                        self.refetchRuns();
+                }, this._handleFetchErrors.bind(this, platformId, metricId));
             this.fetchAnalyticRanges();
         }
     }.observes('platformId', 'metricId').on('init'),
-    _updateChartData: function (result)
+    refetchRuns: function () {
+        var platform = this.get('platform');
+        var metric = this.get('metric');
+        Ember.assert('refetchRuns should be called only after platform and metric are resolved', platform && metric);
+
+        var useCache = false;
+        App.Manifest.fetchRunsWithPlatformAndMetric(this.get('store'), platform.get('id'), metric.get('id'), null, useCache)
+            .then(this._didFetchRuns.bind(this), this._handleFetchErrors.bind(this, platform.get('id'), metric.get('id')));
+    },
+    _didFetchRuns: function (result)
     {
         this.set('platform', result.platform);
         this.set('metric', result.metric);
-        this.set('chartData', result.data);
+        this._setNewChartData(result.data);
+    },
+    _setNewChartData: function (chartData)
+    {
+        var newChartData = {};
+        for (var property in chartData)
+            newChartData[property] = chartData[property];
+
+        var showOutlier = this.get('showOutlier');
+        newChartData.showOutlier(showOutlier);
+        this.set('chartData', newChartData);
         this._updateMovingAverageAndEnvelope();
+
+        if (!this.get('anomalyDetectionStrategies').filterBy('enabled').length)
+            this._highlightPointsMarkedAsOutlier(newChartData);
+    },
+    _highlightPointsMarkedAsOutlier: function (newChartData)
+    {
+        var data = newChartData.current.series();
+        var items = {};
+        for (var i = 0; i < data.length; i++) {
+            if (data[i].measurement.markedOutlier())
+                items[data[i].measurement.id()] = true;
+        }
+
+        this.set('highlightedItems', items);
     },
     _handleFetchErrors: function (platformId, metricId, result)
     {
@@ -518,16 +546,10 @@ App.Pane = Ember.Object.extend({
         this.set('highlightedItems', anomalies);
     },
     _movingAverageOrEnvelopeStrategyDidChange: function () {
-        this._updateMovingAverageAndEnvelope();
-
-        var newChartData = {};
         var chartData = this.get('chartData');
         if (!chartData)
             return;
-        for (var property in chartData)
-            newChartData[property] = chartData[property];
-        this.set('chartData', newChartData);
-
+        this._setNewChartData(chartData);
     }.observes('chosenMovingAverageStrategy', 'chosenMovingAverageStrategy.parameterList.@each.value',
         'chosenEnvelopingStrategy', 'chosenEnvelopingStrategy.parameterList.@each.value',
         'anomalyDetectionStrategies.@each.enabled'),
@@ -925,6 +947,15 @@ App.PaneController = Ember.ObjectController.extend({
                 this.set('showingStatPane', false);
             }
         },
+        toggleShowOutlier: function ()
+        {
+            var pane = this.get('model');
+            pane.toggleProperty('showOutlier');
+            var chartData = pane.get('chartData');
+            if (!chartData)
+                return;
+            pane._setNewChartData(chartData);
+        },
         createAnalysisTask: function ()
         {
             var name = this.get('newAnalysisTaskName');
@@ -978,10 +1009,6 @@ App.PaneController = Ember.ObjectController.extend({
             Ember.run.debounce(this, 'propagateZoom', 100);
         },
     },
-    _detailsChanged: function ()
-    {
-        this.set('showingAnalysisPane', false);
-    }.observes('details'),
     _overviewSelectionChanged: function ()
     {
         var overviewSelection = this.get('overviewSelection');
@@ -1013,9 +1040,43 @@ App.PaneController = Ember.ObjectController.extend({
     }.observes('parentController.sharedZoom').on('init'),
     _updateCanAnalyze: function ()
     {
-        var points = this.get('model').get('selectedPoints');
+        var pane = this.get('model');
+        var points = pane.get('selectedPoints');
         this.set('cannotAnalyze', !this.get('newAnalysisTaskName') || !points || points.length < 2);
-    }.observes('newAnalysisTaskName', 'model.selectedPoints'),
+        this.set('cannotMarkOutlier', !!points || !this.get('selectedItem'));
+
+        var selectedMeasurement = this.selectedMeasurement();
+        this.set('selectedItemIsMarkedOutlier', selectedMeasurement && selectedMeasurement.markedOutlier());
+
+    }.observes('newAnalysisTaskName', 'model.selectedPoints', 'model.selectedItem').on('init'),
+    selectedMeasurement: function () {
+        var chartData = this.get('model').get('chartData');
+        var selectedItem = this.get('selectedItem');
+        if (!chartData || !selectedItem)
+            return null;
+        var point = chartData.current.findPointByMeasurementId(selectedItem);
+        Ember.assert('selectedItem should always be in the current chart data', point);
+        return point.measurement;
+    },
+    showOutlierTitle: function ()
+    {
+        return this.get('showOutlier') ? 'Hide outliers' : 'Show outliers';
+    }.property('showOutlier'),
+    _selectedItemIsMarkedOutlierDidChange: function ()
+    {
+        var selectedMeasurement = this.selectedMeasurement();
+        if (!selectedMeasurement)
+            return;
+        var selectedItemIsMarkedOutlier = this.get('selectedItemIsMarkedOutlier');
+        if (selectedMeasurement.markedOutlier() == selectedItemIsMarkedOutlier)
+            return;
+        var pane = this.get('model');
+        selectedMeasurement.setMarkedOutlier(!!selectedItemIsMarkedOutlier).then(function () {
+            pane.refetchRuns();
+        }, function (error) {
+            alert(error);
+        });
+    }.observes('selectedItemIsMarkedOutlier'),
 });
 
 App.AnalysisRoute = Ember.Route.extend({
index 18e25f2..0064bfc 100755 (executable)
 .chart-pane a.stat-button {
     display: inline-block;
     position: absolute;
-    right: 3.15rem;
+    right: 4.45rem;
     top: 0.55rem;
 }
 
-.chart-pane a.bugs-button {
+.chart-pane a.outlier-button {
+    display: inline-block;
+    position: absolute;
+    right: 3.25rem; /* Shifted to left by 0.1rem for better aesthetics */
+    top: 0.55rem;
+}
+
+a.outlier-button.hide g.show-outlier-icon {
+    fill: transparent;
+    stroke: transparent;
+}
+
+a.outlier-button.show g.hide-outlier-icon {
+    fill: transparent;
+    stroke: transparent;
+}
+
+.chart-pane a.analysis-button {
     display: inline-block;
     position: absolute;
     right: 1.85rem;
     display: none;
 }
 
-.stat-pane {
+.stat-pane,
+.annotation-pane {
     right: 2.6rem;
     padding: 0;
 }
 
-.stat-pane fieldset {
-    border: solid 1px #ccc;
-    border-radius: 0.5rem;
-    margin: 0.2rem;
-    padding: 0;
+.popup-pane > .caution {
+    margin: 0;
+    padding: 0.3rem 0.5rem;
 }
 
-.stat-option {
+.popup-pane > section {
     margin: 0;
     padding: 0;
     font-size: 0.8rem;
     max-width: 17rem;
 }
 
-.stat-option h1 {
+.popup-pane > section > h1 {
     font-size: inherit;
     line-height: 0.8rem;
     padding: 0.3rem 0.5rem;
     border-bottom: solid 1px #ccc;
 }
 
-.stat-option:first-child h1 {
+.popup-pane > section:first-child h1 {
     border-top: none;
 }
 
-.stat-option > * {
+.popup-pane > section > * {
     display: block;
     margin: 0.1rem 0.5rem 0.1rem 1rem;
 }
     width: 4rem;
 }
 
-.analysis-pane {
+.annotation-pane {
     right: 1.3rem;
 }
 
-.analysis-pane > * {
-    margin: 0.2rem;
-}
-
 .search-pane {
     right: 0rem;
 }
index dbf86a9..1e0c32d 100755 (executable)
@@ -39,9 +39,10 @@ PrivilegedAPI._post = function (url, parameters)
             data: parameters ? JSON.stringify(parameters) : '{}',
             dataType: 'json',
         }).done(function (data) {
-            if (data.status != 'OK')
+            if (data.status != 'OK') {
+                console.log('PrivilegedAPI failed', data);
                 reject(data.status);
-            else
+            else
                 resolve(data);
         }).fail(function (xhr, status, error) {
             reject(xhr.status + (error ? ', ' + error : '') + '\n\nWith response:\n' + xhr.responseText);
@@ -288,6 +289,20 @@ Measurement.prototype.hasBugs = function ()
     return bugs && Object.keys(bugs).length;
 }
 
+Measurement.prototype.markedOutlier = function ()
+{
+    return this._raw['markedOutlier'];
+}
+
+Measurement.prototype.setMarkedOutlier = function (markedOutlier)
+{
+    var params = {'run': this.id(), 'markedOutlier': markedOutlier};
+    return PrivilegedAPI.sendRequest('update-run-status', params).then(function (data) {
+    }, function (error) {
+        alert('Failed to update the outlier status: ' + error);
+    });
+}
+
 function RunsData(rawData)
 {
     this._measurements = rawData.map(function (run) { return new Measurement(run); });
@@ -298,29 +313,32 @@ RunsData.prototype.count = function ()
     return this._measurements.length;
 }
 
-RunsData.prototype.timeSeriesByCommitTime = function ()
+RunsData.prototype.timeSeriesByCommitTime = function (includeOutliers)
 {
-    return new TimeSeries(this._measurements.map(function (measurement) {
-        var confidenceInterval = measurement.confidenceInterval();
-        return {
-            measurement: measurement,
-            time: measurement.latestCommitTime(),
-            value: measurement.mean(),
-            interval: measurement.confidenceInterval(),
-        };
-    }));
+    return this._timeSeriesByTimeInternal(true, includeOutliers);
 }
 
-RunsData.prototype.timeSeriesByBuildTime = function ()
+RunsData.prototype.timeSeriesByBuildTime = function (includeOutliers)
 {
-    return new TimeSeries(this._measurements.map(function (measurement) {
-        return {
+    return this._timeSeriesByTimeInternal(false, includeOutliers);
+}
+
+RunsData.prototype._timeSeriesByTimeInternal = function (useCommitType, includeOutliers)
+{
+    var series = new Array();
+    var seriesIndex = 0;
+    for (var measurement of this._measurements) {
+        if (measurement.markedOutlier() && !includeOutliers)
+            continue;
+        series.push({
             measurement: measurement,
-            time: measurement.buildTime(),
+            time: useCommitType ? measurement.latestCommitTime() : measurement.buildTime(),
             value: measurement.mean(),
             interval: measurement.confidenceInterval(),
-        };
-    }));
+            markedOutlier: measurement.markedOutlier(),
+        });
+    }
+    return new TimeSeries(series);
 }
 
 // FIXME: We need to devise a way to fetch runs in multiple chunks so that
index be4e7cc..97d2ae4 100755 (executable)
                     {{#if movingAverageStrategies}}
                         <a href="javascript:false" title="Statistical Tools" class="stat-button" {{action "toggleStatPane"}}>{{partial "stat-button"}}</a>
                     {{/if}}
-                    {{#if App.Manifest.bugTrackers}}
-                        <a href="javascript:false" title="Analysis" class="bugs-button" {{action "toggleBugsPane"}}>
-                            {{partial "analysis-button"}}
-                        </a>
-                    {{/if}}
+                    <a href="javascript:false" {{bind-attr title=showOutlierTitle class=":outlier-button showOutlier:show:hide"}}
+                        {{action "toggleShowOutlier"}}>
+                        {{partial "outlier-button"}}
+                    </a>
+                    <a href="javascript:false" title="Analyze the selected range" class="analysis-button" {{action "toggleBugsPane"}}>
+                        {{partial "analysis-button"}}
+                    </a>
                     {{#if App.Manifest.repositoriesWithReportedCommits}}
-                        <a href="javascript:false" title="Search" class="search-button" {{action "toggleSearchPane"}}>{{partial "search-button"}}</a>
+                        <a href="javascript:false" title="Search commits by a keyword" class="search-button" {{action "toggleSearchPane"}}>{{partial "search-button"}}</a>
                     {{/if}}
                 </header>
 
                 </div>
 
                 <div {{bind-attr class=":popup-pane :analysis-pane showingAnalysisPane::hidden"}}>
-                    <label>Name: {{input type=text value=newAnalysisTaskName}}</label>
-                    <button {{action "createAnalysisTask"}} {{bind-attr disabled=cannotAnalyze}}>Analyze</button>
+                    <section class="analysis-option-option">
+                        <h1>Start A/B testing or associate bugs</h1>
+                        <label>Name: {{input type=text value=newAnalysisTaskName}} <button {{action "createAnalysisTask"}} {{bind-attr disabled=cannotAnalyze}}>Analyze</button></label>
+                    </section>
+                    <section class="analysis-option-option">
+                        <h1>Marking outliers</h1>
+                        <label>{{input type=checkbox checked=selectedItemIsMarkedOutlier disabled=cannotMarkOutlier}} Mark as an outlier and hide it.</label>
+                    </section>
                 </div>
 
                 <form {{bind-attr class=":popup-pane :search-pane showingSearchPane::hidden"}}>
         </section>
     </script>
 
+    <script type="text/x-handlebars" data-template-name="outlier-button">
+        <svg class="outlier-button icon-button" viewBox="0 0 100 100">
+            <g stroke="black" fill="black" stroke-width="15">
+                <line x1="0" y1="70" x2="40" y2="70"/>
+                <circle cx="15" cy="70" r="8"/>
+                <circle cx="45" cy="70" r="8"/>
+                <circle cx="85" cy="70" r="8"/>
+                <line x1="85" y1="70" x2="100" y2="70"/>
+                <g class="show-outlier-icon">
+                    <line x1="45" y1="70" x2="65" y2="20"/>
+                    <line x1="65" y1="20" x2="85" y2="70"/>
+                    <circle cx="65" cy="20" r="8"/>
+                </g>
+                <g class="hide-outlier-icon">
+                    <line x1="45" y1="70" x2="85" y2="70"/>
+                </g>
+            </g>
+        </svg>
+    </script>
+
     <script type="text/x-handlebars" data-template-name="analysis-button">
         <svg class="analysis-button icon-button" viewBox="0 0 100 100">
             <g stroke="black" fill="black" stroke-width="15">
index 1b0c760..94b1175 100644 (file)
@@ -524,9 +524,6 @@ App.InteractiveChartComponent = Ember.Component.extend({
             var yDiff = mY - point.y;
             return xDiff * xDiff + yDiff * yDiff / 16; // Favor horizontal affinity.
         };
-        distanceHeuristics = function (m) {
-            return Math.abs(xScale(m.time) - point.x);
-        }
 
         var newItemIndex;
         if (point && !this._currentSelection()) {
index 4059b2f..6a961aa 100755 (executable)
@@ -333,10 +333,15 @@ App.Manifest = Ember.Controller.extend({
         var unitSuffix = unit ? ' ' + unit : '';
         var deltaFormatterWithoutSign = useSI ? d3.format('.2s') : d3.format('.2g');
 
+        var currentTimeSeries = configurations.current.timeSeriesByCommitTime(false);
+        var baselineTimeSeries = configurations.baseline ? configurations.baseline.timeSeriesByCommitTime(false) : null;
+        var targetTimeSeries = configurations.target ? configurations.target.timeSeriesByCommitTime(false) : null;
+        var unfilteredCurrentTimeSeries, unfilteredBaselineTimeSeries, unfilteredTargetTimeSeries;
+
         return {
-            current: configurations.current.timeSeriesByCommitTime(),
-            baseline: configurations.baseline ? configurations.baseline.timeSeriesByCommitTime() : null,
-            target: configurations.target ? configurations.target.timeSeriesByCommitTime() : null,
+            current: currentTimeSeries,
+            baseline: baselineTimeSeries,
+            target: targetTimeSeries,
             unit: unit,
             formatWithUnit: function (value) { return this.formatter(value) + unitSuffix; },
             formatWithDeltaAndUnit: function (value, delta)
@@ -346,6 +351,17 @@ App.Manifest = Ember.Controller.extend({
             formatter: useSI ? d3.format('.4s') : d3.format('.4g'),
             deltaFormatter: useSI ? d3.format('+.2s') : d3.format('+.2g'),
             smallerIsBetter: smallerIsBetter,
+            showOutlier: function (show)
+            {
+                if (!unfilteredCurrentTimeSeries) {
+                    unfilteredCurrentTimeSeries = configurations.current.timeSeriesByCommitTime(true);
+                    unfilteredBaselineTimeSeries = configurations.baseline ? configurations.baseline.timeSeriesByCommitTime(true) : null;
+                    unfilteredTargetTimeSeries = configurations.target ? configurations.target.timeSeriesByCommitTime(true) : null;
+                }
+                this.current = show ? unfilteredCurrentTimeSeries : currentTimeSeries;
+                this.baseline = show ? unfilteredBaselineTimeSeries : baselineTimeSeries;
+                this.target = show ? unfilteredTargetTimeSeries : targetTimeSeries;
+            },
         };
     }
 }).create();