Introduce the concept of analysis task to perf dashboard
authorrniwa@webkit.org <rniwa@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 7 Nov 2014 23:47:09 +0000 (23:47 +0000)
committerrniwa@webkit.org <rniwa@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 7 Nov 2014 23:47:09 +0000 (23:47 +0000)
https://bugs.webkit.org/show_bug.cgi?id=138517

Reviewed by Andreas Kling.

Introduced the concept of an analysis task, which is created for a range of measurements for a given metric on
a single platform and used to bisect regressions in the range.

Added a new page to see the list of active analysis tasks and a page to view the contents of an analysis task.

* init-database.sql: Added a bunch of tables to store information about analysis tasks.
    analysis_tasks - Represents each analysis task. Associated with a platform and a metric and possibly with two
    test runs. Analysis tasks not associated with test runs are used for try new patches.
    analysis_test_groups - A test group in an analysis task represents a bunch of related A/B testing results.
    root_sets - A root set represents a set of roots (or packages) installed in each A/B testing.
    build_requests - A build request represents a single pending build for A/B testing.

* public/api/analysis-tasks.php: Added. Returns the specified analysis task or all analysis tasks in an array.
(main):
(format_task):

* public/api/test-groups.php: Added. Returns analysis task groups for the specified analysis task or returns
the specified analysis task group as well as build requests associated with them.
(main):
(fetch_test_groups_for_task):
(fetch_build_requests_for_task):
(fetch_build_requests_for_group):
(format_test_group):
(format_build_request):

* public/include/json-header.php:
(remote_user_name): Extracted from compute_token so that we can use it in create-analysis-task.php.
(compute_token):

* public/privileged-api/associate-bug.php:
(main): Fixed a typo.

* public/privileged-api/create-analysis-task.php: Added. Creates a new analysis task for a given test run range.
(main):
(ensure_row_by_id):
(ensure_config_from_runs):

* public/privileged-api/generate-csrf-token.php: Use remote_user_name.

* public/v2/analysis.js: Added. Various Ember data store models to represent analysis tasks and related objects.
(App.AnalysisTask):
(App.AnalysisTask.create):
(App.TestGroup):
(App.TestGroupAdapter):
(App.AnalysisTaskSerializer):
(App.TestGroupSerializer):
(App.BuildRequest):

* public/v2/app.css: Added style rules for the analysis page.

* public/v2/app.js:
(App.Pane._fetch): Use fetchRunsWithPlatformAndMetric, which has been refactored into App.Manifest.

(App.PaneController.actions.toggleBugsPane): Show bugs pane even when there are no bug trackers or there is not
exactly one selected point as we can still create an analysis task.
(App.PaneController.actions.associateBug): Renamed singlySelectedPoint to selectedSinglePoint to be more
grammatical and also alert'ed the error message when there is one.
(App.PaneController.actions.createAnalysisTask): Added. Creates a new analysis task and opens it in a new tab.
Since window.open only works during the click, we open the new "window" preemptively and navigates or closes it
once XHR request has completed.
(App.PaneController._detailsChanged): Changes due to singlySelectedPoint to selectedSinglePoint rename.
(App.PaneController._updateBugs): Fixed a bug that we were showing bugs in the previous point when a single point
is selected in the details pane.

(App.AnalysisRoute): Added.
(App.AnalysisTaskRoute): Added.
(App.AnalysisTaskViewModel): Added.
(App.AnalysisTaskViewModel._taskUpdated): Fetch runs for the associated platform and metric.
(App.AnalysisTaskViewModel._fetchedRuns): Setup the chart data to show.
(App.AnalysisTaskViewModel.testSets): The computed property used to update roots for all repositories/projects.
(App.AnalysisTaskViewModel._rootChangedForTestSet): Updates root selections for all repositories/projects when
the user selects an option for all roots in A or B configuration.
(App.AnalysisTaskViewModel.roots): The computed property used to show root choices for each repository/project.

* public/v2/chart-pane.css: Added style rules for details view in the analysis task page.

* public/v2/data.js:
(Measurement.prototype._formatRevisionRange): Don't prefix a revision number with "At " when there is no previous
point so that we can use it in App.AnalysisTaskViewModel.roots.
(TimeSeries.prototype.findPointByMeasurementId): Added.
(TimeSeries.prototype.seriesBetweenPoints): Added.

* public/v2/index.html: Use Metric.fullName since the same value is needed in the analysis task page. Also added
a button to create an analysis task, and made bugs pane button always enabled since we can an analysis task even
when multiple points are selected. Finally, added a new template for the analysis task page.

* public/v2/manifest.js:
(App.Metric.fullName): Added to share code between the charts page and the analysis task page.
(App.Manifest.fetchRunsWithPlatformAndMetric): Extracted from App.Pane._fetch to be reused in
App.AnalysisTaskViewModel._taskUpdated.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@175768 268f45cc-cd09-0410-ab3c-d52691b4dbfc

15 files changed:
Websites/perf.webkit.org/ChangeLog
Websites/perf.webkit.org/init-database.sql
Websites/perf.webkit.org/public/api/analysis-tasks.php [new file with mode: 0644]
Websites/perf.webkit.org/public/api/test-groups.php [new file with mode: 0644]
Websites/perf.webkit.org/public/include/json-header.php
Websites/perf.webkit.org/public/privileged-api/associate-bug.php
Websites/perf.webkit.org/public/privileged-api/create-analysis-task.php [new file with mode: 0644]
Websites/perf.webkit.org/public/privileged-api/generate-csrf-token.php
Websites/perf.webkit.org/public/v2/analysis.js [new file with mode: 0644]
Websites/perf.webkit.org/public/v2/app.css
Websites/perf.webkit.org/public/v2/app.js
Websites/perf.webkit.org/public/v2/chart-pane.css
Websites/perf.webkit.org/public/v2/data.js
Websites/perf.webkit.org/public/v2/index.html
Websites/perf.webkit.org/public/v2/manifest.js

index 796c7e7..ef9282d 100644 (file)
@@ -1,3 +1,101 @@
+2014-11-07  Ryosuke Niwa  <rniwa@webkit.org>
+
+        Introduce the concept of analysis task to perf dashboard
+        https://bugs.webkit.org/show_bug.cgi?id=138517
+
+        Reviewed by Andreas Kling.
+
+        Introduced the concept of an analysis task, which is created for a range of measurements for a given metric on
+        a single platform and used to bisect regressions in the range.
+        
+        Added a new page to see the list of active analysis tasks and a page to view the contents of an analysis task.
+
+        * init-database.sql: Added a bunch of tables to store information about analysis tasks.
+            analysis_tasks - Represents each analysis task. Associated with a platform and a metric and possibly with two
+            test runs. Analysis tasks not associated with test runs are used for try new patches.
+            analysis_test_groups - A test group in an analysis task represents a bunch of related A/B testing results.
+            root_sets - A root set represents a set of roots (or packages) installed in each A/B testing.
+            build_requests - A build request represents a single pending build for A/B testing.
+
+        * public/api/analysis-tasks.php: Added. Returns the specified analysis task or all analysis tasks in an array.
+        (main):
+        (format_task):
+
+        * public/api/test-groups.php: Added. Returns analysis task groups for the specified analysis task or returns
+        the specified analysis task group as well as build requests associated with them.
+        (main):
+        (fetch_test_groups_for_task):
+        (fetch_build_requests_for_task):
+        (fetch_build_requests_for_group):
+        (format_test_group):
+        (format_build_request):
+
+        * public/include/json-header.php:
+        (remote_user_name): Extracted from compute_token so that we can use it in create-analysis-task.php.
+        (compute_token):
+
+        * public/privileged-api/associate-bug.php:
+        (main): Fixed a typo.
+
+        * public/privileged-api/create-analysis-task.php: Added. Creates a new analysis task for a given test run range.
+        (main):
+        (ensure_row_by_id):
+        (ensure_config_from_runs):
+
+        * public/privileged-api/generate-csrf-token.php: Use remote_user_name.
+
+        * public/v2/analysis.js: Added. Various Ember data store models to represent analysis tasks and related objects.
+        (App.AnalysisTask):
+        (App.AnalysisTask.create):
+        (App.TestGroup):
+        (App.TestGroupAdapter):
+        (App.AnalysisTaskSerializer):
+        (App.TestGroupSerializer):
+        (App.BuildRequest):
+
+        * public/v2/app.css: Added style rules for the analysis page.
+
+        * public/v2/app.js:
+        (App.Pane._fetch): Use fetchRunsWithPlatformAndMetric, which has been refactored into App.Manifest.
+
+        (App.PaneController.actions.toggleBugsPane): Show bugs pane even when there are no bug trackers or there is not
+        exactly one selected point as we can still create an analysis task.
+        (App.PaneController.actions.associateBug): Renamed singlySelectedPoint to selectedSinglePoint to be more
+        grammatical and also alert'ed the error message when there is one.
+        (App.PaneController.actions.createAnalysisTask): Added. Creates a new analysis task and opens it in a new tab.
+        Since window.open only works during the click, we open the new "window" preemptively and navigates or closes it
+        once XHR request has completed.
+        (App.PaneController._detailsChanged): Changes due to singlySelectedPoint to selectedSinglePoint rename.
+        (App.PaneController._updateBugs): Fixed a bug that we were showing bugs in the previous point when a single point
+        is selected in the details pane.
+
+        (App.AnalysisRoute): Added.
+        (App.AnalysisTaskRoute): Added.
+        (App.AnalysisTaskViewModel): Added.
+        (App.AnalysisTaskViewModel._taskUpdated): Fetch runs for the associated platform and metric.
+        (App.AnalysisTaskViewModel._fetchedRuns): Setup the chart data to show.
+        (App.AnalysisTaskViewModel.testSets): The computed property used to update roots for all repositories/projects.
+        (App.AnalysisTaskViewModel._rootChangedForTestSet): Updates root selections for all repositories/projects when
+        the user selects an option for all roots in A or B configuration.
+        (App.AnalysisTaskViewModel.roots): The computed property used to show root choices for each repository/project.
+
+        * public/v2/chart-pane.css: Added style rules for details view in the analysis task page.
+
+        * public/v2/data.js:
+        (Measurement.prototype._formatRevisionRange): Don't prefix a revision number with "At " when there is no previous
+        point so that we can use it in App.AnalysisTaskViewModel.roots.
+        (TimeSeries.prototype.findPointByMeasurementId): Added.
+        (TimeSeries.prototype.seriesBetweenPoints): Added.
+
+        * public/v2/index.html: Use Metric.fullName since the same value is needed in the analysis task page. Also added
+        a button to create an analysis task, and made bugs pane button always enabled since we can an analysis task even
+        when multiple points are selected. Finally, added a new template for the analysis task page.
+
+        * public/v2/manifest.js:
+        (App.Metric.fullName): Added to share code between the charts page and the analysis task page.
+        (App.Manifest.fetchRunsWithPlatformAndMetric): Extracted from App.Pane._fetch to be reused in
+        App.AnalysisTaskViewModel._taskUpdated.
+
 2014-10-28  Ryosuke Niwa  <rniwa@webkit.org>
 
         Remove App.PaneController.bugsChangeCount in the new perf dashboard
index e527ab0..73ed981 100644 (file)
@@ -138,3 +138,33 @@ CREATE TABLE bugs (
     CONSTRAINT bug_tracker_and_run_must_be_unique UNIQUE(bug_tracker, bug_run));
 CREATE INDEX bugs_tracker_number_index ON bugs(bug_tracker, bug_number);
 CREATE INDEX bugs_run_index ON bugs(bug_run);
+
+CREATE TABLE analysis_tasks (
+    task_id serial PRIMARY KEY,
+    task_name varchar(256) NOT NULL,
+    task_author varchar(256),
+    task_created_at timestamp NOT NULL DEFAULT (CURRENT_TIMESTAMP AT TIME ZONE 'UTC'),
+    task_platform integer REFERENCES platforms NOT NULL,
+    task_metric integer REFERENCES test_metrics NOT NULL,
+    task_start_run integer REFERENCES test_runs,
+    task_end_run integer REFERENCES test_runs,
+    CONSTRAINT analysis_task_should_be_unique_for_range UNIQUE(task_start_run, task_end_run));
+
+CREATE TABLE analysis_test_groups (
+    testgroup_id serial PRIMARY KEY,
+    testgroup_task integer REFERENCES analysis_tasks NOT NULL,
+    testgroup_name varchar(256),
+    testgroup_author varchar(256) NOT NULL,
+    testgroup_created_at timestamp NOT NULL DEFAULT (CURRENT_TIMESTAMP AT TIME ZONE 'UTC'));
+CREATE INDEX testgroup_task_index ON analysis_test_groups(testgroup_task);
+
+CREATE TABLE root_sets (
+    rootset_id serial PRIMARY KEY);
+
+CREATE TABLE build_requests (
+    request_id serial PRIMARY KEY,
+    request_group integer REFERENCES analysis_test_groups NOT NULL,
+    request_order integer NOT NULL,
+    request_root_set integer REFERENCES root_sets NOT NULL,
+    request_build integer REFERENCES builds,
+    CONSTRAINT build_request_order_must_be_unique_in_group UNIQUE(request_group, request_order));
diff --git a/Websites/perf.webkit.org/public/api/analysis-tasks.php b/Websites/perf.webkit.org/public/api/analysis-tasks.php
new file mode 100644 (file)
index 0000000..abf0117
--- /dev/null
@@ -0,0 +1,45 @@
+<?php
+
+require('../include/json-header.php');
+
+function main($path) {
+    $db = new Database;
+    if (!$db->connect())
+        exit_with_error('DatabaseConnectionFailure');
+
+    if (count($path) > 1)
+        exit_with_error('InvalidRequest');
+
+    if (count($path) > 0 && $path[0]) {
+        $task_id = intval($path[0]);
+        $task = $db->select_first_row('analysis_tasks', 'task', array('id' => $task_id));
+        if (!$task)
+            exit_with_error('TaskNotFound', array('id' => $task_id));
+        $tasks = array($task);
+    } else {
+        // FIXME: Limit the number of tasks we fetch.
+        $tasks = array_reverse($db->fetch_table('analysis_tasks', 'task_created_at'));
+        if (!is_array($tasks))
+            exit_with_error('FailedToFetchTasks');
+    }
+
+    exit_with_success(array('analysisTasks' => array_map("format_task", $tasks)));
+}
+
+date_default_timezone_set('UTC');
+function format_task($task_row) {
+    return array(
+        'id' => $task_row['task_id'],
+        'name' => $task_row['task_name'],
+        'author' => $task_row['task_author'],
+        'createdAt' => strtotime($task_row['task_created_at']) * 1000,
+        'platform' => $task_row['task_platform'],
+        'metric' => $task_row['task_metric'],
+        'startRun' => $task_row['task_start_run'],
+        'endRun' => $task_row['task_end_run'],
+    );
+}
+
+main(array_key_exists('PATH_INFO', $_SERVER) ? explode('/', trim($_SERVER['PATH_INFO'], '/')) : array());
+
+?>
diff --git a/Websites/perf.webkit.org/public/api/test-groups.php b/Websites/perf.webkit.org/public/api/test-groups.php
new file mode 100644 (file)
index 0000000..94c5540
--- /dev/null
@@ -0,0 +1,89 @@
+<?php
+
+require('../include/json-header.php');
+
+function main($path) {
+    $db = new Database;
+    if (!$db->connect())
+        exit_with_error('DatabaseConnectionFailure');
+
+    if (count($path) > 1)
+        exit_with_error('InvalidRequest');
+
+    if (count($path) > 0 && $path[0]) {
+        $group_id = intval($path[0]);
+        $group = $db->select_first_row('analysis_test_groups', 'testgroup', array('id' => $group_id));
+        if (!$group)
+            exit_with_error('GroupNotFound', array('id' => $group_id));
+        $test_groups = array($group);
+        $build_requests = fetch_build_requests_for_group($db, $group_id);
+    } else {
+        $task_id = array_get($_GET, 'task');
+        if (!$task_id)
+            exit_with_error('TaskIdNotSpecified');
+
+        $test_groups = fetch_test_groups_for_task($db, $task_id);
+        if (!is_array($test_groups))
+            exit_with_error('FailedToFetchTestGroups');
+        $build_requests = fetch_build_requests_for_task($db, $task_id);
+    }
+    if (!is_array($build_requests))
+        exit_with_error('FailedToFetchBuildRequests');
+
+    $test_groups = array_map("format_test_group", $test_groups);
+    $group_by_id = array();
+    foreach ($test_groups as &$group)
+        $group_by_id[$group['id']] = &$group;
+
+    $build_requests = array_map("format_build_request", $build_requests);
+    foreach ($build_requests as $request)
+        array_push($group_by_id[$request['testGroup']]['buildRequests'], $request['id']);
+
+    exit_with_success(array('testGroups' => $test_groups, 'buildRequests' => $build_requests));
+}
+
+function fetch_test_groups_for_task($db, $task_id) {
+    return $db->query_and_fetch_all('SELECT * FROM analysis_test_groups WHERE testgroup_task = $1
+        ORDER BY testgroup_created_at', array($task_id));
+}
+
+function fetch_build_requests_for_task($db, $task_id) {
+    return $db->query_and_fetch_all('SELECT * FROM build_requests, builds
+        WHERE request_build = build_id
+            AND request_group IN (SELECT testgroup_id FROM analysis_test_groups WHERE testgroup_task = $1)
+        ORDER BY request_group, request_order', array($task_id));
+}
+
+function fetch_build_requests_for_group($db, $test_group_id) {
+    return $db->query_and_fetch_all('SELECT * FROM build_requests, builds
+        WHERE request_build = build_id AND request_group = $1 ORDER BY request_order', array($test_group_id));
+}
+
+date_default_timezone_set('UTC');
+function format_test_group($group_row) {
+    return array(
+        'id' => $group_row['testgroup_id'],
+        'task' => $group_row['testgroup_task'],
+        'name' => $group_row['testgroup_name'],
+        'author' => $group_row['testgroup_author'],
+        'createdAt' => strtotime($group_row['testgroup_created_at']) * 1000,
+        'buildRequests' => array(),
+    );
+}
+
+function format_build_request($request_row) {
+    return array(
+        'id' => $request_row['request_id'],
+        'testGroup' => $request_row['request_group'],
+        'order' => $request_row['request_order'],
+        'rootSet' => $request_row['request_root_set'],
+        'build' => $request_row['request_build'],
+        'builder' => $request_row['build_builder'],
+        'buildNumber' => $request_row['build_number'],
+        'buildTime' => $request_row['build_time'] ? strtotime($request_row['build_time']) * 1000 : NULL,
+    );
+}
+
+main(array_key_exists('PATH_INFO', $_SERVER) ? explode('/', trim($_SERVER['PATH_INFO'], '/')) : array());
+
+?>
index 3504319..8da4a76 100644 (file)
@@ -93,10 +93,14 @@ function ensure_privileged_api_data_and_token() {
     return $data;
 }
 
+function remote_user_name() {
+    return array_get($_SERVER, 'REMOTE_USER');
+}
+
 function compute_token() {
     if (!array_key_exists('CSRFSalt', $_COOKIE) || !array_key_exists('CSRFExpiration', $_COOKIE))
         return NULL;
-    $user = array_get($_SERVER, 'REMOTE_USER');
+    $user = remote_user_name();
     $salt = $_COOKIE['CSRFSalt'];
     $expiration = $_COOKIE['CSRFExpiration'];
     return hash('sha256', "$salt|$user|$expiration");
index 6734005..4eda7f7 100644 (file)
@@ -31,7 +31,7 @@ function main() {
     }
     $db->commit_transaction();
 
-    exit_with_success(array('bug_id' => $bug_id));
+    exit_with_success(array('bugId' => $bug_id));
 }
 
 main();
diff --git a/Websites/perf.webkit.org/public/privileged-api/create-analysis-task.php b/Websites/perf.webkit.org/public/privileged-api/create-analysis-task.php
new file mode 100644 (file)
index 0000000..cb8e522
--- /dev/null
@@ -0,0 +1,62 @@
+<?php
+
+require_once('../include/json-header.php');
+
+function main() {
+    $data = ensure_privileged_api_data_and_token();
+
+    $author = remote_user_name();
+    $name = array_get($data, 'name');
+    $start_run_id = array_get($data, 'startRun');
+    $end_run_id = array_get($data, 'endRun');
+
+    if (!$name)
+        exit_with_error('MissingName', array('name' => $name));
+    $range = array('startRunId' => $start_run_id, 'endRunId' => $end_run_id);
+    if (!$start_run_id || !$end_run_id)
+        exit_with_error('MissingRange', $range);
+
+    $db = connect();
+    $start_run = ensure_row_by_id($db, 'test_runs', 'run', $start_run_id, 'InvalidStartRun', $range);
+    $end_run = ensure_row_by_id($db, 'test_runs', 'run', $end_run_id, 'InvalidEndRun', $range);
+
+    $config = ensure_config_from_runs($db, $start_run, $end_run);
+
+    $db->begin_transaction();
+    $duplicate = $db->select_first_row('analysis_tasks', 'task', array('start_run' => $start_run_id, 'end_run' => $end_run_id));
+    if ($duplicate) {
+        $db->rollback_transaction();
+        exit_with_error('DuplicateAnalysisTask', array('duplicate' => $duplicate));
+    }
+
+    $task_id = $db->insert_row('analysis_tasks', 'task', array(
+        'name' => $name,
+        'author' => $author,
+        'platform' => $config['config_platform'],
+        'metric' => $config['config_metric'],
+        'start_run' => $start_run_id,
+        'end_run' => $end_run_id));
+    $db->commit_transaction();
+
+    exit_with_success(array('taskId' => $task_id));
+}
+
+function ensure_row_by_id($db, $table, $prefix, $id, $error_name, $error_params) {
+    $row = $db->select_first_row($table, $prefix, array('id' => $id));
+    if (!$row)
+        exit_with_error($error_name, array($error_params));
+    return $row;
+}
+
+function ensure_config_from_runs($db, $start_run, $end_run) {
+    $range = array('startRun' => $start_run, 'endRun' => $end_run);
+
+    if ($start_run['run_config'] != $end_run['run_config'])
+        exit_with_error('RunConfigMismatch', $range);
+
+    return ensure_row_by_id($db, 'test_configurations', 'config', $start_run['run_config'], 'ConfigNotFound', $range);
+}
+
+main();
+
+?>
index 45d57a5..be3f0fe 100644 (file)
@@ -4,8 +4,6 @@ require_once('../include/json-header.php');
 
 ensure_privileged_api_data();
 
-$user = array_get($_SERVER, 'REMOTE_USER');
-
 $expiritaion = time() + 3600; // Valid for one hour.
 $_COOKIE['CSRFSalt'] = rand();
 $_COOKIE['CSRFExpiration'] = $expiritaion;
@@ -13,6 +11,6 @@ $_COOKIE['CSRFExpiration'] = $expiritaion;
 setcookie('CSRFSalt', $_COOKIE['CSRFSalt']);
 setcookie('CSRFExpiration', $expiritaion);
 
-exit_with_success(array('user' => $user, 'token' => compute_token(), 'expiration' => $expiritaion * 1000));
+exit_with_success(array('user' => remote_user_name(), 'token' => compute_token(), 'expiration' => $expiritaion * 1000));
 
 ?>
diff --git a/Websites/perf.webkit.org/public/v2/analysis.js b/Websites/perf.webkit.org/public/v2/analysis.js
new file mode 100644 (file)
index 0000000..05e4631
--- /dev/null
@@ -0,0 +1,60 @@
+App.AnalysisTask = App.NameLabelModel.extend({
+    author: DS.attr('string'),
+    createdAt: DS.attr('date'),
+    platform: DS.belongsTo('platform'),
+    metric: DS.belongsTo('metric'),
+    startRun: DS.attr('number'),
+    endRun: DS.attr('number'),
+    testGroups: function () {
+        return this.store.find('testGroup', {task: this.get('id')});
+    }.property(),
+});
+
+// FIXME: Use DS.RESTAdapter instead.
+App.AnalysisTask.create = function (name, startMeasurement, endMeasurement)
+{
+    return PrivilegedAPI.sendRequest('create-analysis-task', {
+        name: name,
+        startRun: startMeasurement.id(),
+        endRun: endMeasurement.id(),
+    });
+}
+
+App.AnalysisTaskAdapter = DS.RESTAdapter.extend({
+    buildURL: function (type, id)
+    {
+        return '../api/analysis-tasks/' + (id ? id : '');
+    },
+});
+
+App.TestGroup = App.NameLabelModel.extend({
+    analysisTask: DS.belongsTo('analysisTask'),
+    author: DS.attr('string'),
+    createdAt: DS.attr('date'),
+    buildRequests: DS.hasMany('buildRequests'),
+});
+
+App.TestGroupAdapter = DS.RESTAdapter.extend({
+    buildURL: function (type, id)
+    {
+        return '../api/test-groups/' + (id ? id : '');
+    },
+});
+
+App.AnalysisTaskSerializer = App.TestGroupSerializer = DS.RESTSerializer.extend({
+    normalizePayload: function (payload)
+    {
+        delete payload['status'];
+        return payload;
+    }
+});
+
+App.BuildRequest = DS.Model.extend({
+    group: DS.belongsTo('testGroup'),
+    order: DS.attr('number'),
+    rootSet: DS.attr('number'),
+    build: DS.attr('number'),
+    buildNumber: DS.attr('number'),
+    buildBuilder: DS.belongsTo('builder'),
+    buildTime: DS.attr('date'),
+});
index bb33db9..7a4b780 100755 (executable)
@@ -411,3 +411,63 @@ table.dashboard tbody td .progress {
     line-height: 12rem;
     vertical-align: center;
 }
+
+#analysis-tasks,
+.test-groups > table {
+    border: solid 0px #999;
+    border-collapse: collapse;
+}
+
+#analysis-tasks thead,
+.test-groups > table thead {
+    color: #c93;
+}
+
+#analysis-tasks th,
+.test-groups > table th {
+    font-weight: normal;
+}
+
+#analysis-tasks td,
+#analysis-tasks th,
+.test-groups > table td,
+.test-groups > table th {
+    padding: 0.2rem 0.5rem;
+}
+
+#analysis-tasks tbody td,
+#analysis-tasks tbody th,
+.test-groups > table tbody td,
+.test-groups > table tbody th {
+    border-top: solid 1px #ddd;
+}
+
+#analysis-task-title {
+    font-weight: normal;
+    font-size: 1.2rem;
+    margin: 0 0 0 0.5rem;
+    padding: 0;
+}
+
+#analysis-task-testname {
+    font-weight: normal;
+    font-size: 1rem;
+    margin: 0 0 1rem 0.5rem;
+    padding: 0;
+    color: #333;
+}
+
+.test-groups {
+    border: 1px solid #bbb;
+    border-radius: 0.5rem;
+    box-shadow: rgba(0, 0, 0, 0.03) 1px 1px 0px 0px;
+
+    padding: 0.5rem 1rem;
+    margin-bottom: 1.5rem;
+}
+
+.test-groups caption {
+    font-size: 1.1rem;
+    text-align: left;
+    margin-bottom: 0.5rem;
+}
index e984286..b51fb29 100755 (executable)
@@ -2,6 +2,8 @@ window.App = Ember.Application.create();
 
 App.Router.map(function () {
     this.resource('charts', {path: 'charts'});
+    this.resource('analysis', {path: 'analysis'});
+    this.resource('analysisTask', {path: 'analysis/task/:taskId'});
 });
 
 App.DashboardRow = Ember.Object.extend({
@@ -338,45 +340,19 @@ App.Pane = Ember.Object.extend({
         else {
             var self = this;
 
-            var metric;
-            var manifestPromise = App.Manifest.fetch(this.store).then(function () {
-                return new Ember.RSVP.Promise(function (resolve, reject) {
-                    var platform = App.Manifest.platform(platformId);
-                    metric = App.Manifest.metric(metricId);
-                    if (!platform)
-                        reject('Could not find the platform "' + platformId + '"');
-                    else if (!metric)
-                        reject('Could not find the metric "' + metricId + '"');
-                    else {
-                        self.set('platform', platform);
-                        self.set('metric', metric);
-                        resolve(null);
-                    }
-                });
-            });
-
-            Ember.RSVP.all([
-                RunsData.fetchRuns(platformId, metricId),
-                manifestPromise,
-            ]).then(function (values) {
-                var runs = values[0];
-
-                // FIXME: Include this information in JSON and process it in RunsData.fetchRuns
-                var unit = {'Combined': '', // Assume smaller is better for now.
-                    'FrameRate': 'fps',
-                    'Runs': 'runs/s',
-                    'Time': 'ms',
-                    'Malloc': 'bytes',
-                    'JSHeap': 'bytes',
-                    'Allocations': 'bytes',
-                    'EndAllocations': 'bytes',
-                    'MaxAllocations': 'bytes',
-                    'MeanAllocations': 'bytes'}[metric.get('name')];
-                runs.unit = unit;
-
-                self.set('chartData', runs);
-            }, function (status) {
-                self.set('failure', 'Failed to fetch the JSON with an error: ' + status);
+            App.Manifest.fetchRunsWithPlatformAndMetric(this.store, platformId, metricId).then(function (result) {
+                self.set('platform', result.platform);
+                self.set('metric', result.metric);
+                self.set('chartData', result.runs);
+            }, function (result) {
+                if (!result || typeof(result) === "string")
+                    self.set('failure', 'Failed to fetch the JSON with an error: ' + result);
+                else if (!result.platform)
+                    self.set('failure', 'Could not find the platform "' + platformId + '"');
+                else if (!result.metric)
+                    self.set('failure', 'Could not find the metric "' + metricId + '"');
+                else
+                    self.set('failure', 'An internal error');
             });
         }
     }.observes('platformId', 'metricId').on('init'),
@@ -675,20 +651,40 @@ App.PaneController = Ember.ObjectController.extend({
         },
         toggleBugsPane: function ()
         {
-            if (!App.Manifest.bugTrackers || !this.get('singlySelectedPoint'))
-                return;
             if (this.toggleProperty('showingBugsPane'))
                 this.set('showingSearchPane', false);
         },
         associateBug: function (bugTracker, bugNumber)
         {
-            var point = this.get('singlySelectedPoint');
+            var point = this.get('selectedSinglePoint');
             if (!point)
                 return;
             var self = this;
             point.measurement.associateBug(bugTracker.get('id'), bugNumber).then(function () {
                 self._updateBugs();
                 self._updateMarkedPoints();
+            }, function (error) {
+                alert(error);
+            });
+        },
+        createAnalysisTask: function ()
+        {
+            var name = this.get('newAnalysisTaskName');
+            var points = this._selectedPoints;
+            if (!name || !points || points.length < 2)
+                return;
+
+            var newWindow = window.open();
+            App.AnalysisTask.create(name, points[0].measurement, points[points.length - 1].measurement).then(function (data) {
+                // FIXME: Update the UI to show the new analysis task.
+                var url = App.Router.router.generate('analysisTask', data['taskId']);
+                newWindow.location.href = '#' + url;
+            }, function (error) {
+                newWindow.close();
+                if (error === 'DuplicateAnalysisTask') {
+                    // FIXME: Duplicate this error more gracefully.
+                }
+                alert(error);
             });
         },
         toggleSearchPane: function ()
@@ -732,7 +728,7 @@ App.PaneController = Ember.ObjectController.extend({
     _detailsChanged: function ()
     {
         this.set('showingBugsPane', false);
-        this.set('singlySelectedPoint', !this._hasRange && this._selectedPoints ? this._selectedPoints[0] : null);
+        this.set('selectedSinglePoint', !this._hasRange && this._selectedPoints ? this._selectedPoints[0] : null);
     }.observes('details'),
     _overviewSelectionChanged: function ()
     {
@@ -820,7 +816,9 @@ App.PaneController = Ember.ObjectController.extend({
         var bugTrackers = App.Manifest.get('bugTrackers');
         var trackerToBugNumbers = {};
         bugTrackers.forEach(function (tracker) { trackerToBugNumbers[tracker.get('id')] = new Array(); });
-        this._selectedPoints.map(function (point) {
+
+        var points = this._hasRange ? this._selectedPoints : [this._selectedPoints[1]];
+        points.map(function (point) {
             var bugs = point.measurement.bugs();
             bugTrackers.forEach(function (tracker) {
                 var bugNumber = bugs[tracker.get('id')];
@@ -1582,3 +1580,131 @@ App.CommitsViewerComponent = Ember.Component.extend({
         })
     }.observes('repository').observes('revisionInfo').on('init'),
 });
+
+
+App.AnalysisRoute = Ember.Route.extend({
+    model: function () {
+        return this.store.findAll('analysisTask').then(function (tasks) {
+            return Ember.Object.create({'tasks': tasks});
+        });
+    },
+});
+
+App.AnalysisTaskRoute = Ember.Route.extend({
+    model: function (param) {
+        var store = this.store;
+        return this.store.find('analysisTask', param.taskId).then(function (task) {
+            return App.AnalysisTaskViewModel.create({content: task});
+        });
+    },
+});
+
+App.AnalysisTaskViewModel = Ember.ObjectProxy.extend({
+    testSets: [],
+    roots: [],
+    _taskUpdated: function ()
+    {
+        var platformId = this.get('platform').get('id');
+        var metricId = this.get('metric').get('id');
+        App.Manifest.fetchRunsWithPlatformAndMetric(this.store, platformId, metricId).then(this._fetchedRuns.bind(this));
+    }.observes('platform', 'metric').on('init'),
+    _fetchedRuns: function (data) {
+        var runs = data.runs;
+
+        var currentTimeSeries = runs.current.timeSeriesByCommitTime();
+        if (!currentTimeSeries)
+            return; // FIXME: Report an error.
+
+        var start = currentTimeSeries.findPointByMeasurementId(this.get('startRun'));
+        var end = currentTimeSeries.findPointByMeasurementId(this.get('endRun'));
+        if (!start || !end)
+            return; // FIXME: Report an error.
+
+        var markedPoints = {};
+        markedPoints[start.measurement.id()] = true;
+        markedPoints[end.measurement.id()] = true;
+
+        var formatedPoints = currentTimeSeries.seriesBetweenPoints(start, end).map(function (point, index) {
+            return {
+                id: point.measurement.id(),
+                measurement: point.measurement,
+                label: 'Point ' + (index + 1),
+                value: point.value + (runs.unit ? ' ' + runs.unit : ''),
+            };
+        });
+
+        var margin = (end.time - start.time) * 0.1;
+        this.set('chartData', runs);
+        this.set('chartDomain', [start.time - margin, +end.time + margin]);
+        this.set('markedPoints', markedPoints);
+        this.set('analysisPoints', formatedPoints);
+    },
+    testSets: function ()
+    {
+        var analysisPoints = this.get('analysisPoints');
+        if (!analysisPoints)
+            return;
+        var pointOptions = [{value: ' ', label: 'None'}]
+            .concat(analysisPoints.map(function (point) { return {value: point.id, label: point.label}; }));
+        return [
+            Ember.Object.create({name: "A", options: pointOptions, selection: pointOptions[1]}),
+            Ember.Object.create({name: "B", options: pointOptions, selection: pointOptions[pointOptions.length - 1]}),
+        ];
+    }.property('analysisPoints'),
+    _rootChangedForTestSet: function () {
+        var sets = this.get('testSets');
+        var roots = this.get('roots');
+        if (!sets || !roots)
+            return;
+
+        sets.forEach(function (testSet, setIndex) {
+            var currentSelection = testSet.get('selection');
+            if (currentSelection == testSet.get('previousSelection'))
+                return;
+            testSet.set('previousSelection', currentSelection);
+            var pointIndex = testSet.get('options').indexOf(currentSelection);
+
+            roots.forEach(function (root) {
+                var set = root.sets[setIndex];
+                set.set('selection', set.revisions[pointIndex]);
+            });
+        });
+
+    }.observes('testSets.@each.selection'),
+    _updateRoots: function ()
+    {
+        var analysisPoints = this.get('analysisPoints');
+        if (!analysisPoints)
+            return [];
+        var repositoryToRevisions = {};
+        analysisPoints.forEach(function (point, pointIndex) {
+            var revisions = point.measurement.formattedRevisions();
+            for (var repositoryName in revisions) {
+                if (!repositoryToRevisions[repositoryName])
+                    repositoryToRevisions[repositoryName] = new Array(analysisPoints.length);
+                var revision = revisions[repositoryName];
+                repositoryToRevisions[repositoryName][pointIndex] = {
+                    label: point.label + ': ' + revision.label,
+                    value: revision.currentRevision,
+                };
+            }
+        });
+
+        var roots = [];
+        for (var repositoryName in repositoryToRevisions) {
+            var revisions = [{value: ' ', label: 'None'}].concat(repositoryToRevisions[repositoryName]);
+            roots.push(Ember.Object.create({
+                name: repositoryName,
+                sets: [
+                    Ember.Object.create({name: 'A[' + repositoryName + ']',
+                        revisions: revisions,
+                        selection: revisions[1]}),
+                    Ember.Object.create({name: 'B[' + repositoryName + ']',
+                        revisions: revisions,
+                        selection: revisions[revisions.length - 1]}),
+                ],
+            }));
+        }
+        return rooots;
+    }.property('analysisPoints'),
+});
index 3b0b7f2..d99ffc1 100755 (executable)
     border-left: solid 1px #bbb;
 }
 
+.analysis-chart-pane .details {
+    overflow: scroll;
+}
+
+.analysis-chart-pane .details table {
+    margin: 0.5rem;
+}
+
 .chart-pane .overview {
     height: 5rem;
     border-bottom: solid 0px #eee;
index 68317ce..3d10e38 100755 (executable)
@@ -198,18 +198,18 @@ Measurement.prototype._formatRevisionRange = function (previousRevision, current
     } else if (currentRevision.indexOf(' ') >= 0) // e.g. 10.9 13C64.
         revisionDelimiter = ' - ';
     else if (currentRevision.length == 40) { // e.g. git hash
-        formattedCurrentHash = currentRevision.substring(0, 8);
+        var formattedCurrentHash = currentRevision.substring(0, 8);
         if (previousRevision)
             label = previousRevision.substring(0, 8) + '..' + formattedCurrentHash;
         else
-            label = 'At ' + formattedCurrentHash;
+            label = formattedCurrentHash;
     }
 
     if (!label) {
         if (previousRevision)
             label = revisionPrefix + previousRevision + revisionDelimiter + revisionPrefix + currentRevision;
         else
-            label = 'At ' + revisionPrefix + currentRevision;
+            label = revisionPrefix + currentRevision;
     }
 
     return {
@@ -375,6 +375,18 @@ function TimeSeries(series)
     this._max = max;
 }
 
+TimeSeries.prototype.findPointByMeasurementId = function (measurementId)
+{
+    return this._series.find(function (point) { return point.measurement.id() == measurementId; });
+}
+
+TimeSeries.prototype.seriesBetweenPoints = function (startPoint, endPoint)
+{
+    if (!startPoint.seriesIndex || !endPoint.seriesIndex)
+        return null;
+    return this._series.slice(startPoint.seriesIndex, endPoint.seriesIndex + 1);
+}
+
 TimeSeries.prototype.minMaxForTimeRange = function (startTime, endTime)
 {
     var data = this._series;
index 892d956..433d1cb 100755 (executable)
@@ -12,6 +12,7 @@
     <script src="data.js" defer></script>
     <script src="app.js" defer></script>
     <script src="manifest.js" defer></script>
+    <script src="analysis.js" defer></script>
     <script src="popup.js" defer></script>
     <link rel="stylesheet" href="app.css">
     <link rel="stylesheet" href="chart-pane.css">
         {{#each panes itemController="pane"}}
             <section class="chart-pane" tabindex="0">
                 <header>
-                    <h1 {{action "toggleDetails"}}>
-                    {{#each metric.path}}
-                        {{this}} &ni;
-                    {{/each}}
-                    {{metric.label}}
-                    - {{ platform.name}}</h2>
+                    <h1 {{action "toggleDetails"}}>{{metric.fullName}} - {{ platform.name}}</h1>
                     <a href="#" title="Close" class="close-button" {{action "close"}}>{{partial "close-button"}}</a>
                     {{#if App.Manifest.bugTrackers}}
-                        <a href="#" title="Bugs"
-                            {{bind-attr class=":bugs-button singlySelectedPoint::disabled"}}
-                            {{action "toggleBugsPane"}}>
+                        <a href="#" title="Bugs and Analysis" class="bugs-button" {{action "toggleBugsPane"}}>
                             {{partial "bugs-button"}}
                         </a>
                     {{/if}}
 
                 <div {{bind-attr class=":bugs-pane showingBugsPane::hidden"}}>
                     <table>
-                        {{#each details.bugTrackers}}
-                            <tr>
-                                <th>{{label}}</th>
-                                <td>
-                                    <form {{action "associateBug" this editedBugNumber on="submit"}}>
-                                        {{input type=text value=editedBugNumber}}
-                                    </form>
-                                </td>
-                            </tr>
-                        {{/each}}
+                        {{#if selectedSinglePoint}}
+                            {{#each details.bugTrackers}}
+                                <tr>
+                                    <th>{{label}}</th>
+                                    <td>
+                                        <form {{action "associateBug" this editedBugNumber on="submit"}}>
+                                            {{input type=text value=editedBugNumber}}
+                                        </form>
+                                    </td>
+                                </tr>
+                            {{/each}}
+                        {{/if}}
+                        <tr>
+                            <th>
+                                <label>Name: {{input type=text value=newAnalysisTaskName}}</label>
+                                <button {{action "createAnalysisTask"}}>Analyze</button>
+                            </th>
+                        </tr>
                     </table>
                 </div>
 
                 {{#link-to 'charts' tagName='li'}}
                     {{#link-to 'charts'}}Charts{{/link-to}}
                 {{/link-to}}
+                {{#link-to 'analysis' tagName='li'}}
+                    {{#link-to 'analysis'}}Analysis{{/link-to}}
+                {{/link-to}}
             </ul>
         </nav>
     </script>
         {{/each}}
     </script>
 
+    <script type="text/x-handlebars" data-template-name="analysis">
+        <header id="header">
+            {{partial "navbar"}}
+        </header>
+
+        <table id="analysis-tasks">
+            <thead>
+                <tr>
+                    <td>ID</td>
+                    <td>Name</td>
+                    <td>Created at</td>
+                </tr>
+            </thead>
+            <tbody>
+                {{#each model.tasks}}
+                    <tr>
+                        <td>{{#link-to 'analysisTask' id}}{{id}}{{/link-to}}</td>
+                        <td>{{name}}</td>
+                        <td>{{createdAt}}</td>
+                    </tr>
+                {{/each}}
+            </tbody>
+        </table>
+    </script>
+
+    <script type="text/x-handlebars" data-template-name="analysisTask">
+        <header id="header">
+            {{partial "navbar"}}
+        </header>
+
+        <h2 id="analysis-task-title">{{name}}</h2>
+        {{#if platform.label}}
+            <h3 id="analysis-task-testname">{{metric.fullName}} - {{platform.label}}</h3>
+
+            <section class="analysis-chart-pane chart-pane">
+                <div class="svg-container">
+                    {{interactive-chart
+                        chartData=chartData
+                        enableSelection=false
+                        chartPointRadius=2
+                        domain=chartDomain
+                        markedPoints=markedPoints}}
+                </div>
+                <div class="details">
+                    <table>
+                        <tbody>
+                            {{#each analysisPoints}}
+                                <tr><td>{{label}}</td><td>{{value}}</td></tr>
+                            {{/each}}
+                        </tbody>
+                    </table>
+                </div>
+            </section>
+
+            {{#each testGroups}}
+                <section class="test-groups">
+                    <table>
+                        <caption>{{name}}</caption>
+                        <thead>
+                            <tr>
+                                <td>Configuration</td>
+                                <td>Build</td>
+                                <td>Build Time</td>
+                                <td>{{../metric.fullName}}</td>
+                            </tr>
+                        </thead>
+                        <tbody>
+                            {{#each buildRequests}}
+                                <tr>
+                                    <td>{{id}}</td>
+                                    <td>{{buildNumber}}</td>
+                                    <td>{{buildTime}}</td>
+                                    <td>{{mean}}</td>
+                                </tr>
+                            {{/each}}
+                        </tbody>
+                    </table>
+                </section>
+            {{/each}}
+
+            <form class="test-groups">
+                <table>
+                    <caption><input name="name" placeholder="Test group name" required></caption>
+                    <thead>
+                        <tr>
+                            <th>Root</th>
+                            {{#each testSets}}
+                                <th>
+                                    {{name}}
+                                    {{view Ember.Select
+                                        content=options
+                                        optionValuePath="content.value"
+                                        optionLabelPath="content.label"
+                                        selection=selection}}
+                                </th>
+                            {{/each}}
+                        </tr>
+                    </thead>
+                    <tbody>
+                        {{#each roots}}
+                            <tr>
+                                <th>{{name}}</th>
+                                {{#each sets}}
+                                    <td>{{view Ember.Select name=name content=revisions
+                                        optionValuePath="content.value" optionLabelPath="content.label"
+                                        selection=selection}}</td>
+                                {{/each}}
+                            </tr>
+                        {{/each}}
+                    </tbody>
+                    <tbody>
+                        <tr>
+                            <th>Number of runs</th>
+                            <td colspan=2>
+                                <select>
+                                    <option>1</option>
+                                    <option>2</option>
+                                    <option>3</option>
+                                    <option>4</option>
+                                    <option>5</option>
+                                    <option>6</option>
+                                </select>
+                            </td>
+                        </tr>
+                    </tbody>
+                </table>
+
+                <button type="submit">Start A/B testing</button>
+            </form>
+        {{/if}}
+    </script>
+
 </head>
 <body>
 </body>
index f579d10..64e3787 100755 (executable)
@@ -30,6 +30,11 @@ App.Metric = App.NameLabelModel.extend({
         }
         return path.reverse();
     }.property('name', 'test'),
+    fullName: function ()
+    {
+        return this.get('path').join(' \u2208 ') /* &in; */
+            + ' : ' + this.get('label');
+    }.property('path', 'label'),
 });
 
 App.Builder = App.NameLabelModel.extend({
@@ -205,5 +210,32 @@ App.Manifest = Ember.Controller.extend({
             repositories.filter(function (repository) { return repository.get('hasReportedCommits'); }));
 
         this.set('bugTrackers', store.all('bugTracker').sortBy('name'));
-    }
+    },
+    fetchRunsWithPlatformAndMetric: function (store, platformId, metricId)
+    {
+        return Ember.RSVP.all([
+            RunsData.fetchRuns(platformId, metricId),
+            this.fetch(store),
+        ]).then(function (values) {
+            var runs = values[0];
+
+            var platform = App.Manifest.platform(platformId);
+            var metric = App.Manifest.metric(metricId);
+
+            // FIXME: Include this information in JSON and process it in RunsData.fetchRuns
+            var unit = {'Combined': '', // Assume smaller is better for now.
+                'FrameRate': 'fps',
+                'Runs': 'runs/s',
+                'Time': 'ms',
+                'Malloc': 'bytes',
+                'JSHeap': 'bytes',
+                'Allocations': 'bytes',
+                'EndAllocations': 'bytes',
+                'MaxAllocations': 'bytes',
+                'MeanAllocations': 'bytes'}[metric.get('name')];
+            runs.unit = unit;
+
+            return {platform: platform, metric: metric, runs: runs};
+        });
+    },
 }).create();