https://bugs.webkit.org/show_bug.cgi?id=141786
Reviewed by Andreas Kling.
This patch makes following improvements to analysis task pages:
1. Making the main chart interactive. This change required the use of App.Pane as well as moving the code to
compute the data for the details pane from PaneController.
2. Moving the form to add a new test group to the top of test groups instead of the bottom of them.
3. Grouping the build requests in each test group by root sets instead of the order by which they were ran.
This change required the creation of App.TestGroupPane as well as its methods.
4. Show a box plot for each root set configuration as well as each build request. This change required
App.BoxPlotComponent.
5. Show revisions of each repository (e.g. WebKit) for each root set and build request.
* public/api/build-requests.php:
(main): Update per the rename of BuildRequestsFetcher::root_sets to BuildRequestsFetcher::root_sets_by_id.
* public/api/test-groups.php:
(main): Include root sets and roots in the response.
(format_test_group):
* public/include/build-requests-fetcher.php:
(BuildRequestsFetcher::root_sets_by_id): Renamed from root_sets.
(BuildRequestsFetcher::root_sets): Added.
(BuildRequestsFetcher::roots): Added.
(BuildRequestsFetcher::fetch_roots_for_set): Takes a boolean argument $resolve_ids. This flag is only set to
true in /api/build-requests/ (as done prior to this patch) to use repository names as identifiers since
tools/sync-with-buildbot.py can't convert repository names to their ids.
* public/v2/analysis.js:
(App.Root): Added.
(App.RootSet): Added.
(App.RootSet.revisionForRepository): Added.
(App.TestGroup.rootSets): Deleted the code to compute root set ids from build requests now that the JSON
response at /api/test-groups will include them.
(App.BuildRequest): Ditto. Also deleted 'configLetter' property, which has been moved to a proxy created by
_createConfigurationSummary.
(App.BuildRequest.statusLabel): Use 'Completed' as the human readable label for 'completed' status.
(App.BuildRequest.aggregateStatuses): Added. Generates a human readable status for a set of build requests.
* public/v2/app.css: Updated style rules for analysis task pages.
* public/v2/app.js:
(App.Pane): This class is now used in analysis task pages to make the main chart interactive.
(App.Pane._updateDetails): Moved from App.PaneController.
(App.PaneController._updateCanAnalyze): Updated the code per the move of selectedPoints.
(App.AnalysisTaskController): Added 'details'.
(App.AnalysisTaskController._taskUpdated):
(App.AnalysisTaskController.paneDomain):Renamed from _fetchedRuns.
(App.AnalysisTaskController.updateTestGroupPanes): Added. Creates App.TestGroupPane for each test group.
(App.AnalysisTaskController.actions.toggleShowRequestList): Added.
(App.TestGroupPane): Added.
(App.TestGroupPane._populate): Added. Group build requests by root sets and create a summary for each group.
(App.TestGroupPane._computeRepositoryList): Added. Returns a sorted list of repositories which is the union
of all repositories appearing in root sets and builds associated with A/B testing results.
(App.TestGroupPane._groupRequestsByConfigurations): Added. Groups build requests by root sets.
(App.TestGroupPane._createConfigurationSummary): Added. Creates a summary for a group of build requests that
use the same root set. We start by wrapping "raw" build requests in a proxy with formatted values,
build numbers, etc... obtained from the fetched chart data. The list of revisions shown in the group summary
is a union of revisions in the root set and the first build request in the group. We null-out revision info
for a build request if it is identical to the one in the summary. The range of values is expanded as needed
by the values in the group as well as 95% percentile confidence interval.
(App.BoxPlotComponent): Added. Controls a box plot shown for each test group summary and build request.
(App.BoxPlotComponent.didInsertElement): Added. Inserts a SVG element as well as two indicator rects to show
the mean and the confidence interval.
(App.BoxPlotComponent._updateBars): Added. Updates the dimensions of the indicator rects.
(App.BoxPlotComponent.valueChanged): Added. Computes the relative dimensions of the indicator rects and
calls _updateBars to update the rects.
* public/v2/chart-pane.css: Added some style rules to be used in the details pane in analysis task pages.
* public/v2/data.js:
(Measurement.prototype.formattedRevisions):
(Measurement.formatRevisionRange): Renamed from Measurement.prototype._formatRevisionRange so that it can be
called in _createConfigurationSummary.
* public/v2/index.html: Updated the templates for analysis task pages. Moved the form to create a new test
group above all test groups, and replaced the list of data points by "details" pane used in the charts page.
Also made the fetching of chartData no longer block showing of test groups.
* public/v2/interactive-chart.js:
(App.InteractiveChartComponent._updateDomain): Added an early exit to fix a newly revealed race condition.
(App.InteractiveChartComponent._domainChanged): Ditto.
(App.InteractiveChartComponent._updateSelectionToolbar): Made it respect 'zoomable' boolean property.
* public/v2/js/statistics.js:
(Statistics.min): Added.
(Statistics.max): Added.
* public/v2/manifest.js:
(App.Manifest.fetchRunsWithPlatformAndMetric): Added formatWithDeltaAndUnit to be used in _createConfigurationSummary.
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@180333
268f45cc-cd09-0410-ab3c-
d52691b4dbfc
+2015-02-18 Ryosuke Niwa <rniwa@webkit.org>
+
+ Analysis task pages are unusable
+ https://bugs.webkit.org/show_bug.cgi?id=141786
+
+ Reviewed by Andreas Kling.
+
+ This patch makes following improvements to analysis task pages:
+ 1. Making the main chart interactive. This change required the use of App.Pane as well as moving the code to
+ compute the data for the details pane from PaneController.
+ 2. Moving the form to add a new test group to the top of test groups instead of the bottom of them.
+ 3. Grouping the build requests in each test group by root sets instead of the order by which they were ran.
+ This change required the creation of App.TestGroupPane as well as its methods.
+ 4. Show a box plot for each root set configuration as well as each build request. This change required
+ App.BoxPlotComponent.
+ 5. Show revisions of each repository (e.g. WebKit) for each root set and build request.
+
+ * public/api/build-requests.php:
+ (main): Update per the rename of BuildRequestsFetcher::root_sets to BuildRequestsFetcher::root_sets_by_id.
+
+ * public/api/test-groups.php:
+ (main): Include root sets and roots in the response.
+ (format_test_group):
+
+ * public/include/build-requests-fetcher.php:
+ (BuildRequestsFetcher::root_sets_by_id): Renamed from root_sets.
+ (BuildRequestsFetcher::root_sets): Added.
+ (BuildRequestsFetcher::roots): Added.
+ (BuildRequestsFetcher::fetch_roots_for_set): Takes a boolean argument $resolve_ids. This flag is only set to
+ true in /api/build-requests/ (as done prior to this patch) to use repository names as identifiers since
+ tools/sync-with-buildbot.py can't convert repository names to their ids.
+
+ * public/v2/analysis.js:
+ (App.Root): Added.
+ (App.RootSet): Added.
+ (App.RootSet.revisionForRepository): Added.
+ (App.TestGroup.rootSets): Deleted the code to compute root set ids from build requests now that the JSON
+ response at /api/test-groups will include them.
+ (App.BuildRequest): Ditto. Also deleted 'configLetter' property, which has been moved to a proxy created by
+ _createConfigurationSummary.
+ (App.BuildRequest.statusLabel): Use 'Completed' as the human readable label for 'completed' status.
+ (App.BuildRequest.aggregateStatuses): Added. Generates a human readable status for a set of build requests.
+
+ * public/v2/app.css: Updated style rules for analysis task pages.
+
+ * public/v2/app.js:
+ (App.Pane): This class is now used in analysis task pages to make the main chart interactive.
+ (App.Pane._updateDetails): Moved from App.PaneController.
+
+ (App.PaneController._updateCanAnalyze): Updated the code per the move of selectedPoints.
+
+ (App.AnalysisTaskController): Added 'details'.
+ (App.AnalysisTaskController._taskUpdated):
+ (App.AnalysisTaskController.paneDomain):Renamed from _fetchedRuns.
+ (App.AnalysisTaskController.updateTestGroupPanes): Added. Creates App.TestGroupPane for each test group.
+ (App.AnalysisTaskController.actions.toggleShowRequestList): Added.
+
+ (App.TestGroupPane): Added.
+ (App.TestGroupPane._populate): Added. Group build requests by root sets and create a summary for each group.
+ (App.TestGroupPane._computeRepositoryList): Added. Returns a sorted list of repositories which is the union
+ of all repositories appearing in root sets and builds associated with A/B testing results.
+ (App.TestGroupPane._groupRequestsByConfigurations): Added. Groups build requests by root sets.
+ (App.TestGroupPane._createConfigurationSummary): Added. Creates a summary for a group of build requests that
+ use the same root set. We start by wrapping "raw" build requests in a proxy with formatted values,
+ build numbers, etc... obtained from the fetched chart data. The list of revisions shown in the group summary
+ is a union of revisions in the root set and the first build request in the group. We null-out revision info
+ for a build request if it is identical to the one in the summary. The range of values is expanded as needed
+ by the values in the group as well as 95% percentile confidence interval.
+
+ (App.BoxPlotComponent): Added. Controls a box plot shown for each test group summary and build request.
+ (App.BoxPlotComponent.didInsertElement): Added. Inserts a SVG element as well as two indicator rects to show
+ the mean and the confidence interval.
+ (App.BoxPlotComponent._updateBars): Added. Updates the dimensions of the indicator rects.
+ (App.BoxPlotComponent.valueChanged): Added. Computes the relative dimensions of the indicator rects and
+ calls _updateBars to update the rects.
+
+ * public/v2/chart-pane.css: Added some style rules to be used in the details pane in analysis task pages.
+
+ * public/v2/data.js:
+ (Measurement.prototype.formattedRevisions):
+ (Measurement.formatRevisionRange): Renamed from Measurement.prototype._formatRevisionRange so that it can be
+ called in _createConfigurationSummary.
+
+ * public/v2/index.html: Updated the templates for analysis task pages. Moved the form to create a new test
+ group above all test groups, and replaced the list of data points by "details" pane used in the charts page.
+ Also made the fetching of chartData no longer block showing of test groups.
+
+ * public/v2/interactive-chart.js:
+ (App.InteractiveChartComponent._updateDomain): Added an early exit to fix a newly revealed race condition.
+ (App.InteractiveChartComponent._domainChanged): Ditto.
+ (App.InteractiveChartComponent._updateSelectionToolbar): Made it respect 'zoomable' boolean property.
+
+ * public/v2/js/statistics.js:
+ (Statistics.min): Added.
+ (Statistics.max): Added.
+
+ * public/v2/manifest.js:
+ (App.Manifest.fetchRunsWithPlatformAndMetric): Added formatWithDeltaAndUnit to be used in _createConfigurationSummary.
+
2015-02-14 Ryosuke Niwa <rniwa@webkit.org>
Build URL on new perf dashboard doesn't resolve $builderName
exit_with_success(array(
'buildRequests' => $requests_fetcher->results_with_resolved_ids(),
- 'rootSets' => $requests_fetcher->root_sets(),
+ 'rootSets' => $requests_fetcher->root_sets_by_id(),
'updates' => $updates,
));
}
$group_by_id[$group['id']] = &$group;
$build_requests = $build_requests_fetcher->results();
- foreach ($build_requests as $request)
- array_push($group_by_id[$request['testGroup']]['buildRequests'], $request['id']);
+ foreach ($build_requests as $request) {
+ $request_group = &$group_by_id[$request['testGroup']];
+ array_push($request_group['buildRequests'], $request['id']);
+ array_push($request_group['rootSets'], $request['rootSet']);
+ }
- exit_with_success(array('testGroups' => $test_groups, 'buildRequests' => $build_requests));
+ exit_with_success(array('testGroups' => $test_groups,
+ 'buildRequests' => $build_requests,
+ 'rootSets' => $build_requests_fetcher->root_sets(),
+ 'roots' => $build_requests_fetcher->roots()));
}
function format_test_group($group_row) {
'author' => $group_row['testgroup_author'],
'createdAt' => strtotime($group_row['testgroup_created_at']) * 1000,
'buildRequests' => array(),
+ 'rootSets' => array(),
);
}
function __construct($db) {
$this->db = $db;
$this->rows = null;
+ $this->root_sets = array();
+ $this->roots = array();
$this->root_sets_by_id = array();
}
$root_set_id = $row['request_root_set'];
if (!array_key_exists($root_set_id, $this->root_sets_by_id))
- $this->root_sets_by_id[$root_set_id] = $this->fetch_roots_for_set($root_set_id);
+ $this->root_sets_by_id[$root_set_id] = $this->fetch_roots_for_set($root_set_id, $resolve_ids);
array_push($requests, array(
'id' => $row['request_id'],
return $requests;
}
- function root_sets() {
+ function root_sets_by_id() {
return $this->root_sets_by_id;
}
- private function fetch_roots_for_set($root_set_id) {
+ function root_sets() {
+ return $this->root_sets;
+ }
+
+ function roots() {
+ return $this->roots;
+ }
+
+ private function fetch_roots_for_set($root_set_id, $resolve_ids) {
$root_rows = $this->db->query_and_fetch_all('SELECT *
FROM roots, commits LEFT OUTER JOIN repositories ON commit_repository = repository_id
WHERE root_commit = commit_id AND root_set = $1', array($root_set_id));
$roots = array();
- foreach ($root_rows as $row)
- $roots[$row['repository_name']] = $row['commit_revision'];
+ $root_ids = array();
+ foreach ($root_rows as $row) {
+ $repository = $row['repository_id'];
+ $revision = $row['commit_revision'];
+ $root_id = $root_set_id . '-' . $repository;
+ array_push($root_ids, $root_id);
+ array_push($this->roots, array('id' => $root_id, 'repository' => $repository, 'revision' => $revision));
+ $roots[$resolve_ids ? $row['repository_name'] : $row['repository_id']] = $revision;
+ }
+ array_push($this->root_sets, array('id' => $root_set_id, 'roots' => $root_ids));
return $roots;
}
}
});
+App.Root = App.Model.extend({
+ repository: DS.belongsTo('repository'),
+ revision: DS.attr('string'),
+});
+
+App.RootSet = App.Model.extend({
+ roots: DS.hasMany('roots'),
+ revisionForRepository: function (repository)
+ {
+ var root = this.get('roots').findBy('repository', repository);
+ if (!root)
+ return null;
+ return root.get('revision');
+ }
+});
+
App.TestGroup = App.NameLabelModel.extend({
task: DS.belongsTo('analysisTask'),
author: DS.attr('string'),
createdAt: DS.attr('date'),
buildRequests: DS.hasMany('buildRequests'),
- rootSets: function ()
- {
- var rootSetIds = [];
- this.get('buildRequests').forEach(function (request) {
- var rootSet = request.get('rootSet');
- if (!rootSetIds.contains(rootSet))
- rootSetIds.push(rootSet);
- });
- return rootSetIds;
- }.property('buildRequests'),
+ rootSets: DS.hasMany('rootSets'),
_fetchChartData: function ()
{
var task = this.get('task');
{
return this.get('order') + 1;
}.property('order'),
- rootSet: DS.attr('number'),
- configLetter: function ()
- {
- var rootSets = this.get('testGroup').get('rootSets');
- var index = rootSets.indexOf(this.get('rootSet'));
- return String.fromCharCode('A'.charCodeAt(0) + index);
- }.property('testGroup', 'testGroup.rootSets'),
+ rootSet: DS.belongsTo('rootSet'),
status: DS.attr('string'),
statusLabel: function ()
{
case 'failed':
return 'Failed';
case 'completed':
- return 'Finished';
+ return 'Completed';
}
}.property('status'),
url: DS.attr('string'),
build: DS.attr('number'),
- _fetchMean: function ()
- {
- var testGroup = this.get('testGroup');
- if (!testGroup)
- return;
- var chartData = testGroup.get('chartData');
- if (!chartData)
- return;
-
- var point = chartData.current.findPointByBuild(this.get('build'));
- if (!point)
- return;
- this.set('mean', chartData.formatter(point.value) + (chartData.unit ? ' ' + chartData.unit : ''));
- this.set('buildNumber', point.measurement.buildNumber());
- }.observes('build', 'testGroup', 'testGroup.chartData').on('init'),
});
+
+App.BuildRequest.aggregateStatuses = function (requests)
+{
+ var completeCount = 0;
+ var failureCount = 0;
+ requests.forEach(function (request) {
+ switch (request.get('status')) {
+ case 'failed':
+ failureCount++;
+ break;
+ case 'completed':
+ completeCount++;
+ break;
+ }
+ });
+ if (completeCount == requests.length)
+ return 'Done';
+ if (failureCount == requests.length)
+ return 'All failed';
+ var status = completeCount + ' out of ' + requests.length + ' completed';
+ if (failureCount)
+ status += ', ' + failureCount + ' failed';
+ return status;
+}
border-top: solid 1px #ddd;
}
+.analysis-group .results .summary td {
+ vertical-align: top;
+}
+
+.analysis-group .results thead td {
+ text-align: center;
+}
+
+.analysis-group .results .config-letter,
+.analysis-group .results .summary {
+ cursor: pointer;
+}
+
+.analysis-group .results .request .config-letter {
+ border-color: transparent;
+}
+
+.analysis-group .results .hideRequests .request {
+ display: none;
+}
+
+.box-plot {
+ display: inline-block;
+ width: 100px;
+ height: 0.6rem;
+ border: solid 1px #ddd;
+ padding: 1px;
+ vertical-align: middle;
+}
+
+.box-plot .percentage {
+ fill: #ccc;
+}
+
+.box-plot .delta {
+ fill: #333;
+ opacity: 0.5;
+}
+
+.box-plot svg {
+ display: block;
+}
+
#analysis-task-title {
font-weight: normal;
font-size: 1.2rem;
border: 1px solid #bbb;
border-radius: 0.5rem;
box-shadow: rgba(0, 0, 0, 0.03) 1px 1px 0px 0px;
-
- padding: 0.5rem 1rem;
margin-bottom: 1.5rem;
}
-.analysis-group caption {
+.analysis-group > * {
+ margin: 0.5rem;
+}
+
+.analysis-group > h1 {
font-size: 1.1rem;
+ font-weight: normal;
text-align: left;
margin-bottom: 0.5rem;
+ border-bottom: 1px solid #bbb;
+ margin: 0;
+ padding: 0.2rem 0.5rem;
+}
+
+.analysis-group h1 > input {
+ font-size: 1rem;
+ min-width: 20rem;
+ margin: 0.2rem 0;
}
.analysis-bugs th {
metricId: null,
metric: null,
selectedItem: null,
+ selectedPoints: null,
+ hoveredOrSelectedItem: null,
showFullYAxis: false,
searchCommit: function (repository, keyword) {
var self = this;
if (JSON.stringify(config) != JSON.stringify(this.get(configName)))
this.set(configName, config);
},
+ _updateDetails: function ()
+ {
+ var selectedPoints = this.get('selectedPoints');
+ var currentPoint = this.get('hoveredOrSelectedItem');
+ if (!selectedPoints && !currentPoint) {
+ this.set('details', null);
+ return;
+ }
+
+ var currentMeasurement;
+ var previousPoint;
+ if (!selectedPoints)
+ previousPoint = currentPoint.series.previousPoint(currentPoint);
+ else {
+ currentPoint = selectedPoints[selectedPoints.length - 1];
+ previousPoint = selectedPoints[0];
+ }
+ var currentMeasurement = currentPoint.measurement;
+ var oldMeasurement = previousPoint ? previousPoint.measurement : null;
+
+ var formattedRevisions = currentMeasurement.formattedRevisions(oldMeasurement);
+ var revisions = App.Manifest.get('repositories')
+ .filter(function (repository) { return formattedRevisions[repository.get('id')]; })
+ .map(function (repository) {
+ var revision = Ember.Object.create(formattedRevisions[repository.get('id')]);
+ revision['url'] = revision.previousRevision
+ ? repository.urlForRevisionRange(revision.previousRevision, revision.currentRevision)
+ : repository.urlForRevision(revision.currentRevision);
+ revision['name'] = repository.get('name');
+ revision['repository'] = repository;
+ return revision;
+ });
+
+ var buildNumber = null;
+ var buildURL = null;
+ if (!selectedPoints) {
+ buildNumber = currentMeasurement.buildNumber();
+ var builder = App.Manifest.builder(currentMeasurement.builderId());
+ if (builder)
+ buildURL = builder.urlFromBuildNumber(buildNumber);
+ }
+
+ this.set('details', Ember.Object.create({
+ status: this.computeStatus(currentPoint, previousPoint),
+ buildNumber: buildNumber,
+ buildURL: buildURL,
+ buildTime: currentMeasurement.formattedBuildTime(),
+ revisions: revisions,
+ }));
+ }.observes('hoveredOrSelectedItem', 'selectedPoints'),
});
App.encodePrettifiedJSON = function (plain)
this.set('mainPlotDomain', newSelection || this.get('overviewDomain'));
this.set('overviewSelection', newSelection);
}.observes('parentController.sharedZoom').on('init'),
- _updateDetails: function ()
- {
- var selectedPoints = this.get('selectedPoints');
- var currentPoint = this.get('currentItem');
- if (!selectedPoints && !currentPoint) {
- this.set('details', null);
- return;
- }
-
- var currentMeasurement;
- var previousPoint;
- if (!selectedPoints)
- previousPoint = currentPoint.series.previousPoint(currentPoint);
- else {
- currentPoint = selectedPoints[selectedPoints.length - 1];
- previousPoint = selectedPoints[0];
- }
- var currentMeasurement = currentPoint.measurement;
- var oldMeasurement = previousPoint ? previousPoint.measurement : null;
-
- var formattedRevisions = currentMeasurement.formattedRevisions(oldMeasurement);
- var revisions = App.Manifest.get('repositories')
- .filter(function (repository) { return formattedRevisions[repository.get('id')]; })
- .map(function (repository) {
- var revision = Ember.Object.create(formattedRevisions[repository.get('id')]);
- revision['url'] = revision.previousRevision
- ? repository.urlForRevisionRange(revision.previousRevision, revision.currentRevision)
- : repository.urlForRevision(revision.currentRevision);
- revision['name'] = repository.get('name');
- revision['repository'] = repository;
- return revision;
- });
-
- var buildNumber = null;
- var buildURL = null;
- if (!selectedPoints) {
- buildNumber = currentMeasurement.buildNumber();
- var builder = App.Manifest.builder(currentMeasurement.builderId());
- if (builder)
- buildURL = builder.urlFromBuildNumber(buildNumber);
- }
-
- this.set('details', Ember.Object.create({
- status: this.get('model').computeStatus(currentPoint, previousPoint),
- buildNumber: buildNumber,
- buildURL: buildURL,
- buildTime: currentMeasurement.formattedBuildTime(),
- revisions: revisions,
- }));
- this._updateCanAnalyze();
- }.observes('currentItem', 'selectedPoints'),
_updateCanAnalyze: function ()
{
- var points = this.get('selectedPoints');
+ var points = this.get('model').get('selectedPoints');
this.set('cannotAnalyze', !this.get('newAnalysisTaskName') || !points || points.length < 2);
- }.observes('newAnalysisTaskName'),
+ }.observes('newAnalysisTaskName', 'model.selectedPoints'),
});
-
App.AnalysisRoute = Ember.Route.extend({
model: function () {
return this.store.findAll('analysisTask').then(function (tasks) {
label: Ember.computed.alias('model.name'),
platform: Ember.computed.alias('model.platform'),
metric: Ember.computed.alias('model.metric'),
- testGroups: Ember.computed.alias('model.testGroups'),
+ details: Ember.computed.alias('pane.details'),
testSets: [],
roots: [],
bugTrackers: [],
if (!model)
return;
- var platformId = model.get('platform').get('id');
- var metricId = model.get('metric').get('id');
App.Manifest.fetch(this.store).then(this._fetchedManifest.bind(this));
- App.Manifest.fetchRunsWithPlatformAndMetric(this.store, platformId, metricId).then(this._fetchedRuns.bind(this));
+ this.set('pane', App.Pane.create({
+ store: this.store,
+ platformId: model.get('platform').get('id'),
+ metricId: model.get('metric').get('id'),
+ }));
}.observes('model').on('init'),
_fetchedManifest: function ()
{
});
}));
},
- _fetchedRuns: function (result)
+ paneDomain: function ()
{
- var chartData = result.data;
+ var pane = this.get('pane');
+ if (!pane)
+ return;
+
+ var chartData = pane.get('chartData');
+ if (!chartData)
+ return null;
+
var currentTimeSeries = chartData.current;
if (!currentTimeSeries)
- return; // FIXME: Report an error.
+ return null; // FIXME: Report an error.
var start = currentTimeSeries.findPointByMeasurementId(this.get('model').get('startRun'));
var end = currentTimeSeries.findPointByMeasurementId(this.get('model').get('endRun'));
if (!start || !end)
- return; // FIXME: Report an error.
+ return null; // FIXME: Report an error.
var highlightedItems = {};
highlightedItems[start.measurement.id()] = true;
id: point.measurement.id(),
measurement: point.measurement,
label: 'Point ' + (index + 1),
- value: chartData.formatter(point.value) + (chartData.unit ? ' ' + chartData.unit : ''),
+ value: chartData.formatWithUnit(point.value),
};
});
var margin = (end.time - start.time) * 0.1;
- this.set('chartData', chartData);
- this.set('chartDomain', [start.time - margin, +end.time + margin]);
this.set('highlightedItems', highlightedItems);
this.set('analysisPoints', formatedPoints);
- },
+
+ return [start.time - margin, +end.time + margin];
+ }.property('pane.chartData', 'model', 'model'),
testSets: function ()
{
var analysisPoints = this.get('analysisPoints');
}));
});
}.observes('analysisPoints'),
+ updateTestGroupPanes: function ()
+ {
+ var model = this.get('model');
+ if (!model)
+ return;
+ var self = this;
+ model.get('testGroups').then(function (groups) {
+ self.set('testGroupPanes', groups.map(function (group) { return App.TestGroupPane.create({content: group}); }));
+ });
+ }.observes('model'),
actions: {
associateBug: function (bugTracker, bugNumber)
{
});
},
+ toggleShowRequestList: function (configuration)
+ {
+ configuration.toggleProperty('showRequestList');
+ }
},
});
+
+App.TestGroupPane = Ember.ObjectProxy.extend({
+ _populate: function ()
+ {
+ var buildRequests = this.get('buildRequests');
+ var chartData = this.get('chartData');
+ if (!buildRequests || !chartData)
+ return [];
+
+ var repositories = this._computeRepositoryList();
+ this.set('repositories', repositories);
+
+ var requestsByRooSet = this._groupRequestsByConfigurations(buildRequests);
+
+ var configurations = [];
+ var index = 0;
+ var range = {min: Infinity, max: -Infinity};
+ for (var rootSetId in requestsByRooSet) {
+ var configLetter = String.fromCharCode('A'.charCodeAt(0) + index++);
+ configurations.push(this._createConfigurationSummary(requestsByRooSet[rootSetId], configLetter, range));
+ }
+
+ var margin = 0.1 * (range.max - range.min);
+ range.max += margin;
+ range.min -= margin;
+
+ this.set('configurations', configurations);
+ }.observes('chartData', 'buildRequests'),
+ _computeRepositoryList: function ()
+ {
+ var specifiedRepositories = new Ember.Set();
+ (this.get('rootSets') || []).forEach(function (rootSet) {
+ (rootSet.get('roots') || []).forEach(function (root) {
+ specifiedRepositories.add(root.get('repository'));
+ });
+ });
+ var reportedRepositories = new Ember.Set();
+ var chartData = this.get('chartData');
+ (this.get('buildRequests') || []).forEach(function (request) {
+ var point = chartData.current.findPointByBuild(request.get('build'));
+ if (!point)
+ return;
+
+ var revisionByRepositoryId = point.measurement.formattedRevisions();
+ for (var repositoryId in revisionByRepositoryId) {
+ var repository = App.Manifest.repository(repositoryId);
+ if (!specifiedRepositories.contains(repository))
+ reportedRepositories.add(repository);
+ }
+ });
+ return specifiedRepositories.sortBy('name').concat(reportedRepositories.sortBy('name'));
+ },
+ _groupRequestsByConfigurations: function (requests, repositoryList)
+ {
+ var rootSetIdToRequests = {};
+ var testGroup = this;
+ requests.forEach(function (request) {
+ var rootSetId = request.get('rootSet').get('id');
+ if (!rootSetIdToRequests[rootSetId])
+ rootSetIdToRequests[rootSetId] = [];
+ rootSetIdToRequests[rootSetId].push(request);
+ });
+ return rootSetIdToRequests;
+ },
+ _createConfigurationSummary: function (buildRequests, configLetter, range)
+ {
+ var repositories = this.get('repositories');
+ var chartData = this.get('chartData');
+ var requests = buildRequests.map(function (originalRequest) {
+ var point = chartData.current.findPointByBuild(originalRequest.get('build'));
+ var revisionByRepositoryId = point ? point.measurement.formattedRevisions() : {};
+ return Ember.ObjectProxy.create({
+ content: originalRequest,
+ revisions: repositories.map(function (repository, index) {
+ return (revisionByRepositoryId[repository.get('id')] || {label:null}).label;
+ }),
+ value: point ? point.value : null,
+ valueRange: range,
+ formattedValue: point ? chartData.formatWithUnit(point.value) : null,
+ buildNumber: point ? point.measurement.buildNumber() : null,
+ });
+ });
+
+ var rootSet = requests ? requests[0].get('rootSet') : null;
+ var summaryRevisions = repositories.map(function (repository, index) {
+ var revision = rootSet ? rootSet.revisionForRepository(repository) : null;
+ if (!revision)
+ return requests[0].get('revisions')[index];
+ return Measurement.formatRevisionRange(revision).label;
+ });
+
+ requests.forEach(function (request) {
+ var revisions = request.get('revisions');
+ repositories.forEach(function (repository, index) {
+ if (revisions[index] == summaryRevisions[index])
+ revisions[index] = null;
+ });
+ });
+
+ var valuesInConfig = requests.mapBy('value').filter(function (value) { return typeof(value) === 'number' && !isNaN(value); });
+ var sum = Statistics.sum(valuesInConfig);
+ var ciDelta = Statistics.confidenceIntervalDelta(0.95, valuesInConfig.length, sum, Statistics.squareSum(valuesInConfig));
+ var mean = sum / valuesInConfig.length;
+
+ range.min = Math.min(range.min, Statistics.min(valuesInConfig));
+ range.max = Math.max(range.max, Statistics.max(valuesInConfig));
+ if (ciDelta && !isNaN(ciDelta)) {
+ range.min = Math.min(range.min, mean - ciDelta);
+ range.max = Math.max(range.max, mean + ciDelta);
+ }
+
+ var summary = Ember.Object.create({
+ isAverage: true,
+ configLetter: configLetter,
+ revisions: summaryRevisions,
+ formattedValue: isNaN(mean) ? null : chartData.formatWithDeltaAndUnit(mean, ciDelta),
+ value: mean,
+ confidenceIntervalDelta: ciDelta,
+ valueRange: range,
+ statusLabel: App.BuildRequest.aggregateStatuses(requests),
+ });
+
+ return Ember.Object.create({summary: summary, items: requests});
+ },
+});
+
+App.BoxPlotComponent = Ember.Component.extend({
+ classNames: ['box-plot'],
+ range: null,
+ value: null,
+ delta: null,
+ didInsertElement: function ()
+ {
+ var element = this.get('element');
+ var svg = d3.select(element).append('svg')
+ .attr('viewBox', '0 0 100 20')
+ .attr('preserveAspectRatio', 'none')
+ .style({width: '100%', height: '100%'});
+
+ this._percentageRect = svg
+ .append('rect')
+ .attr('x', 0)
+ .attr('y', 0)
+ .attr('width', 0)
+ .attr('height', 20)
+ .attr('class', 'percentage');
+
+ this._deltaRect = svg
+ .append('rect')
+ .attr('x', 0)
+ .attr('y', 5)
+ .attr('width', 0)
+ .attr('height', 10)
+ .attr('class', 'delta')
+ .attr('opacity', 0.5)
+ this._updateBars();
+ },
+ _updateBars: function ()
+ {
+ if (!this._percentageRect || typeof(this._percentage) !== 'number' || isNaN(this._percentage))
+ return;
+
+ this._percentageRect.attr('width', this._percentage);
+ if (typeof(this._delta) === 'number' && !isNaN(this._delta)) {
+ this._deltaRect.attr('x', this._percentage - this._delta);
+ this._deltaRect.attr('width', this._delta * 2);
+ }
+ },
+ valueChanged: function ()
+ {
+ var range = this.get('range');
+ var value = this.get('value');
+ if (!range || !value)
+ return;
+ var scalingFactor = 100 / (range.max - range.min);
+ var percentage = (value - range.min) * scalingFactor;
+ this._percentage = percentage;
+ this._delta = this.get('delta') * scalingFactor;
+ this._updateBars();
+ }.observes('value', 'range').on('init'),
+});
border-left: solid 1px #bbb;
}
+.analysis-chart-pane {
+ height: 15rem;
+}
+
.analysis-chart-pane .details {
overflow: scroll;
}
height: 13rem;
overflow: scroll;
}
+.analysis-chart-pane .details-table-container {
+ position: static;
+ height: 15rem;
+}
.chart-pane .details-table,
.chart-pane .commits-viewer {
for (var repositoryId in revisions) {
var currentRevision = revisions[repositoryId][0];
var previousRevision = previousRevisions ? previousRevisions[repositoryId][0] : null;
- var formatttedRevision = this._formatRevisionRange(previousRevision, currentRevision);
+ var formatttedRevision = Measurement.formatRevisionRange(currentRevision, previousRevision);
formattedRevisions[repositoryId] = formatttedRevision;
}
return formattedRevisions;
}
-Measurement.prototype._formatRevisionRange = function (previousRevision, currentRevision)
+Measurement.formatRevisionRange = function (currentRevision, previousRevision)
{
var revisionChanged = false;
if (previousRevision == currentRevision)
domain=mainPlotDomain
interactive=true
chartPointRadius=2
- currentItem=currentItem
+ currentItem=hoveredOrSelectedItem
currentTime=sharedTime
selectedItem=selectedItem
highlightedItems=highlightedItems
rangeRoute="analysisTask"
selection=timeRange
selectedPoints=selectedPoints
- markedPoints=markedPoints
showFullYAxis=showFullYAxis
+ zoomable=true
zoom="zoomed"}}
{{else}}
{{#if failure}}
selection=overviewSelection}}
{{/if}}
</div>
- {{#if details}}
- {{partial "chart-details"}}
- {{/if}}
+ {{partial "chart-details"}}
</div>
</div>
</script>
<script type="text/x-handlebars" data-template-name="chart-details">
+ {{#if details}}
<div class="details-table-container">
<table class="details-table">
<tbody class="bugs">
{{/each}}
</div>
</div>
+ {{/if}}
</script>
<script type="text/x-handlebars" data-template-name="components/commits-viewer">
<h3 id="analysis-task-testname">{{metric.fullName}} - {{platform.label}}</h3>
{{/if}}
- {{#if chartData}}
- <section class="analysis-chart-pane chart-pane">
+ {{#if pane}}
+ <section class="analysis-chart-pane chart-pane" tabindex="0">
<div class="svg-container">
{{interactive-chart
- chartData=chartData
- enableSelection=false
+ chartData=pane.chartData
+ ranges=pane.analyticRanges
+ domain=paneDomain
+ interactive=true
chartPointRadius=2
- domain=chartDomain
- highlightedItems=highlightedItems}}
+ currentItem=pane.hoveredOrSelectedItem
+ selectedPoints=pane.selectedPoints
+ selection=timeRange
+ highlightedItems=highlightedItems
+ rangeRoute="analysisTask"}}
</div>
<div class="details">
- <table class="analysis-bugs">
- <tbody>
- {{#each bugTrackers}}
- <tr>
- <th>{{label}}</th>
- <td>
- <form {{action "associateBug" this editedBugNumber on="submit"}}>
- {{input type=text value=editedBugNumber}}
- </form>
- </td>
- </tr>
- {{/each}}
- </tbody>
- </table>
- <table>
- <tbody>
- {{#each analysisPoints}}
- <tr><td>{{label}}</td><td>{{value}}</td></tr>
- {{/each}}
- </tbody>
- </table>
+ {{partial "chart-details"}}
</div>
</section>
- {{#each testGroups}}
- <section class="analysis-group">
- <table>
- <caption>{{name}}</caption>
- <thead>
- <tr>
- <td>Order</td>
- <td>Configuration</td>
- <td>Status</td>
- <td>Build</td>
- <td>{{../metric.fullName}}</td>
+ {{/if}}
+
+ {{partial "testGroupForm"}}
+
+ {{#each testGroupPanes}}
+ {{partial "testGroup"}}
+ {{/each}}
+ </script>
+
+ <script type="text/x-handlebars" data-template-name="testGroup">
+ <section class="analysis-group">
+ <h1>{{name}}</h1>
+ <table class="results">
+ <thead>
+ <tr>
+ <td colspan="2">Configuration</td>
+ {{#each repositories}}
+ <td>{{name}}</td>
+ {{/each}}
+ <td>Results</td>
+ <td>Status</td>
+ </tr>
+ </thead>
+ {{#each configurations}}
+ <tbody {{bind-attr class="showRequestList::hideRequests"}}>
+ <tr class="summary" {{action toggleShowRequestList this}}>
+ <td class="config-letter" colspan="2">{{summary.configLetter}}</td>
+ {{#with summary}}
+ {{partial "testGroupRow"}}
+ {{/with}}
+ </tr>
+ {{#each items}}
+ <tr class="request">
+ {{#with ../this}}
+ <td class="config-letter" {{action toggleShowRequestList this}}></td>
+ {{/with}}
+ <td>Run {{orderLabel}}</td>
+ {{partial "testGroupRow"}}
</tr>
- </thead>
- <tbody>
- {{#each buildRequests}}
- <tr>
- <td>{{orderLabel}}</td>
- <td>{{configLetter}}</td>
- <td><a {{bind-attr href=url}}>{{statusLabel}}</a></td>
- <td>{{buildNumber}}</td>
- <td>{{mean}}</td>
- </tr>
- {{/each}}
- </tbody>
- </table>
- </section>
- {{/each}}
+ {{/each}}
+ </tbody>
+ {{/each}}
+ </table>
+ </section>
+ </script>
+
+ <script type="text/x-handlebars" data-template-name="testGroupRow">
+ {{#each revisions}}
+ <td>{{this}}</td>
+ {{/each}}
+ <td>
+ {{#if value}}
+ {{box-plot range=valueRange value=value delta=confidenceIntervalDelta}}
+ {{/if}}
+ {{formattedValue}}
+ </td>
+ <td>
+ {{#if buildNumber}}
+ {{statusLabel}} / <a {{bind-attr href=url}}>Build {{buildNumber}}</a>
+ {{else}}
+ <a {{bind-attr href=url}}>{{statusLabel}}</a>
+ {{/if}}
+ </td>
+ </script>
- {{#if roots}}
- <form method="POST" {{action "createTestGroup" newTestGroupName repetitionCount on="submit"}} class="analysis-group">
+ <script type="text/x-handlebars" data-template-name="testGroupForm">
+ {{#if roots}}
+ <form method="POST" {{action "createTestGroup" newTestGroupName repetitionCount on="submit"}}>
+ <section class="analysis-group">
+ <h1>{{input name="name" value=newTestGroupName placeholder="Test group name" required=true type="text"}}</h1>
<table>
- <caption>{{input name="name" value=newTestGroupName placeholder="Test group name" required=true type="text"}}</caption>
<thead>
<tr>
<th>Root</th>
</table>
<button type="submit">Start A/B testing</button>
- </form>
- {{/if}}
- {{/if}}
+ </section>
+ </form>
+ {{/if}}
</script>
</head>
_updateDomain: function ()
{
var xDomain = this.get('domain');
+ if (!xDomain || !this._currentTimeSeriesData)
+ return null;
var intrinsicXDomain = this._computeXAxisDomain(this._currentTimeSeriesData);
if (!xDomain)
xDomain = intrinsicXDomain;
{
var selection = this._currentSelection() || this.get('sharedSelection');
var newXDomain = this._updateDomain();
+ if (!newXDomain)
+ return;
if (selection && newXDomain && selection[0] <= newXDomain[0] && newXDomain[1] <= selection[1])
selection = null; // Otherwise the user has no way of clearing the selection.
},
_updateSelectionToolbar: function ()
{
- if (!this.get('interactive'))
+ if (!this.get('zoomable'))
return;
var selection = this._currentSelection();
var Statistics = new (function () {
+ this.min = function (values) {
+ return Math.min.apply(Math, values);
+ }
+
+ this.max = function (values) {
+ return Math.max.apply(Math, values);
+ }
+
this.sum = function (values) {
return values.length ? values.reduce(function (a, b) { return a + b; }) : 0;
}
var smallerIsBetter = unit != 'fps' && unit != '/s'; // Assume smaller is better for unit-less metrics.
var useSI = unit == 'bytes';
+ var unitSuffix = unit ? ' ' + unit : '';
+ var deltaFormatterWithoutSign = useSI ? d3.format('.2s') : d3.format('.2g');
return {
platform: platform,
metric: metric,
baseline: runs.baseline ? runs.baseline.timeSeriesByCommitTime() : null,
target: runs.target ? runs.target.timeSeriesByCommitTime() : null,
unit: unit,
+ formatWithUnit: function (value) { return this.formatter(value) + unitSuffix; },
+ formatWithDeltaAndUnit: function (value, delta)
+ {
+ return this.formatter(value) + (delta && !isNaN(delta) ? ' \u00b1 ' + deltaFormatterWithoutSign(delta) : '') + unitSuffix;
+ },
formatter: useSI ? d3.format('.4s') : d3.format('.4g'),
deltaFormatter: useSI ? d3.format('+.2s') : d3.format('+.2g'),
smallerIsBetter: smallerIsBetter,
git://git.webkit.org / WebKit-https.git / commitdiff