perf-o-matic should have a way to hide some platforms and tests
authorrniwa@webkit.org <rniwa@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 2 Apr 2012 01:46:41 +0000 (01:46 +0000)
committerrniwa@webkit.org <rniwa@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 2 Apr 2012 01:46:41 +0000 (01:46 +0000)
https://bugs.webkit.org/show_bug.cgi?id=82842

Reviewed by Hajime Morita.

* Websites/webkit-perf.appspot.com/admin.html:
* Websites/webkit-perf.appspot.com/admin_handlers.py:
(AdminDashboardHandler.get_branches): Change the json format to allow platforms and tests to have
"hidden" boolean states.
(AdminDashboardHandler.get_platforms): Ditto.
(AdminDashboardHandler.get_builders): Just a cleanup. There is no clean for it to have a limit.
(AdminDashboardHandler.get_tests): Change the json format to add "hidden" boolean states.
(ChangeVisibilityHandler): Added.
(ChangeVisibilityHandler.post): Added. Changes the hidden-state (visibility) of a platform and a test.
* Websites/webkit-perf.appspot.com/app.yaml: Make sure everything under /admin/ requires admin privilege.
* Websites/webkit-perf.appspot.com/create_handler.py:
(CreateHandler.post): Don't emit LF after 'OK'.
* Websites/webkit-perf.appspot.com/css/admin.css: Added a bunch of rules for hide/show button.
* Websites/webkit-perf.appspot.com/js/admin.js:
(submitXHR): Extracted.
(createKeyNameReloader): Added hide/show button on each item and the corresponding ajax request.
* Websites/webkit-perf.appspot.com/json_generators.py:
(DashboardJSONGenerator.__init__): Skip hidden tests and platforms.
(ManifestJSONGenerator.__init__): Ditto.
* Websites/webkit-perf.appspot.com/json_generators_unittest.py: Added tests to ensure perf-o-matic
doesn't include hidden tests and platforms in dashboard and manifest json responses.
(DashboardJSONGeneratorTest.test_value_with_hidden_platform_and_tesst):
(ManifestJSONGeneratorTest.test_value_two_tests):
(ManifestJSONGeneratorTest.test_value_with_hidden_platform_and_test):
* Websites/webkit-perf.appspot.com/main.py:
* Websites/webkit-perf.appspot.com/models.py:
(Platform): Added the "hidden" property.
(Test): Ditto. Also removed the comment about this class only exists for efficiency purposes since that's
no longer true.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@112822 268f45cc-cd09-0410-ab3c-d52691b4dbfc

ChangeLog
Websites/webkit-perf.appspot.com/admin.html
Websites/webkit-perf.appspot.com/admin_handlers.py
Websites/webkit-perf.appspot.com/app.yaml
Websites/webkit-perf.appspot.com/create_handler.py
Websites/webkit-perf.appspot.com/css/admin.css
Websites/webkit-perf.appspot.com/js/admin.js
Websites/webkit-perf.appspot.com/json_generators.py
Websites/webkit-perf.appspot.com/json_generators_unittest.py
Websites/webkit-perf.appspot.com/main.py
Websites/webkit-perf.appspot.com/models.py

index cd1307a..a7573f6 100644 (file)
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,40 @@
+2012-04-01  Ryosuke Niwa  <rniwa@webkit.org>
+
+        perf-o-matic should have a way to hide some platforms and tests
+        https://bugs.webkit.org/show_bug.cgi?id=82842
+
+        Reviewed by Hajime Morita.
+
+        * Websites/webkit-perf.appspot.com/admin.html:
+        * Websites/webkit-perf.appspot.com/admin_handlers.py:
+        (AdminDashboardHandler.get_branches): Change the json format to allow platforms and tests to have
+        "hidden" boolean states.
+        (AdminDashboardHandler.get_platforms): Ditto.
+        (AdminDashboardHandler.get_builders): Just a cleanup. There is no clean for it to have a limit.
+        (AdminDashboardHandler.get_tests): Change the json format to add "hidden" boolean states.
+        (ChangeVisibilityHandler): Added.
+        (ChangeVisibilityHandler.post): Added. Changes the hidden-state (visibility) of a platform and a test.
+        * Websites/webkit-perf.appspot.com/app.yaml: Make sure everything under /admin/ requires admin privilege.
+        * Websites/webkit-perf.appspot.com/create_handler.py:
+        (CreateHandler.post): Don't emit LF after 'OK'.
+        * Websites/webkit-perf.appspot.com/css/admin.css: Added a bunch of rules for hide/show button.
+        * Websites/webkit-perf.appspot.com/js/admin.js:
+        (submitXHR): Extracted.
+        (createKeyNameReloader): Added hide/show button on each item and the corresponding ajax request.
+        * Websites/webkit-perf.appspot.com/json_generators.py:
+        (DashboardJSONGenerator.__init__): Skip hidden tests and platforms.
+        (ManifestJSONGenerator.__init__): Ditto.
+        * Websites/webkit-perf.appspot.com/json_generators_unittest.py: Added tests to ensure perf-o-matic
+        doesn't include hidden tests and platforms in dashboard and manifest json responses.
+        (DashboardJSONGeneratorTest.test_value_with_hidden_platform_and_tesst):
+        (ManifestJSONGeneratorTest.test_value_two_tests):
+        (ManifestJSONGeneratorTest.test_value_with_hidden_platform_and_test):
+        * Websites/webkit-perf.appspot.com/main.py:
+        * Websites/webkit-perf.appspot.com/models.py:
+        (Platform): Added the "hidden" property.
+        (Test): Ditto. Also removed the comment about this class only exists for efficiency purposes since that's
+        no longer true.
+
 2012-04-01  Gyuyoung Kim  <gyuyoung.kim@samsung.com>
 
         Support the Network Information API
index 7e93242..4d90ea3 100644 (file)
@@ -33,7 +33,7 @@
 </dl>
 
 <!-- FIXME: Merge /admin/report-logs here -->
-<a href="/admin/report-logs">Manage report logs</a>
+<p><a href="/admin/report-logs">Manage report logs</a></p>
 </section>
 
 <section id="branches">
index 8e7ea32..38740b6 100644 (file)
@@ -67,24 +67,58 @@ class AdminDashboardHandler(webapp2.RequestHandler):
     def get_branches(self):
         self.response.headers['Content-Type'] = 'application/json'
         result = {}
-        for branch in Branch.all().fetch(limit=100):
-            result[branch.key().name()] = branch.name
+        for branch in Branch.all():
+            result[branch.key().name()] = {'name': branch.name}
         self.response.out.write(json.dumps(result))
 
     def get_platforms(self):
         self.response.headers['Content-Type'] = 'application/json'
         result = {}
-        for platform in Platform.all().fetch(limit=100):
-            result[platform.key().name()] = platform.name
+        for platform in Platform.all():
+            result[platform.key().name()] = {'name': platform.name, 'hidden': platform.hidden}
         self.response.out.write(json.dumps(result))
 
     def get_builders(self):
         self.response.headers['Content-Type'] = 'application/json'
-        self.response.out.write(json.dumps([builder.name for builder in Builder.all().fetch(limit=100)]))
+        self.response.out.write(json.dumps([builder.name for builder in Builder.all()]))
 
     def get_tests(self):
         self.response.headers['Content-Type'] = 'application/json'
-        self.response.out.write(json.dumps([test.name for test in Test.all().fetch(limit=200)]))
+        result = {}
+        for test in Test.all():
+            result[test.key().name()] = {'name': test.name, 'hidden': test.hidden}
+        self.response.out.write(json.dumps(result))
+
+
+class ChangeVisibilityHandler(webapp2.RequestHandler):
+    def post(self):
+        self.response.headers['Content-Type'] = 'text/plain; charset=utf-8'
+
+        try:
+            payload = json.loads(self.request.body)
+            hide = payload['hide']
+        except:
+            self.response.out.write("Failed to parse the payload: %s" % self.request.body)
+            return
+
+        if 'platform' in payload:
+            model = Platform.get_by_key_name(payload['platform'])
+        elif 'test' in payload:
+            model = Test.get_by_key_name(payload['test'])
+        else:
+            self.response.out.write('Not supported')
+            return
+
+        if not model:
+            self.response.out.write('Could not find the model')
+            return
+
+        model.hidden = hide
+        model.put()
+        schedule_dashboard_update()
+        schedule_manifest_update()
+
+        self.response.out.write('OK')
 
 
 class MergeTestsHandler(webapp2.RequestHandler):
index 9287f09..d6c727f 100644 (file)
@@ -1,6 +1,6 @@
 application: webkit-perf
 version: 17
-runtime: python27
+runtime: python
 api_version: 1
 threadsafe: false
 
@@ -30,7 +30,7 @@ handlers:
   script: main.py
   secure: always
 
-- url: /admin/
+- url: /admin/.*
   script: main.py
   secure: always
   login: admin
index 5adafb7..5eb40a1 100644 (file)
@@ -64,7 +64,7 @@ class CreateHandler(webapp2.RequestHandler):
 
         # No need to clear manifest or runs since they only contain ones with test results
         schedule_dashboard_update()
-        self.response.out.write(error + '\n' if error else 'OK\n')
+        self.response.out.write(error if error else 'OK')
 
     def _create_builder(self, name, password):
         if not name or not password:
index f2e601a..2005899 100644 (file)
@@ -31,12 +31,60 @@ h2 {
     border-bottom: solid 1px lightgrey;
 }
 
-dl, section > ul > li {
+dl {
     padding: 0 10px;
 }
 
+section > ul {
+}
+
+section > ul > li {
+    width: 100%;
+    display: table;
+}
+
+section > p {
+    border-top: solid 1px lightgrey;
+    margin: 0px;
+    padding: 0 10px;
+    text-align: center;
+}
+
+section > ul > li > h3 {
+    font-size: 1em;
+    font-weight: normal;
+    display: table-cell;
+    padding: 10px;
+}
+
+section .hidden {
+    color: #ccc;
+}
+
+section .hide {
+    display: table-cell;
+    padding: 0 10px;
+    width: 16px;
+    height: 16px;
+    background-image: url('visible.png');
+    background-repeat: no-repeat;
+    background-position: center center;
+}
+
+section .hide:hover {
+    background-image: url('visible-hover.png');
+}
+
+section .hidden .hide {
+    background-image: url('hidden.png');
+}
+
+section .hidden .hide:hover {
+    background-image: url('hidden-hover.png');
+}
+
 dt, dd {
-    float: left;
+    display: table-cell;
 }
 
 dt {
index 03c769d..0d37b38 100644 (file)
@@ -1,3 +1,19 @@
+function submitXHR(method, action, payload, callback) {
+    var xhr = new XMLHttpRequest;
+    xhr.onreadystatechange = function () {
+        if (xhr.readyState != 4)
+            return;
+        if (xhr.status != 200)
+            error('HTTP status: ' + xhr.status);
+        else if (xhr.responseText != 'OK')
+            error(xhr.responseText);
+        if (callback)
+            callback()
+    }
+    xhr.open(method, action, true);
+    xhr.send(payload);
+}
+
 function removeNonFormListItems(list) {
     list.children().each(function () {
         if ($.inArray('form', this.classList))
@@ -5,13 +21,32 @@ function removeNonFormListItems(list) {
     });
 }
 
-function createKeyNameReloader(name) {
+function createKeyNameReloader(name, visibilityAction, callback) {
     return function () {
         $.getJSON(name, function (platforms) {
             var list = $('#' + name + ' ul');
             removeNonFormListItems(list);
-            $.each(platforms, function (key, name) {
-                list.append('<li>' + key + ' : ' + name + '</li>');
+            $.each(platforms, function (key, values) {
+                var label = key == values['name'] ? key : key + ' : ' + values['name'];
+                list.append('<li><h3 id="' + key + '">' + label + '</h3></li>');
+                var item = list[0].lastChild;
+
+                if (values['hidden'])
+                    item.className = 'hidden';
+
+                if (visibilityAction) {
+                    $(item).append(' <a class="hide"></a>');
+                    $(item.lastChild).click(function () {
+                        var json = {'hide': !this.parentNode.classList.contains('hidden')}
+                        json[visibilityAction] = this.parentNode.firstChild.id;
+
+                        submitXHR('POST', 'change-visibility', JSON.stringify(json));
+                        list.find('form').trigger('reload');
+                    });
+                }
+
+                if (callback)
+                    callback.call(item, values['hidden']);
             });
             list.append($('#' + name + ' ul .form'));
         });
@@ -19,7 +54,7 @@ function createKeyNameReloader(name) {
 }
 
 $('#branches form').bind('reload', createKeyNameReloader('branches'));
-$('#platforms form').bind('reload', createKeyNameReloader('platforms'));
+$('#platforms form').bind('reload', createKeyNameReloader('platforms', 'platform'));
 
 $('#builders form').bind('reload', function () {
     $.getJSON('builders', function (builders) {
@@ -27,26 +62,20 @@ $('#builders form').bind('reload', function () {
         removeNonFormListItems(list);
         builders = builders.sort();
         for (var i = 0; i < builders.length; i++)
-            list.append('<li><a href="http://build.webkit.org/builders/' + builders[i] + '">' + builders[i] + '</a></li>');
+            list.append('<li><h3><a href="http://build.webkit.org/builders/' + builders[i] + '">' + builders[i] + '</a></h3></li>');
         list.append($('#builders ul .form'));
     });
 });
 
-$('#tests form').bind('reload', function () {
-    $.getJSON('tests', function (tests) {
-        var list = $('#tests ul');
-        removeNonFormListItems(list);
-        var select = $('#tests select');
-        select.children().remove();
-
-        tests = tests.sort();
-        for (var i = 0; i < tests.length; i++) {
-            list.append('<li>' + tests[i] + '</li>'); // FIXME: Add a link to trac page.
-            select.append('<option value="' + tests[i] + '">' + tests[i] + '</option>');
-        }
+var testReloader = createKeyNameReloader('tests', 'test', function (hidden) {
+    var testName = this.firstChild.id;
+    $('#tests select').append('<option value="' + testName + '">' + testName + '</option>');
 
-        list.append($('#tests ul .form'));
-    });
+});
+$('#tests form').bind('reload', function () {
+    var select = $('#tests select');
+    select.children().remove();
+    testReloader();
 });
 
 $.ajaxSetup({
@@ -69,19 +98,9 @@ $('form').bind('submit', function (event) {
         payload = JSON.stringify(contents);
     }
 
-    var xhr = new XMLHttpRequest;
-    xhr.onreadystatechange = function () {
-        if (xhr.readyState != 4)
-            return;
-        if (xhr.status != 200)
-            error('HTTP status: ' + xhr.status);
-        else if (xhr.responseText != 'OK\n')
-            error(xhr.responseText);
-    }
-    xhr.open(this.method, this.action, true);
-    xhr.send(payload);
-
-    $(this).trigger('reload');
+    submitXHR(this.method, this.action, payload, function () {
+        $(this).trigger('reload');
+    })
 });
 
 $('#manual-submission textarea').val(JSON.stringify({
index 84dc1bb..cea5080 100644 (file)
@@ -62,10 +62,12 @@ class DashboardJSONGenerator(JSONGeneratorBase):
         }
 
         for platform in Platform.all():
-            self._dashboard['platformToId'][platform.name] = platform.id
+            if not platform.hidden:
+                self._dashboard['platformToId'][platform.name] = platform.id
 
         for test in Test.all():
-            self._dashboard['testToId'][test.name] = test.id
+            if not test.hidden:
+                self._dashboard['testToId'][test.name] = test.id
 
     def value(self):
         return self._dashboard
@@ -78,6 +80,9 @@ class ManifestJSONGenerator(JSONGeneratorBase):
         platform_id_map = {}
         branch_id_map = {}
         for test in Test.all():
+            if test.hidden:
+                continue
+
             branch_ids = [Branch.get(branch_key).id for branch_key in test.branches]
             platform_ids = [Platform.get(platform_key).id for platform_key in test.platforms]
             self._test_map[test.id] = {
@@ -100,6 +105,14 @@ class ManifestJSONGenerator(JSONGeneratorBase):
         for platform in Platform.all():
             if platform.id not in platform_id_map:
                 continue
+
+            if platform.hidden:
+                for test_id in platform_id_map[platform.id]['tests']:
+                    self._test_map[test_id]['platformIds'].remove(platform.id)
+                for branch_id in platform_id_map[platform.id]['branches']:
+                    branch_id_map[branch_id]['platforms'].remove(platform.id)
+                continue
+
             self._platform_map[platform.id] = {
                 'name': platform.name,
                 'testIds': list(set(platform_id_map[platform.id]['tests'])),
index adef9f6..e54df05 100644 (file)
@@ -115,6 +115,25 @@ class DashboardJSONGeneratorTest(DataStoreTestsBase):
             'testToId': {'some-test': Test.get_by_key_name('some-test').id},
         })
 
+    def test_value_with_hidden_platform_and_tesst(self):
+        webkit_trunk = Branch.create_if_possible('webkit-trunk', 'WebKit trunk')
+        some_platform = Platform.create_if_possible('some-platform', 'Some Platform')
+        hidden_platform = Platform.create_if_possible('hidden-platform', 'Hidden Platform')
+        hidden_platform.hidden = True
+        hidden_platform.put()
+        Test.update_or_insert('some-test', webkit_trunk, some_platform)
+        Test.update_or_insert('some-test', webkit_trunk, hidden_platform)
+        Test.update_or_insert('other-test', webkit_trunk, some_platform)
+        Test.update_or_insert('other-test', webkit_trunk, hidden_platform)
+        Test.update_or_insert('hidden-test', webkit_trunk, some_platform)
+        Test.update_or_insert('hidden-test', webkit_trunk, hidden_platform)
+        hidden_test = Test.get_by_key_name('hidden-test')
+        hidden_test.hidden = True
+        hidden_test.put()
+        self.assertEqual(DashboardJSONGenerator().value()['platformToId'], {'Some Platform': some_platform.id})
+        self.assertEqual(DashboardJSONGenerator().value()['testToId'],
+            {'some-test': Test.get_by_key_name('some-test').id, 'other-test': Test.get_by_key_name('other-test').id})
+
 
 class ManifestJSONGeneratorTest(DataStoreTestsBase):
     def test_value_no_branch(self):
@@ -176,7 +195,7 @@ class ManifestJSONGeneratorTest(DataStoreTestsBase):
         value = ManifestJSONGenerator().value()
         expected_test_ids = [some_test.id, other_test.id]
         self.assertEqualUnorderedList(value.keys(), ['branchMap', 'platformMap', 'testMap'])
-        self.assertEqualUnorderedList(value['branchMap'],
+        self.assertEqual(value['branchMap'],
             {some_branch.id: {'name': some_branch.name, 'testIds': expected_test_ids, 'platformIds': [some_platform.id]}})
         self.assertEqual(value['platformMap'],
             {some_platform.id: {'name': some_platform.name, 'branchIds': [some_branch.id], 'testIds': expected_test_ids}})
@@ -184,6 +203,31 @@ class ManifestJSONGeneratorTest(DataStoreTestsBase):
             {some_test.id: {'name': some_test.name, 'branchIds': [some_branch.id], 'platformIds': [some_platform.id]},
             other_test.id: {'name': other_test.name, 'branchIds': [some_branch.id], 'platformIds': [some_platform.id]}})
 
+    def test_value_with_hidden_platform_and_test(self):
+        some_branch = Branch.create_if_possible('some-branch', 'Some Branch')
+        some_platform = Platform.create_if_possible('some-platform', 'Some Platform')
+        hidden_platform = Platform.create_if_possible('hidden-platform', 'Hidden Platform')
+        hidden_platform.hidden = True
+        hidden_platform.put()
+
+        Test.update_or_insert('some-test', some_branch, some_platform)
+        some_test = Test.update_or_insert('some-test', some_branch, hidden_platform)
+
+        Test.update_or_insert('hidden-test', some_branch, some_platform)
+        hidden_test = Test.update_or_insert('hidden-test', some_branch, hidden_platform)
+        hidden_test.hidden = True
+        hidden_test.put()
+
+        value = ManifestJSONGenerator().value()
+        expected_test_ids = []
+        self.assertEqualUnorderedList(value.keys(), ['branchMap', 'platformMap', 'testMap'])
+        self.assertEqual(value['branchMap'],
+            {some_branch.id: {'name': some_branch.name, 'testIds': [some_test.id], 'platformIds': [some_platform.id]}})
+        self.assertEqual(value['platformMap'],
+            {some_platform.id: {'name': some_platform.name, 'branchIds': [some_branch.id], 'testIds': [some_test.id]}})
+        self.assertEqual(value['testMap'],
+            {some_test.id: {'name': some_test.name, 'branchIds': [some_branch.id], 'platformIds': [some_platform.id]}})
+
 
 if __name__ == '__main__':
     unittest.main()
index 2553663..889d7b8 100644 (file)
@@ -21,6 +21,7 @@ from google.appengine.ext.webapp import util
 import json
 
 from admin_handlers import AdminDashboardHandler
+from admin_handlers import ChangeVisibilityHandler
 from admin_handlers import IsAdminHandler
 from admin_handlers import MergeTestsHandler
 from controller import CachedDashboardHandler
@@ -42,6 +43,7 @@ routes = [
     (r'/admin/merge-tests(?:/(.*))?', MergeTestsHandler),
     ('/admin/report-logs/?', ReportLogsHandler),
     ('/admin/create/(.*)', CreateHandler),
+    ('/admin/change-visibility/?', ChangeVisibilityHandler),
     (r'/admin/([A-Za-z\-]*)', AdminDashboardHandler),
 
     ('/api/user/is-admin', IsAdminHandler),
index df5e19f..9ad5d1e 100644 (file)
@@ -95,6 +95,7 @@ class Branch(db.Model):
 class Platform(db.Model):
     id = db.IntegerProperty(required=True)
     name = db.StringProperty(required=True)
+    hidden = db.BooleanProperty()
 
     @staticmethod
     def create_if_possible(key, name):
@@ -140,7 +141,6 @@ class Build(db.Model):
             revision=log.webkit_revision(), chromiumRevision=log.chromium_revision())
 
 
-# Used to generate TestMap in the manifest efficiently
 class Test(db.Model):
     id = db.IntegerProperty(required=True)
     name = db.StringProperty(required=True)
@@ -148,6 +148,7 @@ class Test(db.Model):
     # one platform but only on some branch and vice versa.
     branches = db.ListProperty(db.Key)
     platforms = db.ListProperty(db.Key)
+    hidden = db.BooleanProperty()
 
     @staticmethod
     def update_or_insert(test_name, branch, platform):