Add an option to make the graphics benchmark runs a specific test with fixed complexity
authorcommit-queue@webkit.org <commit-queue@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 2 Nov 2015 02:31:02 +0000 (02:31 +0000)
committercommit-queue@webkit.org <commit-queue@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 2 Nov 2015 02:31:02 +0000 (02:31 +0000)
https://bugs.webkit.org/show_bug.cgi?id=150529

Patch by Said Abou-Hallawa <sabouhallawa@apple,com> on 2015-11-01
Reviewed by Darin Adler.

Beside each test in the suites tree, we are going to show the complexity
arithmetic mean of the of the last run in an edit control. Based on a
new option these edit controls will all be visible or hidden. If they are
visible their values can be changed. The benchmark runner if it run in
the non-adaptive mode will set the complexity of the test to the passed
value and will not change it ever. The animator will animate the test and
frame rate will also be measured.

* Animometer/runner/animometer.html: Add a new option for the non-adaptive mode.

* Animometer/runner/resources/animometer.css:
(section#home input[type="number"]): Define the width of all the edit control in the <home> section.
(section#home > suites input[type="number"]): The edit controls in the <suites> box will be right aligned and hidden by default.
(section#home > suites input[type="number"].selected): When the class "selected" is added, the edit controls will be visible.
(section#home > options > label > input[type="number"]): Deleted.

* Animometer/runner/resources/animometer.js:
(window.benchmarkRunnerClient.didFinishLastIteration): Update the local storage with the results of each test.
(window.optionsManager._adaptiveTestElement): Returns the checkbox for setting the adaptive test option.
(window.suitesManager._editElement):  Returns the edit element associated with each test element in the suites tree.
(window.suitesManager._editsElements): Returns a list of all the elements in the <suites> box.
(window.suitesManager._localStorageNameForTest): Change this function to take strings.
(window.suitesManager._createTestElement): Adds an edit control beside each test.
(window.suitesManager.updateEditsElementsState): Adds/Removes the 'selected' class to/from all the tests edit elements.
(window.suitesManager.updateUIFromLocalStorage): Reads the edit control value from the local storage.
(window.suitesManager.updateLocalStorageFromUI): Saves the edit control value to the local storage.
(window.suitesManager.updateLocalStorageFromJSON): Saves the last run results to the local storage.
(window.benchmarkController.initialize): Shows/Hides the test edit controls based on the adaptive test option.
(window.benchmarkController.onChangeAdaptiveTestCheckbox): An onchange event handler for the adaptive test checkbox.

* Animometer/tests/resources/main.js:
(Benchmark.prototype.update): Fix the complexity of the test if the running mode is non-adaptive test and desired complexity is not zero.
(window.runBenchmark): Add the test complexity as a new benchmark option.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@191873 268f45cc-cd09-0410-ab3c-d52691b4dbfc

PerformanceTests/Animometer/runner/animometer.html
PerformanceTests/Animometer/runner/resources/animometer.css
PerformanceTests/Animometer/runner/resources/animometer.js
PerformanceTests/Animometer/tests/resources/main.js
PerformanceTests/ChangeLog

index 0aa24672f6bfa7155d2408fbcac9371b5a6b0956..875528f59d08f122249df829742882587aa2fa8f 100644 (file)
@@ -32,7 +32,8 @@
                 <label><input id="estimated-frame-rate" type="checkbox" checked> Estimated Frame Rate</label><br>
                 <label><input id="fix-test-complexity" type="checkbox"> Fix test complexity after warmup</label><br>
                 <label><input id="show-running-results" type="checkbox"> Show running results</label><br>
                 <label><input id="estimated-frame-rate" type="checkbox" checked> Estimated Frame Rate</label><br>
                 <label><input id="fix-test-complexity" type="checkbox"> Fix test complexity after warmup</label><br>
                 <label><input id="show-running-results" type="checkbox"> Show running results</label><br>
-                <label><input id="normalize-for-device-scale-factor" type="checkbox"> Normalize for device scale factor</label>
+                <label><input id="normalize-for-device-scale-factor" type="checkbox"> Normalize for device scale factor</label><br>
+                <label><input id="adaptive-test" type="checkbox" checked onchange="benchmarkController.onChangeAdaptiveTestCheckbox()"> Adaptive test</label>
             </options>
             <footer>
                 <button class="large-button" onclick="benchmarkController.startTest()">Start Test</button>
             </options>
             <footer>
                 <button class="large-button" onclick="benchmarkController.startTest()">Start Test</button>
index e08b4c407f7f94f2da7c9007806d30eb74d58f2d..005ccc1cac477915aa117c065c5b248d69348741 100644 (file)
@@ -279,8 +279,17 @@ section#home > header > h2 {
     text-align: center;
 }
 
     text-align: center;
 }
 
-section#home > options > label > input[type="number"] {
-   width: 50px;
+section#home input[type="number"] {
+    width: 70px;
+}
+section#home > suites input[type="number"] {
+    display: none;
+    float: right;
+}
+
+section#home > suites input[type="number"].selected {
+    display: inline;
 }
 
 /* -------------------------------------------------------------------------- */
 }
 
 /* -------------------------------------------------------------------------- */
index c14e65b27585c99c9bcf638da2c38ac2dc5f189a..e567681a2e1cf41076ba1b623f23e4feab8195b0 100644 (file)
@@ -47,6 +47,7 @@ window.benchmarkRunnerClient = {
         this.score = json[Strings["JSON_SCORE"]];
         this._resultsTable.showIterations(json[Strings["JSON_RESULTS"][0]]);
         sectionsManager.showJSON("json", json[Strings["JSON_RESULTS"][0]][0]);
         this.score = json[Strings["JSON_SCORE"]];
         this._resultsTable.showIterations(json[Strings["JSON_RESULTS"][0]]);
         sectionsManager.showJSON("json", json[Strings["JSON_RESULTS"][0]][0]);
+        suitesManager.updateLocalStorageFromJSON(json[Strings["JSON_RESULTS"][0]][0]);
         benchmarkController.showResults();
     }
 }
         benchmarkController.showResults();
     }
 }
@@ -142,6 +143,11 @@ window.optionsManager =
         return document.querySelectorAll("section#home > options input");;
     },
     
         return document.querySelectorAll("section#home > options input");;
     },
     
+    _adaptiveTestElement: function()
+    {
+        return document.querySelector("section#home > options #adaptive-test");;
+    },
+    
     updateUIFromLocalStorage: function()
     {
         var optionsElements = this._optionsElements();
     updateUIFromLocalStorage: function()
     {
         var optionsElements = this._optionsElements();
@@ -197,9 +203,19 @@ window.suitesManager =
         return element.querySelector("input[type='checkbox']:not(.expand-button)");
     },
 
         return element.querySelector("input[type='checkbox']:not(.expand-button)");
     },
 
-    _localStorageNameForTest: function(suite, test)
+    _editElement: function(element)
+    {
+        return element.querySelector("input[type='number']");
+    },
+
+    _editsElements: function()
     {
     {
-        return suite.name + "/" + test.name;
+        return document.querySelectorAll("section#home > suites input[type='number']");
+    },
+        
+    _localStorageNameForTest: function(suiteName, testName)
+    {
+        return suiteName + "/" + testName;
     },
 
     _updateSuiteCheckboxState: function(suiteCheckbox)
     },
 
     _updateSuiteCheckboxState: function(suiteCheckbox)
@@ -276,6 +292,7 @@ window.suitesManager =
 
         suiteCheckbox.testsElements.push(testElement);
         span.appendChild(document.createTextNode(" " + test.name));
 
         suiteCheckbox.testsElements.push(testElement);
         span.appendChild(document.createTextNode(" " + test.name));
+        DocumentExtension.createElement("input", { type: "number" }, testElement);
         return testElement;
     },
 
         return testElement;
     },
 
@@ -294,6 +311,20 @@ window.suitesManager =
         }, this);
     },
     
         }, this);
     },
     
+    updateEditsElementsState: function()
+    {
+        var editsElements = this._editsElements();
+        var show = !optionsManager._adaptiveTestElement().checked;
+
+        for (var i = 0; i < editsElements.length; ++i) {
+            var editElement = editsElements[i];
+            if (show)
+                editElement.classList.add("selected");
+            else
+                editElement.classList.remove("selected");
+        }
+    },
+    
     updateUIFromLocalStorage: function()
     {
         var suitesElements = this._suitesElements();
     updateUIFromLocalStorage: function()
     {
         var suitesElements = this._suitesElements();
@@ -305,14 +336,16 @@ window.suitesManager =
             
             suiteCheckbox.testsElements.forEach(function(testElement) {
                 var testCheckbox = this._checkboxElement(testElement);
             
             suiteCheckbox.testsElements.forEach(function(testElement) {
                 var testCheckbox = this._checkboxElement(testElement);
+                var testEdit = this._editElement(testElement);
                 var test = testCheckbox.test;
                 
                 var test = testCheckbox.test;
                 
-                var str = localStorage.getItem(this._localStorageNameForTest(suite, test));
+                var str = localStorage.getItem(this._localStorageNameForTest(suite.name, test.name));
                 if (str === null)
                     return;
 
                 var value = JSON.parse(str);
                 testCheckbox.checked = value.checked;
                 if (str === null)
                     return;
 
                 var value = JSON.parse(str);
                 testCheckbox.checked = value.checked;
+                testEdit.value = value.complexity;
             }, this);
 
             this._updateSuiteCheckboxState(suiteCheckbox);
             }, this);
 
             this._updateSuiteCheckboxState(suiteCheckbox);
@@ -334,13 +367,16 @@ window.suitesManager =
             var tests = [];
             suiteCheckbox.testsElements.forEach(function(testElement) {
                 var testCheckbox = this._checkboxElement(testElement);
             var tests = [];
             suiteCheckbox.testsElements.forEach(function(testElement) {
                 var testCheckbox = this._checkboxElement(testElement);
+                var testEdit = this._editElement(testElement);
                 var test = testCheckbox.test;
                 
                 var test = testCheckbox.test;
                 
-                if (testCheckbox.checked)
+                if (testCheckbox.checked) {
+                    test.complexity = testEdit.value;
                     tests.push(test);
                     tests.push(test);
+                }
 
 
-                var value = { checked: testCheckbox.checked }; 
-                localStorage.setItem(this._localStorageNameForTest(suite, test), JSON.stringify(value));
+                var value = { checked: testCheckbox.checked, complexity: testEdit.value }; 
+                localStorage.setItem(this._localStorageNameForTest(suite.name, test.name), JSON.stringify(value));
             }, this);
 
             if (tests.length)
             }, this);
 
             if (tests.length)
@@ -348,6 +384,22 @@ window.suitesManager =
         }
 
         return suites;
         }
 
         return suites;
+    },
+    
+    updateLocalStorageFromJSON: function(iterationResults)
+    {
+        for (var suiteName in iterationResults[Strings["JSON_RESULTS"][1]]) {
+            var suiteResults = iterationResults[Strings["JSON_RESULTS"][1]][suiteName];
+
+            for (var testName in suiteResults[Strings["JSON_RESULTS"][2]]) {
+                var testResults = suiteResults[Strings["JSON_RESULTS"][2]][testName];
+                var data = testResults[Strings["JSON_EXPERIMENTS"][0]]
+                var complexity = Math.round(data[Strings["JSON_MEASUREMENTS"][0]]);
+
+                var value = { checked: true, complexity: complexity };
+                localStorage.setItem(this._localStorageNameForTest(suiteName, testName), JSON.stringify(value));
+            }
+        }
     }
 }
 
     }
 }
 
@@ -359,8 +411,14 @@ window.benchmarkController =
         optionsManager.updateUIFromLocalStorage();
         suitesManager.createElements();
         suitesManager.updateUIFromLocalStorage();
         optionsManager.updateUIFromLocalStorage();
         suitesManager.createElements();
         suitesManager.updateUIFromLocalStorage();
+        suitesManager.updateEditsElementsState();
     },
 
     },
 
+    onChangeAdaptiveTestCheckbox: function()
+    {
+        suitesManager.updateEditsElementsState();
+    },
+    
     _runBenchmark: function(suites, options)
     {
         benchmarkRunnerClient.initialize(suites, options);
     _runBenchmark: function(suites, options)
     {
         benchmarkRunnerClient.initialize(suites, options);
index d645610b8d2a7e0c08f0818c0f4c72c0956dc640..aa92e81d785d886090e857519928934943616595 100644 (file)
@@ -179,7 +179,11 @@ Benchmark.prototype =
         }
 
         var tuneValue = 0;
         }
 
         var tuneValue = 0;
-        if (!(this._isSampling && this.options["fix-test-complexity"])) {
+        if (this.options["complexity"] && !this.options["adaptive-test"]) {
+            // this.tune(0) returns the current complexity of the test.
+            tuneValue = this.options["complexity"] - this.tune(0);
+        }
+        else if (!(this._isSampling && this.options["fix-test-complexity"])) {
             // The relationship between frameRate and test complexity is inverse-proportional so we
             // need to use the negative of PIDController.tune() to change the complexity of the test.
             tuneValue = -this._controller.tune(currentFrameRate, timeDelta / 1000);
             // The relationship between frameRate and test complexity is inverse-proportional so we
             // need to use the negative of PIDController.tune() to change the complexity of the test.
             tuneValue = -this._controller.tune(currentFrameRate, timeDelta / 1000);
@@ -237,7 +241,9 @@ window.addEventListener("load", function()
 // This function is called from the suite controller run-callback when running the benchmark runner.
 window.runBenchmark = function(suite, test, options, recordTable, progressBar)
 {
 // This function is called from the suite controller run-callback when running the benchmark runner.
 window.runBenchmark = function(suite, test, options, recordTable, progressBar)
 {
-    var mergedOptions = Utilities.mergeObjects(options, Utilities.parseParameters());
-    window.benchmark = window.benchmarkClient.create(suite, test, mergedOptions, recordTable, progressBar);
+    var benchmarkOptions = { complexity: test.complexity };
+    benchmarkOptions = Utilities.mergeObjects(benchmarkOptions, options);
+    benchmarkOptions = Utilities.mergeObjects(benchmarkOptions, Utilities.parseParameters());
+    window.benchmark = window.benchmarkClient.create(suite, test, benchmarkOptions, recordTable, progressBar);
     return window.benchmark.run();
 }
     return window.benchmark.run();
 }
index deba5a8f0ebbed6b0104cdab0223244a4aa42031..9b1b9fa7a634c00451c0a5adade7a0b1faf7402b 100644 (file)
@@ -1,3 +1,44 @@
+2015-11-01  Said Abou-Hallawa  <sabouhallawa@apple,com>
+
+        Add an option to make the graphics benchmark runs a specific test with fixed complexity
+        https://bugs.webkit.org/show_bug.cgi?id=150529
+
+        Reviewed by Darin Adler.
+
+        Beside each test in the suites tree, we are going to show the complexity
+        arithmetic mean of the of the last run in an edit control. Based on a
+        new option these edit controls will all be visible or hidden. If they are
+        visible their values can be changed. The benchmark runner if it run in
+        the non-adaptive mode will set the complexity of the test to the passed
+        value and will not change it ever. The animator will animate the test and
+        frame rate will also be measured.
+
+        * Animometer/runner/animometer.html: Add a new option for the non-adaptive mode.
+        
+        * Animometer/runner/resources/animometer.css:
+        (section#home input[type="number"]): Define the width of all the edit control in the <home> section.
+        (section#home > suites input[type="number"]): The edit controls in the <suites> box will be right aligned and hidden by default.
+        (section#home > suites input[type="number"].selected): When the class "selected" is added, the edit controls will be visible.
+        (section#home > options > label > input[type="number"]): Deleted.
+        
+        * Animometer/runner/resources/animometer.js:
+        (window.benchmarkRunnerClient.didFinishLastIteration): Update the local storage with the results of each test.
+        (window.optionsManager._adaptiveTestElement): Returns the checkbox for setting the adaptive test option.
+        (window.suitesManager._editElement):  Returns the edit element associated with each test element in the suites tree.
+        (window.suitesManager._editsElements): Returns a list of all the elements in the <suites> box.
+        (window.suitesManager._localStorageNameForTest): Change this function to take strings.
+        (window.suitesManager._createTestElement): Adds an edit control beside each test.
+        (window.suitesManager.updateEditsElementsState): Adds/Removes the 'selected' class to/from all the tests edit elements.
+        (window.suitesManager.updateUIFromLocalStorage): Reads the edit control value from the local storage.
+        (window.suitesManager.updateLocalStorageFromUI): Saves the edit control value to the local storage.
+        (window.suitesManager.updateLocalStorageFromJSON): Saves the last run results to the local storage.
+        (window.benchmarkController.initialize): Shows/Hides the test edit controls based on the adaptive test option.
+        (window.benchmarkController.onChangeAdaptiveTestCheckbox): An onchange event handler for the adaptive test checkbox.
+        
+        * Animometer/tests/resources/main.js: 
+        (Benchmark.prototype.update): Fix the complexity of the test if the running mode is non-adaptive test and desired complexity is not zero.
+        (window.runBenchmark): Add the test complexity as a new benchmark option.
+
 2015-11-01  Said Abou-Hallawa  <sabouhallawa@apple,com>
 
         Make the size of the benchmark canvas adaptive to the screen size and screen resolution
 2015-11-01  Said Abou-Hallawa  <sabouhallawa@apple,com>
 
         Make the size of the benchmark canvas adaptive to the screen size and screen resolution