build.webkit.org should have a clean build button
authorrniwa@webkit.org <rniwa@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 31 Oct 2013 21:15:08 +0000 (21:15 +0000)
committerrniwa@webkit.org <rniwa@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 31 Oct 2013 21:15:08 +0000 (21:15 +0000)
https://bugs.webkit.org/show_bug.cgi?id=123559

Reviewed by Darin Adler.

Add a new "clean" forced scheduler. When a build was created by this scheduler, CleanBuildIfScheduled
step executes clean-build script that removes the WebKitBuild directory.

Also added make-passwords-json.py which creates a fake passwords.json from config.json for testing purposes.

* BuildSlaveSupport/build.webkit.org-config/make-passwords-json.py: Added.
* BuildSlaveSupport/build.webkit.org-config/master.cfg:
(CleanBuildIfScheduled): Added.
(CleanBuildIfScheduled.start): Added. Don't do a clean build if if this build wasn't schecueld by
the clean build scheduler.
(DeleteStaleBuildFiles): Added.
(DeleteStaleBuildFiles.start): Added. Don't delete stale build files if this build was scheduled by
the clean build scheduler since everything in the WebKitBuild directory had already been removed.
(Factory.__init__): Always add CleanBuildIfScheduled.  The step itself checks whether to run
clean-build script or hide itself.
(loadBuilderConfig): Added the new clean build scheduler. Also make ForceScheduler future proof; in newer
versions of buildbot, builderNames needs to be a list of regular strings.
* BuildSlaveSupport/clean-build: Added.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@158393 268f45cc-cd09-0410-ab3c-d52691b4dbfc

Tools/BuildSlaveSupport/build.webkit.org-config/make-passwords-json.py [new file with mode: 0755]
Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg
Tools/BuildSlaveSupport/clean-build [new file with mode: 0755]
Tools/ChangeLog

diff --git a/Tools/BuildSlaveSupport/build.webkit.org-config/make-passwords-json.py b/Tools/BuildSlaveSupport/build.webkit.org-config/make-passwords-json.py
new file mode 100755 (executable)
index 0000000..ff4d67d
--- /dev/null
@@ -0,0 +1,12 @@
+#!/usr/bin/python
+
+import json
+
+with open('config.json', 'r') as config_file:
+    config = json.load(config_file)
+    passwords = {}
+    for slave in config['slaves']:
+        passwords[slave['name']] = 'a'
+
+with open('passwords.json', 'w') as passwords_file:
+    passwords_file.write(json.dumps(passwords))
index 8a6b4a9035806e8db116b3e0d7c60d07cf1e25dd..27b1189b6e34c6747afe32121da3ee189088a42a 100644 (file)
@@ -4,7 +4,7 @@
 from buildbot.buildslave import BuildSlave
 from buildbot.changes.pb import PBChangeSource
 from buildbot.scheduler import AnyBranchScheduler, Triggerable
-from buildbot.schedulers.forcesched import FixedParameter, ForceScheduler, StringParameter
+from buildbot.schedulers.forcesched import FixedParameter, ForceScheduler, StringParameter, BooleanParameter
 from buildbot.schedulers.filter import ChangeFilter
 from buildbot.status import html
 from buildbot.status.web.authz import Authz
@@ -145,12 +145,30 @@ class KillOldProcesses(shell.Compile):
     descriptionDone = ["killed old processes"]
     command = ["python", "./Tools/BuildSlaveSupport/kill-old-processes"]
 
+class CleanBuildIfScheduled(shell.Compile):
+    name = "delete WebKitBuild directory"
+    description = ["deleting WebKitBuild directory"]
+    descriptionDone = ["deleted WebKitBuild directory"]
+    command = ["python", "./Tools/BuildSlaveSupport/clean-build", WithProperties("--platform=%(fullPlatform)s"), WithProperties("--%(configuration)s")]
+
+    def start(self):
+        if not self.getProperty('is_clean'):
+            self.hideStepIf = True
+            return SKIPPED
+        return shell.Compile.start(self)
+
 class DeleteStaleBuildFiles(shell.Compile):
     name = "delete stale build files"
     description = ["deleting stale build files"]
-    descriptionDone = ["delete stale build files"]
+    descriptionDone = ["deleted stale build files"]
     command = ["python", "./Tools/BuildSlaveSupport/delete-stale-build-files", WithProperties("--platform=%(fullPlatform)s"), WithProperties("--%(configuration)s")]
 
+    def start(self):
+        if self.getProperty('is_clean'): # Nothing to be done if WebKitBuild had been removed.
+            self.hideStepIf = True
+            return SKIPPED
+        return shell.Compile.start(self)
+
 class InstallEflDependencies(shell.ShellCommand):
     name = "jhbuild"
     description = ["updating efl dependencies"]
@@ -717,6 +735,7 @@ class Factory(factory.BuildFactory):
             self.addStep(WaitForSVNServer())
         self.addStep(CheckOutSource(SVNMirror=SVNMirror))
         self.addStep(KillOldProcesses())
+        self.addStep(CleanBuildIfScheduled())
         self.addStep(DeleteStaleBuildFiles())
         if platform == "win":
             self.addStep(InstallWin32Dependencies())
@@ -876,17 +895,17 @@ def loadBuilderConfig(c):
 
     forceScheduler = ForceScheduler(
         name="force",
-        builderNames=[builder['name'] for builder in config['builders']],
+        builderNames=[str(builder['name']) for builder in config['builders']],
         reason=StringParameter(name="reason", default="", size=40),
 
-        # Validate SVN revision: number or emtpy string
+        # Validate SVN revision: number or empty string
         revision=StringParameter(name="revision", default="", regex=re.compile(r'^(\d*)$')),
 
         # Disable default enabled input fields: branch, repository, project, additional properties
         branch=FixedParameter(name="branch"),
         repository=FixedParameter(name="repository"),
         project=FixedParameter(name="project"),
-        properties=[]
+        properties=[BooleanParameter(name="is_clean", label="Force Clean build")]
     )
     c['schedulers'].append(forceScheduler)
 
diff --git a/Tools/BuildSlaveSupport/clean-build b/Tools/BuildSlaveSupport/clean-build
new file mode 100755 (executable)
index 0000000..eaecb12
--- /dev/null
@@ -0,0 +1,54 @@
+#!/usr/bin/python
+# Copyright (C) 2013 Apple Inc.  All rights reserved.
+# Copyright (C) 2012 Google Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# 1.  Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+# 2.  Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in the
+#     documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY APPLE AND ITS CONTRIBUTORS "AS IS" AND ANY
+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+# DISCLAIMED. IN NO EVENT SHALL APPLE OR ITS CONTRIBUTORS BE LIABLE FOR ANY
+# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
+# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+import optparse
+import os
+import shutil
+import subprocess
+import sys
+
+def main():
+    parser = optparse.OptionParser("usage: %prog [options]")
+    parser.add_option("--platform")
+    parser.add_option("--debug", action="store_const", const="debug", dest="configuration")
+    parser.add_option("--release", action="store_const", const="release", dest="configuration")
+
+    options, parameters = parser.parse_args()
+    if not options.platform:
+        parser.error("Platform is required")
+        return -1
+    if not options.configuration:
+        parser.error("Configuration is required")
+        return -2
+
+    generic_platform = options.platform.split('-', 1)[0]
+
+    webkit_build_directory = subprocess.Popen(['perl', os.path.join(os.path.dirname(__file__), "..", "Scripts", "webkit-build-directory"),
+        "--" + generic_platform, "--" + options.configuration, '--top-level'], stdout=subprocess.PIPE).communicate()[0].strip()
+
+    shutil.rmtree(webkit_build_directory)
+
+if __name__ == '__main__':
+    sys.exit(main())
index 9a759a328abee59f4dd18604ea8311b5efea3301..e8f00b64e5f64668ac54f5fd039cbcaaa2f2f4cb 100644 (file)
@@ -1,3 +1,29 @@
+2013-10-30  Ryosuke Niwa  <rniwa@webkit.org>
+
+        build.webkit.org should have a clean build button
+        https://bugs.webkit.org/show_bug.cgi?id=123559
+
+        Reviewed by Darin Adler.
+
+        Add a new "clean" forced scheduler. When a build was created by this scheduler, CleanBuildIfScheduled
+        step executes clean-build script that removes the WebKitBuild directory.
+
+        Also added make-passwords-json.py which creates a fake passwords.json from config.json for testing purposes.
+
+        * BuildSlaveSupport/build.webkit.org-config/make-passwords-json.py: Added.
+        * BuildSlaveSupport/build.webkit.org-config/master.cfg:
+        (CleanBuildIfScheduled): Added.
+        (CleanBuildIfScheduled.start): Added. Don't do a clean build if if this build wasn't schecueld by
+        the clean build scheduler.
+        (DeleteStaleBuildFiles): Added.
+        (DeleteStaleBuildFiles.start): Added. Don't delete stale build files if this build was scheduled by
+        the clean build scheduler since everything in the WebKitBuild directory had already been removed.
+        (Factory.__init__): Always add CleanBuildIfScheduled.  The step itself checks whether to run
+        clean-build script or hide itself.
+        (loadBuilderConfig): Added the new clean build scheduler. Also make ForceScheduler future proof; in newer
+        versions of buildbot, builderNames needs to be a list of regular strings.
+        * BuildSlaveSupport/clean-build: Added.
+
 2013-10-31  Filip Pizlo  <fpizlo@apple.com>
 
         Remove CachedTranscendentalFunction because caching math functions is an ugly idea