Each build request should be associated with a repository group
authorrniwa@webkit.org <rniwa@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 6 Apr 2017 21:56:59 +0000 (21:56 +0000)
committerrniwa@webkit.org <rniwa@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 6 Apr 2017 21:56:59 +0000 (21:56 +0000)
https://bugs.webkit.org/show_bug.cgi?id=170528

Rubber-stamped by Chris Dumez.

Make the buildbot syncing script use the concept of repository groups so that each repository group can post
a different set of properties to buildbot. In order to do this, we associate each build request with
a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via
/api/update-triggerable just the same way the set of the supported platform, test pairs are updated.

Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name
to either a string value or a repository name enclosed in < and >:

```js
"repositoryGroups": {
    "webkit-svn": {
        "repositories": ["WebKit", "macOS"],
        "properties": {"os": "<macOS>", "wk": "<WebKit>"}
    }
}
```

With this, removed the support for specifying a repository to use in generic dictionary of properties via
a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of
repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in
build requests.

After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group.
Run the appropriate database queries to set the repository group on each build request. Because of this change,
this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests.
Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same
test group if the next build request to be scheduled does not specify a repository group.

* init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for
repository and group pair in triggerable_repositories table.

* public/api/update-triggerable.php:
(main): Validate and insert repository groups.
(validate_configurations): Extracted from main.
(validate_repository_groups): Added.

* public/v3/models/repository.js:
(Repository.findTopLevelByName): Added.

* public/include/build-requests-fetcher.php:
(BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response.

* public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given
triggerable for a list of repositories.
(RepositoryGroupFinder): Added.
(RepositoryGroupFinder::__construct): Added.
(RepositoryGroupFinder::find_by_repositories): Added.
(RepositoryGroupFinder::populate_map): Added.

* public/privileged-api/create-test-group.php:
(main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains
"set", the list of commit IDs, and "repository_group", the repository group identified for each commit set.
Use that to set the repository group in each new build request.
(commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group.
(ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its
argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]}

* public/v3/models/build-request.js:
(BuildRequest):
(BuildRequest.prototype.triggerable): Added.
(BuildRequest.prototype.repositoryGroup): Added.
(BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group.

* public/v3/models/triggerable.js:
(Triggerable.prototype.name): Added.
(Triggerable.prototype.acceptedRepositories): Deleted.
(TriggerableRepositoryGroup):
(TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group

* server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request.
* server-tests/api-manifest-tests.js: Added assertions for the repository groups.
* server-tests/api-report-tests.js:
(.emptyReport):
(.reportWithTwoLevelsOfAggregations):
* server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with
a triggerable.
(.updateWithOSXRepositoryGroup):
(.mapRepositoriesByGroup):
* server-tests/privileged-api-create-test-group-tests.js:
(addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups
in existing test cases, and added a test case for creating a test group with two different repository groups.

* server-tests/resources/mock-data.js:
(MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps.
(MockData.emptyTriggeragbleId): Added.
(MockData.macosRepositoryId): Added.
(MockData.webkitRepositoryId): Added.
(MockData.gitWebkitRepositoryId): Added.
(MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some
tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks.
(MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js.
(MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js.
(MockData.addAnotherMockTestGroup): Cleanup.
(MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes.
(MockData.mockTestSyncConfigWithTwoBuilders): Ditto.

* server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test
updating the set of repository groups in addition to the set of test, platform pairs.
(.refetchManifest): Added.

* tools/js/buildbot-syncer.js:
(BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument,
and buildRequestArgument as the third argument.
(BuildbotSyncer.prototype.repositoryGroups): Added.
(BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find.
Also added an assertion that the build request is associated with a repository group.
(BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to
specify a revision in favor of explicity listing each property and repository name in a repository group.
(BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties
shared across syncers, the name of properties which specifies the build slave name and build request ID. These
values are not stored as top-level properties and superseded by the concept of repository groups.
(BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups.
(BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of
properties or buildRequestArgument (often inherited from shared).
(BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of
allowed proeprties in each configuration now that they're specified as top-level properties.

* tools/js/buildbot-triggerable.js:
(BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups.
(BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is
not included in the list of valid build requests.
(BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes
those that lack a repository group set.
(BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds
the next build request to be scheduled for the test group.
(BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from
_scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of
a test group information since syncOnce now calls _nextRequestInGroup to find the next build request.

* tools/js/v3-models.js:

* unit-tests/build-request-tests.js: Fixed the test name.

* unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests
for parsing repository groups.
(sampleiOSConfig): Updated the mock configuration per code changes.
(sampleiOSConfigWithExpansions): Ditto.
(smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration.
Various test cases have been updated to reflect this.
(createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock
configurations. The git hash was there to test "rootOptions", which this patch removed.
(samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding"
which, again, this patch removed.
(sampleInProgressBuild): Ditto.
(sampleFinishedBuild): Ditto.

* unit-tests/resources/mock-v3-models.js:
(MockModels.inject): Added ock repository groups so that existing tests will continue to function.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@215061 268f45cc-cd09-0410-ab3c-d52691b4dbfc

22 files changed:
Websites/perf.webkit.org/ChangeLog
Websites/perf.webkit.org/init-database.sql
Websites/perf.webkit.org/public/api/update-triggerable.php
Websites/perf.webkit.org/public/include/build-requests-fetcher.php
Websites/perf.webkit.org/public/include/repository-group-finder.php [new file with mode: 0644]
Websites/perf.webkit.org/public/privileged-api/create-test-group.php
Websites/perf.webkit.org/public/v3/models/build-request.js
Websites/perf.webkit.org/public/v3/models/repository.js
Websites/perf.webkit.org/public/v3/models/triggerable.js
Websites/perf.webkit.org/server-tests/api-build-requests-tests.js
Websites/perf.webkit.org/server-tests/api-manifest-tests.js
Websites/perf.webkit.org/server-tests/api-report-tests.js
Websites/perf.webkit.org/server-tests/api-update-triggerable.js
Websites/perf.webkit.org/server-tests/privileged-api-create-test-group-tests.js
Websites/perf.webkit.org/server-tests/resources/mock-data.js
Websites/perf.webkit.org/server-tests/tools-buildbot-triggerable-tests.js
Websites/perf.webkit.org/tools/js/buildbot-syncer.js
Websites/perf.webkit.org/tools/js/buildbot-triggerable.js
Websites/perf.webkit.org/tools/js/v3-models.js
Websites/perf.webkit.org/unit-tests/build-request-tests.js
Websites/perf.webkit.org/unit-tests/buildbot-syncer-tests.js
Websites/perf.webkit.org/unit-tests/resources/mock-v3-models.js

index 6d0edd3..829695c 100644 (file)
@@ -1,3 +1,159 @@
+2017-04-06  Ryosuke Niwa  <rniwa@webkit.org>
+
+        Each build request should be associated with a repository group
+        https://bugs.webkit.org/show_bug.cgi?id=170528
+
+        Rubber-stamped by Chris Dumez.
+
+        Make the buildbot syncing script use the concept of repository groups so that each repository group can post
+        a different set of properties to buildbot. In order to do this, we associate each build request with
+        a repository group to use. Each triggerable's repository groups is now updated by the syncing scripts via
+        /api/update-triggerable just the same way the set of the supported platform, test pairs are updated.
+
+        Each repository group specifies the list of repositories, a dictionary that maps the buildbot property name
+        to either a string value or a repository name enclosed in < and >:
+
+        ```js
+        "repositoryGroups": {
+            "webkit-svn": {
+                "repositories": ["WebKit", "macOS"],
+                "properties": {"os": "<macOS>", "wk": "<WebKit>"}
+            }
+        }
+        ```
+
+        With this, removed the support for specifying a repository to use in generic dictionary of properties via
+        a dictionary with a single key of "root", "rootOptions", and "rootsExcluding". We now validate that the list of
+        repositories in each repository group matches exactly the ones used in buildbot properties as well as ones in
+        build requests.
+
+        After this patch, sync-with-buildbot.js will no longer schedule a build request without a repository group.
+        Run the appropriate database queries to set the repository group on each build request. Because of this change,
+        this patch also makes BuildbotTriggerable.prototype.syncOnce more robust against invalid build requests.
+        Instead of throwing an exception and exiting early, it simply skips all build requests that belong to the same
+        test group if the next build request to be scheduled does not specify a repository group.
+
+        * init-database.sql: Add request_repository_group column to build_requests table, and a unique constraint for
+        repository and group pair in triggerable_repositories table.
+
+        * public/api/update-triggerable.php:
+        (main): Validate and insert repository groups.
+        (validate_configurations): Extracted from main.
+        (validate_repository_groups): Added.
+
+        * public/v3/models/repository.js:
+        (Repository.findTopLevelByName): Added.
+
+        * public/include/build-requests-fetcher.php:
+        (BuildRequestsFetcher::results_internal): Include the repository group of each request in the JSON response.
+
+        * public/include/repository-group-finder.php: Added. A helper class to find the repository group for a given
+        triggerable for a list of repositories.
+        (RepositoryGroupFinder): Added. 
+        (RepositoryGroupFinder::__construct): Added.
+        (RepositoryGroupFinder::find_by_repositories): Added.
+        (RepositoryGroupFinder::populate_map): Added.
+
+        * public/privileged-api/create-test-group.php:
+        (main): Each element in an array returned by ensure_commit_sets and commit_sets_from_revision_sets now contains
+        "set", the list of commit IDs, and "repository_group", the repository group identified for each commit set.
+        Use that to set the repository group in each new build request. 
+        (commit_sets_from_revision_sets): Use RepositoryGroupFinder to find the right repository group.
+        (ensure_commit_sets): Ditto. There is no need to find a repository group for each commit set here since its
+        argument is keyed by the repository name. e.g. {"WebKit": [123, 456], "macOS": ["16A323", "16A323"]}
+
+        * public/v3/models/build-request.js:
+        (BuildRequest):
+        (BuildRequest.prototype.triggerable): Added.
+        (BuildRequest.prototype.repositoryGroup): Added.
+        (BuildRequest.constructBuildRequestsFromData): Resolve the triggerable and the repository group.
+
+        * public/v3/models/triggerable.js:
+        (Triggerable.prototype.name): Added.
+        (Triggerable.prototype.acceptedRepositories): Deleted.
+        (TriggerableRepositoryGroup):
+        (TriggerableRepositoryGroup.prototype.accepts): Added. Retruns true if the repository group
+
+        * server-tests/api-build-requests-tests.js: Added a test for getting the repository group of a build request.
+        * server-tests/api-manifest-tests.js: Added assertions for the repository groups.
+        * server-tests/api-report-tests.js:
+        (.emptyReport):
+        (.reportWithTwoLevelsOfAggregations):
+        * server-tests/api-update-triggerable.js: Added test cases for updating the repository groups associated with
+        a triggerable.
+        (.updateWithOSXRepositoryGroup):
+        (.mapRepositoriesByGroup):
+        * server-tests/privileged-api-create-test-group-tests.js:
+        (addTriggerableAndCreateTask): Add two repository groups for testing. Added assertions for repository groups
+        in existing test cases, and added a test case for creating a test group with two different repository groups.
+
+        * server-tests/resources/mock-data.js:
+        (MockData.resetV3Models): Reset TriggerableRepositoryGroup's static maps.
+        (MockData.emptyTriggeragbleId): Added.
+        (MockData.macosRepositoryId): Added.
+        (MockData.webkitRepositoryId): Added.
+        (MockData.gitWebkitRepositoryId): Added.
+        (MockData.addMockData): Create repository groups as needed. Renamed the "OS X" repository to "macOS" since some
+        tests were using the latter, and now we need mock data to be consistent across tests due to stricter checks.
+        (MockData.addEmptyTriggerable): Added. Used in api-update-triggerable.js.
+        (MockData.addMockTestGroupWithGitWebKit): Added. Used in api-build-requests-tests.js.
+        (MockData.addAnotherMockTestGroup): Cleanup.
+        (MockData.mockTestSyncConfigWithSingleBuilder): Updated the mock configuration per code changes.
+        (MockData.mockTestSyncConfigWithTwoBuilders): Ditto.
+
+        * server-tests/tools-buildbot-triggerable-tests.js: Updated a test case testing /api/update-triggerable to test
+        updating the set of repository groups in addition to the set of test, platform pairs.
+        (.refetchManifest): Added.
+
+        * tools/js/buildbot-syncer.js:
+        (BuildbotSyncer): Now takes a set of configurations shared across syncers: repositoryGroups, slaveArgument,
+        and buildRequestArgument as the third argument.
+        (BuildbotSyncer.prototype.repositoryGroups): Added.
+        (BuildbotSyncer.prototype._testGroupMapForBuildRequests): Cleaned up the code to use Array.prototype.find.
+        Also added an assertion that the build request is associated with a repository group.
+        (BuildbotSyncer.prototype._propertiesForBuildRequest): Removed the support for using an arbitary property to
+        specify a revision in favor of explicity listing each property and repository name in a repository group.
+        (BuildbotSyncer._loadConfig): Removed the support for "shared", which specified the set of buildbot properties
+        shared across syncers, the name of properties which specifies the build slave name and build request ID. These
+        values are not stored as top-level properties and superseded by the concept of repository groups.
+        (BuildbotSyncer._parseRepositoryGroup): Parses and validates repository groups.
+        (BuildbotSyncer._createTestConfiguration): We no longer expect each configuration to specify a dictionary of
+        properties or buildRequestArgument (often inherited from shared).
+        (BuildbotSyncer._validateAndMergeConfig): Removed "slaveArgument" and "buildRequestArgument" from the list of
+        allowed proeprties in each configuration now that they're specified as top-level properties.
+
+        * tools/js/buildbot-triggerable.js:
+        (BuildbotTriggerable.prototype.updateTriggerable): Update the associated repository groups.
+        (BuildbotTriggerable.prototype.syncOnce): Skip test groups for which the next build request to be scheduled is
+        not included in the list of valid build requests.
+        (BuildbotTriggerable.prototype._validateRequests): Now returns the list of valid build requests, which excludes
+        those that lack a repository group set.
+        (BuildbotTriggerable.prototype._nextRequestInGroup): Extracted from _scheduleRequestIfSlaveIsAvailable. Finds
+        the next build request to be scheduled for the test group.
+        (BuildbotTriggerable.prototype._scheduleRequestIfSlaveIsAvailable): Renamed from
+        _scheduleNextRequestInGroupIfSlaveIsAvailable. Now takes the syncer and the slave name as arguments instead of
+        a test group information since syncOnce now calls _nextRequestInGroup to find the next build request.
+
+        * tools/js/v3-models.js:
+
+        * unit-tests/build-request-tests.js: Fixed the test name.
+
+        * unit-tests/buildbot-syncer-tests.js: Removed tests for "rootOptions" and "rootsExcluding", and added tests
+        for parsing repository groups.
+        (sampleiOSConfig): Updated the mock configuration per code changes.
+        (sampleiOSConfigWithExpansions): Ditto.
+        (smallConfiguration): Ditto. Now returns the entire configuration instead of a single builder configuration.
+        Various test cases have been updated to reflect this.
+        (createSampleBuildRequest): Removed the git hash of WebKit to match the repository groups listed in the mock
+        configurations. The git hash was there to test "rootOptions", which this patch removed.
+        (samplePendingBuild): Removed "root_dict" from the list of properties. This was used to test "rootsExcluding"
+        which, again, this patch removed.
+        (sampleInProgressBuild): Ditto.
+        (sampleFinishedBuild): Ditto.
+
+        * unit-tests/resources/mock-v3-models.js:
+        (MockModels.inject): Added ock repository groups so that existing tests will continue to function.
+
 2017-04-05  Ryosuke Niwa  <rniwa@webkit.org>
 
         Introduce the notion of repository groups to triggerables
index 05851e1..a448cbf 100644 (file)
@@ -243,7 +243,8 @@ CREATE TABLE triggerable_repository_groups (
 
 CREATE TABLE triggerable_repositories (
     trigrepo_repository integer REFERENCES repositories NOT NULL,
-    trigrepo_group integer REFERENCES triggerable_repository_groups NOT NULL);
+    trigrepo_group integer REFERENCES triggerable_repository_groups NOT NULL,
+    CONSTRAINT repository_must_be_unique_for_repository_group UNIQUE(trigrepo_repository, trigrepo_group));
 
 CREATE TABLE triggerable_configurations (
     trigconfig_test integer REFERENCES tests NOT NULL,
@@ -285,6 +286,7 @@ CREATE TYPE build_request_status_type as ENUM ('pending', 'scheduled', 'running'
 CREATE TABLE build_requests (
     request_id serial PRIMARY KEY,
     request_triggerable integer REFERENCES build_triggerables NOT NULL,
+    request_repository_group integer REFERENCES triggerable_repository_groups,
     request_platform integer REFERENCES platforms NOT NULL,
     request_test integer REFERENCES tests NOT NULL,
     request_group integer REFERENCES analysis_test_groups NOT NULL,
index c0a1f7a..e823966 100644 (file)
@@ -1,8 +1,10 @@
 <?php
 
-require('../include/json-header.php');
+require_once('../include/json-header.php');
+require_once('../include/repository-group-finder.php');
 
-function main($post_data) {
+function main($post_data)
+{
     $db = new Database;
     if (!$db->connect())
         exit_with_error('DatabaseConnectionFailure');
@@ -17,13 +19,14 @@ function main($post_data) {
     $triggerable_id = $triggerable['triggerable_id'];
 
     $configurations = array_get($report, 'configurations');
-    if (!is_array($configurations))
-        exit_with_error('InvalidConfigurations', array('configurations' => $configurations));
+    validate_configurations($db, $configurations);
 
-    foreach ($configurations as $entry) {
-        if (!is_array($entry) || !array_key_exists('test', $entry) || !array_key_exists('platform', $entry))
-            exit_with_error('InvalidConfigurationEntry', array('configurationEntry' => $entry));
-    }
+    $repository_groups = array_get($report, 'repositoryGroups', array());
+    validate_repository_groups($db, $repository_groups);
+
+    $finder = new RepositoryGroupFinder($db, $triggerable_id);
+    foreach ($repository_groups as &$group)
+        $group['existingGroup'] = $finder->find_by_repositories($group['repositories']);
 
     $db->begin_transaction();
     if ($db->query_and_get_affected_rows('DELETE FROM triggerable_configurations WHERE trigconfig_triggerable = $1', array($triggerable_id)) === false) {
@@ -31,7 +34,7 @@ function main($post_data) {
         exit_with_error('FailedToDeleteExistingConfigurations', array('triggerable' => $triggerable_id));
     }
 
-    foreach ($configurations as $entry) {
+    foreach ($configurations as &$entry) {
         $config_info = array('test' => $entry['test'], 'platform' => $entry['platform'], 'triggerable' => $triggerable_id);
         if (!$db->insert_row('triggerable_configurations', 'trigconfig', $config_info, null)) {
             $db->rollback_transaction();
@@ -39,10 +42,73 @@ function main($post_data) {
         }
     }
 
+    foreach ($repository_groups as &$group) {
+        $group_id = $group['existingGroup'];
+        if ($group_id) {
+            $group_info = array('name' => $group['name'], 'description' => array_get($group, 'description'));
+            if (!$db->update_row('triggerable_repository_groups', 'repositorygroup', array('id' => $group_id), $group_info)) {
+                $db->rollback_transaction();
+                exit_with_error('FailedToInsertRepositoryGroup', array('repositoryGroup' => $group));
+            }
+        } else {
+            $group_id = $db->update_or_insert_row('triggerable_repository_groups', 'repositorygroup',
+                array('triggerable' => $triggerable_id, 'name' => $group['name']),
+                array('triggerable' => $triggerable_id, 'name' => $group['name'], 'description' => array_get($group, 'description')));
+            if (!$group_id) {
+                $db->rollback_transaction();
+                exit_with_error('FailedToInsertRepositoryGroup', array('repositoryGroup' => $group));
+            }
+        }
+        if ($db->query_and_get_affected_rows('DELETE FROM triggerable_repositories WHERE trigrepo_group = $1', array($group_id)) === FALSE) {
+            $db->rollback_transaction();
+            exit_with_error('FailedToDisassociateRepositories', array('repositoryGroup' => $group));
+        }
+        foreach ($group['repositories'] as $repository_id) {
+            if (!$db->insert_row('triggerable_repositories', 'trigrepo', array('group' => $group_id, 'repository' => $repository_id), null)) {
+                $db->rollback_transaction();
+                exit_with_error('FailedToAssociateRepository', array('repositoryGroup' => $group, 'repository' => $repository_id));
+            }
+        }
+    }
+
     $db->commit_transaction();
     exit_with_success();
 }
 
+function validate_configurations($db, $configurations)
+{
+    if (!is_array($configurations))
+        exit_with_error('InvalidConfigurations', array('configurations' => $configurations));
+
+    foreach ($configurations as $entry) {
+        if (!is_array($entry) || !array_key_exists('test', $entry) || !array_key_exists('platform', $entry))
+            exit_with_error('InvalidConfigurationEntry', array('configurationEntry' => $entry));
+    }
+}
+
+function validate_repository_groups($db, $repository_groups)
+{
+    if (!is_array($repository_groups))
+        exit_with_error('InvalidRepositoryGroups', array('repositoryGroups' => $repository_groups));
+
+    $top_level_repositories = $db->select_rows('repositories', 'repository', array('owner' => null));
+    $top_level_repository_ids = array();
+    foreach ($top_level_repositories as $repository_row)
+        $top_level_repository_ids[$repository_row['repository_id']] = true;
+
+    foreach ($repository_groups as &$group) {
+        if (!is_array($group) || !array_key_exists('name', $group) || !array_key_exists('repositories', $group) || !is_array($group['repositories']))
+            exit_with_error('InvalidRepositoryGroup', array('repositoryGroup' => $group));
+        $repository_list = $group['repositories'];
+        $group_repository_list = array();
+        foreach ($repository_list as $repository_id) {
+            if (!array_key_exists($repository_id, $top_level_repository_ids) || array_key_exists($repository_id, $group_repository_list))
+                exit_with_error('InvalidRepository', array('repositoryGroup' => $group, 'repository' => $repository_id));
+            $group_repository_list[$repository_id] = true;
+        }
+    }
+}
+
 main($HTTP_RAW_POST_DATA);
 
 ?>
index 2f2464c..9a929b4 100644 (file)
@@ -68,6 +68,7 @@ class BuildRequestsFetcher {
                 'id' => $row['request_id'],
                 'task' => $row['task_id'],
                 'triggerable' => $row['request_triggerable'],
+                'repositoryGroup' => $row['request_repository_group'],
                 'test' => $resolve_ids ? $test_path_resolver->path_for_test($test_id) : $test_id,
                 'platform' => $resolve_ids ? $id_to_platform_name[$platform_id] : $platform_id,
                 'testGroup' => $row['request_group'],
diff --git a/Websites/perf.webkit.org/public/include/repository-group-finder.php b/Websites/perf.webkit.org/public/include/repository-group-finder.php
new file mode 100644 (file)
index 0000000..7264153
--- /dev/null
@@ -0,0 +1,43 @@
+<?php
+
+class RepositoryGroupFinder
+{
+
+    function __construct($db, $triggerable_id) {
+        $this->db = $db;
+        $this->triggerable_id = $triggerable_id;
+        $this->repositories_by_group = NULL;
+    }
+
+    function find_by_repositories($repositories)
+    {
+        if ($this->repositories_by_group === NULL)
+            $this->populate_map();
+        sort($repositories, SORT_NUMERIC);
+        foreach ($this->repositories_by_group as $group_id => $group_repositories) {
+            if (count($repositories) == count($group_repositories) && !array_diff($repositories, $group_repositories))
+                return $group_id;
+        }
+        return NULL;
+    }
+
+    private function populate_map()
+    {
+        $repository_rows = $this->db->query_and_fetch_all('SELECT * FROM triggerable_repositories WHERE trigrepo_group IN
+            (SELECT repositorygroup_id FROM triggerable_repository_groups WHERE repositorygroup_triggerable = $1)
+            ORDER BY trigrepo_group, trigrepo_repository', array($this->triggerable_id));
+        if ($repository_rows === FALSE)
+            exit_with_error('FailedToFetchRepositoryGroups', array('triggerable' => $this->triggerable_id));
+
+        $repositories_by_group = array();
+        foreach ($repository_rows as &$row) {
+            $group_id = $row['trigrepo_group'];
+            array_ensure_item_has_array($repositories_by_group, $group_id);
+            array_push($repositories_by_group[$group_id], $row['trigrepo_repository']);
+        }
+
+        $this->repositories_by_group = &$repositories_by_group;
+    }
+}
+
+?>
index 56289a5..6561050 100644 (file)
@@ -1,8 +1,10 @@
 <?php
 
 require_once('../include/json-header.php');
+require_once('../include/repository-group-finder.php');
 
-function main() {
+function main()
+{
     $db = connect();
     $data = ensure_privileged_api_data_and_token_or_slave($db);
     $author = remote_user_name($data);
@@ -35,18 +37,18 @@ function main() {
         exit_with_error('TriggerableNotFoundForTask', array('task' => $task_id));
 
     if ($revision_set_list)
-        $commit_sets = commit_sets_from_revision_sets($db, $revision_set_list);
+        $commit_sets = commit_sets_from_revision_sets($db, $triggerable['id'], $revision_set_list);
     else // V2 UI compatibility
-        $commit_sets = ensure_commit_sets($db, $commit_sets_info);
+        $commit_sets = ensure_commit_sets($db, $triggerable['id'], $commit_sets_info);
 
     $db->begin_transaction();
 
-    $commit_set_id_list = array();
+    $configuration_list = array();
     foreach ($commit_sets as $commit_list) {
         $commit_set_id = $db->insert_row('commit_sets', 'commitset', array());
-        foreach ($commit_list as $commit)
+        foreach ($commit_list['set'] as $commit)
             $db->insert_row('commit_set_relationships', 'commitset', array('set' => $commit_set_id, 'commit' => $commit), 'commit');
-        array_push($commit_set_id_list, $commit_set_id);
+        array_push($configuration_list, array('commit_set' => $commit_set_id, 'repository_group' => $commit_list['repository_group']));
     }
 
     $group_id = $db->insert_row('analysis_test_groups', 'testgroup',
@@ -54,14 +56,15 @@ function main() {
 
     $order = 0;
     for ($i = 0; $i < $repetition_count; $i++) {
-        foreach ($commit_set_id_list as $commit_set_id) {
+        foreach ($configuration_list as $config) {
             $db->insert_row('build_requests', 'request', array(
                 'triggerable' => $triggerable['id'],
+                'repository_group' => $config['repository_group'],
                 'platform' => $triggerable['platform'],
                 'test' => $triggerable['test'],
                 'group' => $group_id,
                 'order' => $order,
-                'commit_set' => $commit_set_id));
+                'commit_set' => $config['commit_set'],));
             $order++;
         }
     }
@@ -71,18 +74,20 @@ function main() {
     exit_with_success(array('testGroupId' => $group_id));
 }
 
-function commit_sets_from_revision_sets($db, $revision_set_list)
+function commit_sets_from_revision_sets($db, $triggerable_id, $revision_set_list)
 {
     if (count($revision_set_list) < 2)
         exit_with_error('InvalidRevisionSets', array('revisionSets' => $revision_set_list));
 
+    $finder = new RepositoryGroupFinder($db, $triggerable_id);
     $commit_set_list = array();
     foreach ($revision_set_list as $revision_set) {
-        $commit_set = array();
-
         if (!count($revision_set))
             exit_with_error('InvalidRevisionSets', array('revisionSets' => $revision_set_list));
 
+        $commit_set = array();
+        $repository_list = array();
+
         foreach ($revision_set as $repository_id => $revision) {
             if (!is_numeric($repository_id))
                 exit_with_error('InvalidRepository', array('repository' => $repository_id));
@@ -91,39 +96,53 @@ function commit_sets_from_revision_sets($db, $revision_set_list)
             if (!$commit)
                 exit_with_error('RevisionNotFound', array('repository' => $repository_id, 'revision' => $revision));
             array_push($commit_set, $commit['commit_id']);
+            array_push($repository_list, $repository_id);
         }
-        array_push($commit_set_list, $commit_set);
+
+        $repository_group_id = $finder->find_by_repositories($repository_list);
+        if (!$repository_group_id)
+            exit_with_error('NoMatchingRepositoryGroup', array('repositoris' => $repository_list));
+
+        array_push($commit_set_list, array('repository_group' => $repository_group_id, 'set' => $commit_set));
     }
 
     return $commit_set_list;
 }
 
-function ensure_commit_sets($db, $commit_sets_info) {
+function ensure_commit_sets($db, $triggerable_id, $commit_sets_info) {
     $repository_name_to_id = array();
     foreach ($db->select_rows('repositories', 'repository', array('owner' => NULL)) as $row)
         $repository_name_to_id[$row['repository_name']] = $row['repository_id'];
 
     $commit_sets = array();
+    $repository_list = array();
     foreach ($commit_sets_info as $repository_name => $revisions) {
         $repository_id = array_get($repository_name_to_id, $repository_name);
         if (!$repository_id)
             exit_with_error('RepositoryNotFound', array('name' => $repository_name));
+        array_push($repository_list, $repository_id);
 
         foreach ($revisions as $i => $revision) {
             $commit = $db->select_first_row('commits', 'commit', array('repository' => $repository_id, 'revision' => $revision));
             if (!$commit)
                 exit_with_error('RevisionNotFound', array('repository' => $repository_name, 'revision' => $revision));
-            array_set_default($commit_sets, $i, array());
-            array_push($commit_sets[$i], $commit['commit_id']);
+            array_set_default($commit_sets, $i, array('set' => array()));
+            array_push($commit_sets[$i]['set'], $commit['commit_id']);
         }
     }
 
+    $finder = new RepositoryGroupFinder($db, $triggerable_id);
+    $repository_group_id = $finder->find_by_repositories($repository_list);
+    if (!$repository_group_id)
+        exit_with_error('NoMatchingRepositoryGroup', array('repositoris' => $repository_list));
+
     if (count($commit_sets) < 2)
         exit_with_error('InvalidCommitSets', array('commitSets' => $commit_sets_info));
 
-    $commit_count_per_set = count($commit_sets[0]);
-    foreach ($commit_sets as $commits) {
-        if ($commit_count_per_set != count($commits))
+    $commit_count_per_set = count($commit_sets[0]['set']);
+    foreach ($commit_sets as &$commits) {
+        $commits['repository_group'] = $repository_group_id;
+        if ($commit_count_per_set != count($commits['set']))
             exit_with_error('InvalidCommitSets', array('commitSets' => $commit_sets));
     }
 
index 4f30945..299012d 100644 (file)
@@ -6,12 +6,14 @@ class BuildRequest extends DataModelObject {
     {
         super(id, object);
         this._triggerable = object.triggerable;
+        console.assert(!object.repositoryGroup || object.repositoryGroup instanceof TriggerableRepositoryGroup);
         this._analysisTaskId = object.task;
         this._testGroupId = object.testGroupId;
         console.assert(!object.testGroup || object.testGroup instanceof TestGroup);
         this._testGroup = object.testGroup;
         if (this._testGroup)
             this._testGroup.addBuildRequest(this);
+        this._repositoryGroup = object.repositoryGroup;
         console.assert(object.platform instanceof Platform);
         this._platform = object.platform;
         console.assert(object.test instanceof Test);
@@ -36,9 +38,11 @@ class BuildRequest extends DataModelObject {
         this._buildId = object.build;
     }
 
+    triggerable() { return this._triggerable; }
     analysisTaskId() { return this._analysisTaskId; }
     testGroupId() { return this._testGroupId; }
     testGroup() { return this._testGroup; }
+    repositoryGroup() { return this._repositoryGroup; }
     platform() { return this._platform; }
     test() { return this._test; }
     order() { return +this._order; }
@@ -129,6 +133,8 @@ class BuildRequest extends DataModelObject {
         });
 
         return data['buildRequests'].map(function (rawData) {
+            rawData.triggerable = Triggerable.findById(rawData.triggerable);
+            rawData.repositoryGroup = TriggerableRepositoryGroup.findById(rawData.repositoryGroup);
             rawData.platform = Platform.findById(rawData.platform);
             rawData.test = Test.findById(rawData.test);
             rawData.testGroupId = rawData.testGroup;
index 8c2acae..7e2e6a3 100644 (file)
@@ -8,6 +8,15 @@ class Repository extends LabeledObject {
         this._blameUrl = object.blameUrl;
         this._hasReportedCommits = object.hasReportedCommits;
         this._owner = object.owner;
+
+        if (!object.owner)
+            this.ensureNamedStaticMap('topLevelName')[this.name()] = this;
+    }
+
+    static findTopLevelByName(name)
+    {
+        const map = this.namedStaticMap('topLevelName');
+        return map ? map[name] : null;
     }
 
     hasUrlForRevision() { return !!this._url; }
index 53d5090..75331e3 100644 (file)
@@ -19,8 +19,8 @@ class Triggerable extends LabeledObject {
         }
     }
 
+    name() { return this._name; }
     isDisabled() { return this._isDisabled; }
-    acceptedRepositories() { return this._acceptedRepositories; }
     repositoryGroups() { return this._repositoryGroups; }
 
     acceptsTest(test) { return this._acceptedTests.has(test); }
@@ -46,7 +46,19 @@ class TriggerableRepositoryGroup extends LabeledObject {
         super(id, object);
         this._description = object.description;
         this._acceptsCustomRoots = !!object.acceptsCustomRoots;
-        this._repositories = object.repositories;
+        this._repositories = Repository.sortByName(object.repositories);
+    }
+
+    accepts(commitSet)
+    {
+        const commitSetRepositories = Repository.sortByName(commitSet.repositories());
+        if (this._repositories.length != commitSetRepositories.length)
+            return false;
+        for (let i = 0; i < this._repositories.length; i++) {
+            if (this._repositories[i] != commitSetRepositories[i])
+                return false;
+        }
+        return true;
     }
 
     description() { return this._description || this.name(); }
index fdf0c87..d32fd55 100644 (file)
@@ -95,7 +95,7 @@ describe('/api/build-requests', function () {
 
             assert.equal(content['commits'].length, 3);
             assert.equal(content['commits'][0].id, 87832);
-            assert.equal(content['commits'][0].repository, 'OS X');
+            assert.equal(content['commits'][0].repository, 'macOS');
             assert.equal(content['commits'][0].revision, '10.11 15A284');
             assert.equal(content['commits'][1].id, 93116);
             assert.equal(content['commits'][1].repository, 'WebKit');
@@ -194,7 +194,7 @@ describe('/api/build-requests', function () {
             assert.equal(buildRequests[3].statusLabel(), 'Waiting');
 
             let osx = Repository.findById(9);
-            assert.equal(osx.name(), 'OS X');
+            assert.equal(osx.name(), 'macOS');
 
             let webkit = Repository.findById(11);
             assert.equal(webkit.name(), 'WebKit');
@@ -325,6 +325,42 @@ describe('/api/build-requests', function () {
         });
     });
 
+    it('should specify the repository group for build requests if set', () => {
+        const db = TestServer.database();
+        let groups;
+        return MockData.addMockData(db).then(() => {
+            return MockData.addMockTestGroupWithGitWebKit(db);
+        }).then(() => {
+            return Manifest.fetch();
+        }).then(() => {
+            const triggerable = Triggerable.all().find((triggerable) => triggerable.name() == 'build-webkit');
+            assert.equal(triggerable.repositoryGroups().length, 2);
+            groups = TriggerableRepositoryGroup.sortByName(triggerable.repositoryGroups());
+            assert.equal(groups[0].name(), 'webkit-git');
+            assert.equal(groups[1].name(), 'webkit-svn');
+            return BuildRequest.fetchForTriggerable('build-webkit');
+        }).then((buildRequests) => {
+            assert.equal(buildRequests.length, 8);
+            assert.equal(buildRequests[0].id(), 700);
+            assert.equal(buildRequests[0].repositoryGroup(), groups[1]);
+            assert.equal(buildRequests[1].id(), 701);
+            assert.equal(buildRequests[1].repositoryGroup(), groups[1]);
+            assert.equal(buildRequests[2].id(), 702);
+            assert.equal(buildRequests[2].repositoryGroup(), groups[1]);
+            assert.equal(buildRequests[3].id(), 703);
+            assert.equal(buildRequests[3].repositoryGroup(), groups[1]);
+
+            assert.equal(buildRequests[4].id(), 1700);
+            assert.equal(buildRequests[4].repositoryGroup(), groups[0]);
+            assert.equal(buildRequests[5].id(), 1701);
+            assert.equal(buildRequests[5].repositoryGroup(), groups[0]);
+            assert.equal(buildRequests[6].id(), 1702);
+            assert.equal(buildRequests[6].repositoryGroup(), groups[0]);
+            assert.equal(buildRequests[7].id(), 1703);
+            assert.equal(buildRequests[7].repositoryGroup(), groups[0]);
+        });
+    });
+
     it('should place build requests created by user before automatically created ones', () => {
         let db = TestServer.database();
         return Promise.all([MockData.addMockData(db), MockData.addAnotherMockTestGroup(db, null, 'rniwa')]).then(() => {
index 54acb16..f77123c 100644 (file)
@@ -62,7 +62,7 @@ describe('/api/manifest', function () {
             db.insert('bug_trackers', bugzillaData),
             db.insert('bug_trackers', radarData),
             db.insert('repositories', {id: 11, name: 'WebKit', url: 'https://trac.webkit.org/$1'}),
-            db.insert('repositories', {id: 9, name: 'OS X'}),
+            db.insert('repositories', {id: 9, name: 'macOS'}),
             db.insert('repositories', {id: 22, name: 'iOS'}),
             db.insert('tracker_repositories', {tracker: bugzillaData.id, repository: 11}),
             db.insert('tracker_repositories', {tracker: radarData.id, repository: 9}),
@@ -77,9 +77,9 @@ describe('/api/manifest', function () {
             assert.equal(webkit.name(), 'WebKit');
             assert.equal(webkit.urlForRevision(123), 'https://trac.webkit.org/123');
 
-            let osx = Repository.findById(9);
-            assert(osx);
-            assert.equal(osx.name(), 'OS X');
+            let macos = Repository.findById(9);
+            assert(macos);
+            assert.equal(macos.name(), 'macOS');
 
             let ios = Repository.findById(22);
             assert(ios);
@@ -95,7 +95,7 @@ describe('/api/manifest', function () {
             tracker = BugTracker.findById(2);
             assert(tracker);
             assert.equal(tracker.name(), 'Radar');
-            assert.deepEqual(Repository.sortByName(tracker.repositories()), [osx, ios]);
+            assert.deepEqual(Repository.sortByName(tracker.repositories()), [ios, macos]);
         });
     });
 
@@ -264,7 +264,7 @@ describe('/api/manifest', function () {
         let db = TestServer.database();
         return Promise.all([
             db.insert('repositories', {id: 11, name: 'WebKit', url: 'https://trac.webkit.org/$1'}),
-            db.insert('repositories', {id: 9, name: 'OS X'}),
+            db.insert('repositories', {id: 9, name: 'macOS'}),
             db.insert('repositories', {id: 101, name: 'WebKit', owner: 9, url: 'https://trac.webkit.org/$1'}),
             db.insert('build_triggerables', {id: 200, name: 'build.webkit.org'}),
             db.insert('build_triggerables', {id: 201, name: 'ios-build.webkit.org'}),
@@ -314,8 +314,8 @@ describe('/api/manifest', function () {
             assert.equal(osWebkit1.owner(), 9);
             assert.equal(osWebkit1.urlForRevision(123), 'https://trac.webkit.org/123');
 
-            const osx = Repository.findById(9);
-            assert.equal(osx.name(), 'OS X');
+            const macos = Repository.findById(9);
+            assert.equal(macos.name(), 'macOS');
 
             const someTest = Test.findById(1);
             assert.equal(someTest.name(), 'SomeTest');
@@ -337,38 +337,54 @@ describe('/api/manifest', function () {
 
             assert.equal(Triggerable.all().length, 3);
 
-            const osxTriggerable = Triggerable.findByTestConfiguration(someTest, mavericks);
-            assert.equal(osxTriggerable.name(), 'build.webkit.org');
-            assert.deepEqual(osxTriggerable.acceptedRepositories(), [webkit]);
+            const macosTriggerable = Triggerable.findByTestConfiguration(someTest, mavericks);
+            assert.equal(macosTriggerable.name(), 'build.webkit.org');
 
-            assert.equal(Triggerable.findByTestConfiguration(someOtherTest, mavericks), osxTriggerable);
-            assert.equal(Triggerable.findByTestConfiguration(childTest, mavericks), osxTriggerable);
+            assert.equal(Triggerable.findByTestConfiguration(someOtherTest, mavericks), macosTriggerable);
+            assert.equal(Triggerable.findByTestConfiguration(childTest, mavericks), macosTriggerable);
 
             const iosTriggerable = Triggerable.findByTestConfiguration(someOtherTest, ios9iphone5s);
-            assert.notEqual(iosTriggerable, osxTriggerable);
+            assert.notEqual(iosTriggerable, macosTriggerable);
             assert.equal(iosTriggerable.name(), 'ios-build.webkit.org');
-            assert.deepEqual(iosTriggerable.acceptedRepositories(), [webkit]);
 
             assert.equal(Triggerable.findByTestConfiguration(someOtherTest, ios9iphone5s), iosTriggerable);
             assert.equal(Triggerable.findByTestConfiguration(childTest, ios9iphone5s), iosTriggerable);
 
             const macTriggerable = Triggerable.findByTestConfiguration(someTest, sierra);
             assert.equal(macTriggerable.name(), 'mac-build.webkit.org');
-            assert.deepEqual(Repository.sortByName(macTriggerable.acceptedRepositories()), [osx, webkit]);
             assert(macTriggerable.acceptsTest(someTest));
 
             const groups = macTriggerable.repositoryGroups();
             assert.deepEqual(groups.length, 2);
             TriggerableRepositoryGroup.sortByName(groups);
 
+            const emptyCustomSet = new CustomCommitSet;
+
+            const customSetWithOSX = new CustomCommitSet;
+            customSetWithOSX.setRevisionForRepository(macos, '10.11 15A284');
+
+            const cusomSetWithOSXAndWebKit = new CustomCommitSet;
+            cusomSetWithOSXAndWebKit.setRevisionForRepository(webkit, '191622');
+            cusomSetWithOSXAndWebKit.setRevisionForRepository(macos, '10.11 15A284');
+
+            const cusomSetWithWebKit = new CustomCommitSet;
+            cusomSetWithWebKit.setRevisionForRepository(webkit, '191622');
+
             assert.equal(groups[0].name(), 'system-and-roots');
             assert.equal(groups[0].acceptsCustomRoots(), true);
-            assert.deepEqual(Repository.sortByName(groups[0].repositories()), [osx]);
+            assert.deepEqual(Repository.sortByName(groups[0].repositories()), [macos]);
+            assert.equal(groups[0].accepts(emptyCustomSet), false);
+            assert.equal(groups[0].accepts(customSetWithOSX), true);
+            assert.equal(groups[0].accepts(cusomSetWithOSXAndWebKit), false);
+            assert.equal(groups[0].accepts(cusomSetWithWebKit), false);
 
             assert.equal(groups[1].name(), 'system-and-webkit');
             assert.equal(groups[1].acceptsCustomRoots(), false);
-            assert.deepEqual(Repository.sortByName(groups[1].repositories()), [osx, webkit]);
-
+            assert.deepEqual(Repository.sortByName(groups[1].repositories()), [webkit, macos]);
+            assert.equal(groups[1].accepts(emptyCustomSet), false);
+            assert.equal(groups[1].accepts(customSetWithOSX), false);
+            assert.equal(groups[1].accepts(cusomSetWithOSXAndWebKit), true);
+            assert.equal(groups[1].accepts(cusomSetWithWebKit), false);
         });
     });
 
index 4174004..1a2eef3 100644 (file)
@@ -21,7 +21,7 @@ describe("/api/report", function () {
             "platform": "Mountain Lion",
             "tests": {},
             "revisions": {
-                "OS X": {
+                "macOS": {
                     "revision": "10.8.2 12C60"
                 },
                 "WebKit": {
@@ -43,7 +43,7 @@ describe("/api/report", function () {
             "platform": "Mountain Lion",
             "tests": {},
             "revisions": {
-                "OS X": {
+                "macOS": {
                     "revision": "10.8.2 12C60"
                 },
                 "WebKit": {
@@ -232,7 +232,7 @@ describe("/api/report", function () {
             const commits = result[1];
             const buildCommitsRelations = result[2];
             assert.equal(repositories.length, 2);
-            assert.deepEqual(repositories.map((row) => { return row['name']; }).sort(), ['OS X', 'WebKit']);
+            assert.deepEqual(repositories.map((row) => row['name']).sort(), ['WebKit', 'macOS']);
 
             assert.equal(commits.length, 2);
             assert.equal(buildCommitsRelations.length, 2);
@@ -248,7 +248,7 @@ describe("/api/report", function () {
             for (let commit of commits)
                 repositoryNameToRevisionRow[repositoryIdToName[commit['repository']]] = commit;
 
-            assert.equal(repositoryNameToRevisionRow['OS X']['revision'], '10.8.2 12C60');
+            assert.equal(repositoryNameToRevisionRow['macOS']['revision'], '10.8.2 12C60');
             assert.equal(repositoryNameToRevisionRow['WebKit']['revision'], '141977');
             assert.equal(repositoryNameToRevisionRow['WebKit']['time'].toString(),
                 new Date('2013-02-06 08:55:20.9').toString());
@@ -368,7 +368,7 @@ describe("/api/report", function () {
             }
         },
         "revisions": {
-            "OS X": {
+            "macOS": {
                 "revision": "10.8.2 12C60"
             },
             "WebKit": {
index 5f209f1..af9a964 100644 (file)
@@ -68,7 +68,7 @@ describe('/api/update-triggerable/', function () {
             return Promise.all([
                 addSlaveForReport(emptyUpdate),
                 db.insert('triggerable_configurations',
-                    {'triggerable': 1 /* build-webkit */, 'test': MockData.someTestId(), 'platform': MockData.somePlatformId()})
+                    {'triggerable': 1000 /* build-webkit */, 'test': MockData.someTestId(), 'platform': MockData.somePlatformId()})
             ]);
         }).then(() => {
             return TestServer.remoteAPI().postJSON('/api/update-triggerable/', emptyUpdate);
@@ -112,4 +112,291 @@ describe('/api/update-triggerable/', function () {
         });
     });
 
+    function updateWithOSXRepositoryGroup()
+    {
+        return {
+            'slaveName': 'someSlave',
+            'slavePassword': 'somePassword',
+            'triggerable': 'empty-triggerable',
+            'configurations': [
+                {test: MockData.someTestId(), platform: MockData.somePlatformId()}
+            ],
+            'repositoryGroups': [
+                {name: 'system-only', repositories: [MockData.macosRepositoryId()]},
+            ]
+        };
+    }
+
+    it('should reject when repositoryGroups is not an array', () => {
+        const update = updateWithOSXRepositoryGroup();
+        update.repositoryGroups = 1;
+        return MockData.addEmptyTriggerable(TestServer.database()).then(() => {
+            return addSlaveForReport(update);
+        }).then(() => {
+            return TestServer.remoteAPI().postJSON('/api/update-triggerable/', update);
+        }).then((response) => {
+            assert.equal(response['status'], 'InvalidRepositoryGroups');
+        });
+    });
+
+    it('should reject when the name of a repository group is not specified', () => {
+        const update = updateWithOSXRepositoryGroup();
+        delete update.repositoryGroups[0].name;
+        return MockData.addEmptyTriggerable(TestServer.database()).then(() => {
+            return addSlaveForReport(update);
+        }).then(() => {
+            return TestServer.remoteAPI().postJSON('/api/update-triggerable/', update);
+        }).then((response) => {
+            assert.equal(response['status'], 'InvalidRepositoryGroup');
+        });
+    });
+
+    it('should reject when the repository list is not specified for a repository group', () => {
+        const update = updateWithOSXRepositoryGroup();
+        delete update.repositoryGroups[0].repositories;
+        return MockData.addEmptyTriggerable(TestServer.database()).then(() => {
+            return addSlaveForReport(update);
+        }).then(() => {
+            return TestServer.remoteAPI().postJSON('/api/update-triggerable/', update);
+        }).then((response) => {
+            assert.equal(response['status'], 'InvalidRepositoryGroup');
+        });
+    });
+
+    it('should reject when the repository list of a repository group is not an array', () => {
+        const update = updateWithOSXRepositoryGroup();
+        update.repositoryGroups[0].repositories = 'hi';
+        return MockData.addEmptyTriggerable(TestServer.database()).then(() => {
+            return addSlaveForReport(update);
+        }).then(() => {
+            return TestServer.remoteAPI().postJSON('/api/update-triggerable/', update);
+        }).then((response) => {
+            assert.equal(response['status'], 'InvalidRepositoryGroup');
+        });
+    });
+
+    it('should reject when a repository group contains an invalid repository id', () => {
+        const update = updateWithOSXRepositoryGroup();
+        update.repositoryGroups[0].repositories[0] = 999;
+        return MockData.addEmptyTriggerable(TestServer.database()).then(() => {
+            return addSlaveForReport(update);
+        }).then(() => {
+            return TestServer.remoteAPI().postJSON('/api/update-triggerable/', update);
+        }).then((response) => {
+            assert.equal(response['status'], 'InvalidRepository');
+        });
+    });
+
+    it('should reject when a repository group contains a duplicate repository id', () => {
+        const update = updateWithOSXRepositoryGroup();
+        const group = update.repositoryGroups[0];
+        group.repositories.push(group.repositories[0]);
+        return MockData.addEmptyTriggerable(TestServer.database()).then(() => {
+            return addSlaveForReport(update);
+        }).then(() => {
+            return TestServer.remoteAPI().postJSON('/api/update-triggerable/', update);
+        }).then((response) => {
+            assert.equal(response['status'], 'InvalidRepository');
+        });
+    });
+
+    it('should add a new repository group when there are none', () => {
+        const db = TestServer.database();
+        return MockData.addEmptyTriggerable(db).then(() => {
+            return addSlaveForReport(updateWithOSXRepositoryGroup());
+        }).then(() => {
+            return TestServer.remoteAPI().postJSON('/api/update-triggerable/', updateWithOSXRepositoryGroup());
+        }).then((response) => {
+            assert.equal(response['status'], 'OK');
+            return Promise.all([db.selectAll('triggerable_configurations', 'test'), db.selectAll('triggerable_repository_groups')]);
+        }).then((result) => {
+            const [configurations, repositoryGroups] = result;
+
+            assert.equal(configurations.length, 1);
+            assert.equal(configurations[0]['test'], MockData.someTestId());
+            assert.equal(configurations[0]['platform'], MockData.somePlatformId());
+
+            assert.equal(repositoryGroups.length, 1);
+            assert.equal(repositoryGroups[0]['name'], 'system-only');
+            assert.equal(repositoryGroups[0]['triggerable'], MockData.emptyTriggeragbleId());
+        });
+    });
+
+    it('should not add a duplicate repository group when there is a group of the same name', () => {
+        const db = TestServer.database();
+        let initialResult;
+        return MockData.addEmptyTriggerable(db).then(() => {
+            return addSlaveForReport(updateWithOSXRepositoryGroup());
+        }).then(() => {
+            return TestServer.remoteAPI().postJSONWithStatus('/api/update-triggerable/', updateWithOSXRepositoryGroup());
+        }).then((response) => {
+            return Promise.all([db.selectAll('triggerable_configurations', 'test'), db.selectAll('triggerable_repository_groups')]);
+        }).then((result) => {
+            initialResult = result;
+            return TestServer.remoteAPI().postJSONWithStatus('/api/update-triggerable/', updateWithOSXRepositoryGroup());
+        }).then(() => {
+            return Promise.all([db.selectAll('triggerable_configurations', 'test'), db.selectAll('triggerable_repository_groups')]);
+        }).then((result) => {
+            const [initialConfigurations, initialRepositoryGroups] = initialResult;
+            const [configurations, repositoryGroups] = result;
+            assert.deepEqual(configurations, initialConfigurations);
+            assert.deepEqual(repositoryGroups, initialRepositoryGroups);
+        })
+    });
+
+    it('should not add a duplicate repository group when there is a group of the same name', () => {
+        const db = TestServer.database();
+        let initialResult;
+        return MockData.addEmptyTriggerable(db).then(() => {
+            return addSlaveForReport(updateWithOSXRepositoryGroup());
+        }).then(() => {
+            return TestServer.remoteAPI().postJSONWithStatus('/api/update-triggerable/', updateWithOSXRepositoryGroup());
+        }).then((response) => {
+            return Promise.all([db.selectAll('triggerable_configurations', 'test'), db.selectAll('triggerable_repository_groups')]);
+        }).then((result) => {
+            initialResult = result;
+            return TestServer.remoteAPI().postJSONWithStatus('/api/update-triggerable/', updateWithOSXRepositoryGroup());
+        }).then(() => {
+            return Promise.all([db.selectAll('triggerable_configurations', 'test'), db.selectAll('triggerable_repository_groups')]);
+        }).then((result) => {
+            const [initialConfigurations, initialRepositoryGroups] = initialResult;
+            const [configurations, repositoryGroups] = result;
+            assert.deepEqual(configurations, initialConfigurations);
+            assert.deepEqual(repositoryGroups, initialRepositoryGroups);
+        })
+    });
+
+    it('should update the description of a repository group when the name matches', () => {
+        const db = TestServer.database();
+        const initialUpdate = updateWithOSXRepositoryGroup();
+        const secondUpdate = updateWithOSXRepositoryGroup();
+        secondUpdate.repositoryGroups[0].description = 'this group is awesome';
+        return MockData.addEmptyTriggerable(db).then(() => {
+            return addSlaveForReport(initialUpdate);
+        }).then(() => {
+            return TestServer.remoteAPI().postJSONWithStatus('/api/update-triggerable/', initialUpdate);
+        }).then((response) => db.selectAll('triggerable_repository_groups')).then((repositoryGroups) => {
+            assert.equal(repositoryGroups.length, 1);
+            assert.equal(repositoryGroups[0]['name'], 'system-only');
+            assert.equal(repositoryGroups[0]['description'], null);
+            return TestServer.remoteAPI().postJSONWithStatus('/api/update-triggerable/', secondUpdate);
+        }).then(() => db.selectAll('triggerable_repository_groups')).then((repositoryGroups) => {
+            assert.equal(repositoryGroups.length, 1);
+            assert.equal(repositoryGroups[0]['name'], 'system-only');
+            assert.equal(repositoryGroups[0]['description'], 'this group is awesome');
+        });
+    });
+
+    function updateWithMacWebKitRepositoryGroups()
+    {
+        return {
+            'slaveName': 'someSlave',
+            'slavePassword': 'somePassword',
+            'triggerable': 'empty-triggerable',
+            'configurations': [
+                {test: MockData.someTestId(), platform: MockData.somePlatformId()}
+            ],
+            'repositoryGroups': [
+                {name: 'system-only', repositories: [MockData.macosRepositoryId()]},
+                {name: 'system-and-webkit', repositories: [MockData.webkitRepositoryId(), MockData.macosRepositoryId()]},
+            ]
+        };
+    }
+
+    function mapRepositoriesByGroup(repositories)
+    {
+        const map = {};
+        for (const row of repositories) {
+            const groupId = row['group'];
+            if (!(groupId in map))
+                map[groupId] = [];
+            map[groupId].push(row['repository']);
+        }
+        return map;
+    }
+
+    it('should replace a repository when the repository group name matches', () => {
+        const db = TestServer.database();
+        const initialUpdate = updateWithMacWebKitRepositoryGroups();
+        const secondUpdate = updateWithMacWebKitRepositoryGroups();
+        let initialGroups;
+        secondUpdate.repositoryGroups[1].repositories[0] = MockData.gitWebkitRepositoryId();
+        return MockData.addEmptyTriggerable(db).then(() => {
+            return addSlaveForReport(initialUpdate);
+        }).then(() => {
+            return TestServer.remoteAPI().postJSONWithStatus('/api/update-triggerable/', initialUpdate);
+        }).then((response) => {
+            return Promise.all([db.selectAll('triggerable_repository_groups', 'name'), db.selectAll('triggerable_repositories', 'repository')]);
+        }).then((result) => {
+            const [repositoryGroups, repositories] = result;
+            assert.equal(repositoryGroups.length, 2);
+            assert.equal(repositoryGroups[0]['name'], 'system-and-webkit');
+            assert.equal(repositoryGroups[0]['triggerable'], MockData.emptyTriggeragbleId());
+            assert.equal(repositoryGroups[1]['name'], 'system-only');
+            assert.equal(repositoryGroups[1]['triggerable'], MockData.emptyTriggeragbleId());
+            initialGroups = repositoryGroups;
+
+            const repositoriesByGroup = mapRepositoriesByGroup(repositories);
+            assert.equal(Object.keys(repositoriesByGroup).length, 2);
+            assert.deepEqual(repositoriesByGroup[repositoryGroups[0]['id']], [MockData.macosRepositoryId(), MockData.webkitRepositoryId()]);
+            assert.deepEqual(repositoriesByGroup[repositoryGroups[1]['id']], [MockData.macosRepositoryId()]);
+
+            return TestServer.remoteAPI().postJSONWithStatus('/api/update-triggerable/', secondUpdate);
+        }).then(() => {
+            return Promise.all([db.selectAll('triggerable_repository_groups', 'name'), db.selectAll('triggerable_repositories', 'repository')]);
+        }).then((result) => {
+            const [repositoryGroups, repositories] = result;
+            assert.deepEqual(repositoryGroups, initialGroups);
+
+            const repositoriesByGroup = mapRepositoriesByGroup(repositories);
+            assert.equal(Object.keys(repositoriesByGroup).length, 2);
+            assert.deepEqual(repositoriesByGroup[initialGroups[0]['id']], [MockData.macosRepositoryId(), MockData.gitWebkitRepositoryId()]);
+            assert.deepEqual(repositoriesByGroup[initialGroups[1]['id']], [MockData.macosRepositoryId()]);
+        });
+    });
+
+    it('should replace a repository when the list of repositories matches', () => {
+        const db = TestServer.database();
+        const initialUpdate = updateWithMacWebKitRepositoryGroups();
+        const secondUpdate = updateWithMacWebKitRepositoryGroups();
+        let initialGroups;
+        let initialRepositories;
+        secondUpdate.repositoryGroups[0].name = 'mac-only';
+        return MockData.addEmptyTriggerable(db).then(() => {
+            return addSlaveForReport(initialUpdate);
+        }).then(() => {
+            return TestServer.remoteAPI().postJSONWithStatus('/api/update-triggerable/', initialUpdate);
+        }).then((response) => {
+            return Promise.all([db.selectAll('triggerable_repository_groups', 'name'), db.selectAll('triggerable_repositories', 'repository')]);
+        }).then((result) => {
+            const [repositoryGroups, repositories] = result;
+            assert.equal(repositoryGroups.length, 2);
+            assert.equal(repositoryGroups[0]['name'], 'system-and-webkit');
+            assert.equal(repositoryGroups[0]['triggerable'], MockData.emptyTriggeragbleId());
+            assert.equal(repositoryGroups[1]['name'], 'system-only');
+            assert.equal(repositoryGroups[1]['triggerable'], MockData.emptyTriggeragbleId());
+            initialGroups = repositoryGroups;
+
+            const repositoriesByGroup = mapRepositoriesByGroup(repositories);
+            assert.equal(Object.keys(repositoriesByGroup).length, 2);
+            assert.deepEqual(repositoriesByGroup[repositoryGroups[0]['id']], [MockData.macosRepositoryId(), MockData.webkitRepositoryId()]);
+            assert.deepEqual(repositoriesByGroup[repositoryGroups[1]['id']], [MockData.macosRepositoryId()]);
+            initialRepositories = repositories;
+
+            return TestServer.remoteAPI().postJSONWithStatus('/api/update-triggerable/', secondUpdate);
+        }).then(() => {
+            return Promise.all([db.selectAll('triggerable_repository_groups', 'name'), db.selectAll('triggerable_repositories', 'repository')]);
+        }).then((result) => {
+            const [repositoryGroups, repositories] = result;
+
+            assert.equal(repositoryGroups.length, 2);
+            assert.equal(repositoryGroups[0]['name'], 'mac-only');
+            assert.equal(repositoryGroups[0]['triggerable'], initialGroups[1]['triggerable']);
+            assert.equal(repositoryGroups[1]['name'], 'system-and-webkit');
+            assert.equal(repositoryGroups[1]['triggerable'], initialGroups[0]['triggerable']);
+
+            assert.deepEqual(repositories, initialRepositories);
+        });
+    });
+
 });
index 41486f3..a22eaf6 100644 (file)
@@ -100,6 +100,10 @@ function addTriggerableAndCreateTask(name)
         'configurations': [
             {test: MockData.someTestId(), platform: MockData.somePlatformId()}
         ],
+        'repositoryGroups': [
+            {name: 'webkit-only', repositories: [MockData.webkitRepositoryId()]},
+            {name: 'system-and-webkit', repositories: [MockData.macosRepositoryId(), MockData.webkitRepositoryId()]},
+        ]
     };
     return MockData.addMockData(TestServer.database()).then(() => {
         return addSlaveForReport(report);
@@ -281,7 +285,7 @@ describe('/privileged-api/create-test-group', function () {
     it('should create a test group from commitSets with the repetition count of one when repetitionCount is omitted', () => {
         return addTriggerableAndCreateTask('some task').then((taskId) => {
             let insertedGroupId;
-            return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, commitSets: {'WebKit': ['191622', '191623']}}).then((content) => {
+            return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, commitSets: {'macOS': ['15A284', '15A284'], 'WebKit': ['191622', '191623']}}).then((content) => {
                 insertedGroupId = content['testGroupId'];
                 return TestGroup.fetchByTask(taskId);
             }).then((testGroups) => {
@@ -291,18 +295,31 @@ describe('/privileged-api/create-test-group', function () {
                 assert.equal(group.repetitionCount(), 1);
                 const requests = group.buildRequests();
                 assert.equal(requests.length, 2);
-                const webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
-                assert.deepEqual(requests[0].commitSet().repositories(), [webkit]);
-                assert.deepEqual(requests[1].commitSet().repositories(), [webkit]);
-                assert.equal(requests[0].commitSet().revisionForRepository(webkit), '191622');
-                assert.equal(requests[1].commitSet().revisionForRepository(webkit), '191623');
+
+                const macos = Repository.findById(MockData.macosRepositoryId());
+                const webkit = Repository.findById(MockData.webkitRepositoryId());
+                const set0 = requests[0].commitSet();
+                const set1 = requests[1].commitSet();
+                assert.deepEqual(Repository.sortByNamePreferringOnesWithURL(set0.repositories()), [webkit, macos]);
+                assert.deepEqual(Repository.sortByNamePreferringOnesWithURL(set1.repositories()), [webkit, macos]);
+                assert.equal(set0.revisionForRepository(macos), '15A284');
+                assert.equal(set0.revisionForRepository(webkit), '191622');
+                assert.equal(set1.revisionForRepository(macos), '15A284');
+                assert.equal(set1.revisionForRepository(webkit), '191623');
+
+                const repositoryGroup = requests[0].repositoryGroup();
+                assert.equal(repositoryGroup.name(), 'system-and-webkit');
+                assert.equal(requests[1].repositoryGroup(), repositoryGroup);
+                assert(repositoryGroup.accepts(set0));
+                assert(repositoryGroup.accepts(set1));
             });
         });
     });
 
     it('should create a test group from revisionSets with the repetition count of one when repetitionCount is omitted', () => {
+        let webkit;
         return addTriggerableAndCreateTask('some task').then((taskId) => {
-            const webkit = Repository.all().find((repository) => repository.name() == 'WebKit');
+            const webkit = Repository.findById(MockData.webkitRepositoryId());
             const params = {name: 'test', task: taskId, revisionSets: [{[webkit.id()]: '191622'}, {[webkit.id()]: '191623'}]};
             let insertedGroupId;
             return PrivilegedAPI.sendRequest('create-test-group', params).then((content) => {
@@ -315,11 +332,19 @@ describe('/privileged-api/create-test-group', function () {
                 assert.equal(group.repetitionCount(), 1);
                 const requests = group.buildRequests();
                 assert.equal(requests.length, 2);
-                const webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
-                assert.deepEqual(requests[0].commitSet().repositories(), [webkit]);
-                assert.deepEqual(requests[1].commitSet().repositories(), [webkit]);
-                assert.equal(requests[0].commitSet().revisionForRepository(webkit), '191622');
-                assert.equal(requests[1].commitSet().revisionForRepository(webkit), '191623');
+
+                const set0 = requests[0].commitSet();
+                const set1 = requests[1].commitSet();
+                assert.deepEqual(set0.repositories(), [webkit]);
+                assert.deepEqual(set1.repositories(), [webkit]);
+                assert.equal(set0.revisionForRepository(webkit), '191622');
+                assert.equal(set1.revisionForRepository(webkit), '191623');
+
+                const repositoryGroup = requests[0].repositoryGroup();
+                assert.equal(repositoryGroup.name(), 'webkit-only');
+                assert.equal(repositoryGroup, requests[1].repositoryGroup());
+                assert(repositoryGroup.accepts(set0));
+                assert(repositoryGroup.accepts(set1));
             });
         });
     });
@@ -340,17 +365,70 @@ describe('/privileged-api/create-test-group', function () {
                 assert.equal(requests.length, 4);
                 const webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
                 const macos = Repository.all().filter((repository) => repository.name() == 'macOS')[0];
-                const set1 = requests[0].commitSet();
-                const set2 = requests[1].commitSet();
-                assert.equal(requests[2].commitSet(), set1);
-                assert.equal(requests[3].commitSet(), set2);
+
+                const set0 = requests[0].commitSet();
+                const set1 = requests[1].commitSet();
+                assert.equal(requests[2].commitSet(), set0);
+                assert.equal(requests[3].commitSet(), set1);
+                assert.deepEqual(Repository.sortByNamePreferringOnesWithURL(set0.repositories()), [webkit, macos]);
                 assert.deepEqual(Repository.sortByNamePreferringOnesWithURL(set1.repositories()), [webkit, macos]);
-                assert.deepEqual(Repository.sortByNamePreferringOnesWithURL(set2.repositories()), [webkit, macos]);
-                assert.equal(set1.revisionForRepository(webkit), '191622');
+                assert.equal(set0.revisionForRepository(webkit), '191622');
+                assert.equal(set0.revisionForRepository(macos), '15A284');
+                assert.equal(set1.revisionForRepository(webkit), '191623');
                 assert.equal(set1.revisionForRepository(macos), '15A284');
-                assert.equal(set2.revisionForRepository(webkit), '191623');
-                assert.equal(set2.revisionForRepository(macos), '15A284');
-                assert.equal(set1.commitForRepository(macos), set2.commitForRepository(macos));
+                assert.equal(set0.commitForRepository(macos), set1.commitForRepository(macos));
+
+                const repositoryGroup = requests[0].repositoryGroup();
+                assert.equal(repositoryGroup.name(), 'system-and-webkit');
+                assert.equal(requests[1].repositoryGroup(), repositoryGroup);
+                assert.equal(requests[2].repositoryGroup(), repositoryGroup);
+                assert.equal(requests[3].repositoryGroup(), repositoryGroup);
+                assert(repositoryGroup.accepts(set0));
+                assert(repositoryGroup.accepts(set1));
+            });
+        });
+    });
+
+    it('should create a test group using different repository groups if needed', () => {
+        let webkit;
+        let macos;
+        return addTriggerableAndCreateTask('some task').then((taskId) => {
+            webkit = Repository.findById(MockData.webkitRepositoryId());
+            macos = Repository.findById(MockData.macosRepositoryId());
+            const params = {name: 'test', task: taskId, repetitionCount: 2,
+                revisionSets: [{[macos.id()]: '15A284', [webkit.id()]: '191622'}, {[webkit.id()]: '191623'}]};
+            let insertedGroupId;
+            return PrivilegedAPI.sendRequest('create-test-group', params).then((content) => {
+                insertedGroupId = content['testGroupId'];
+                return TestGroup.fetchByTask(taskId);
+            }).then((testGroups) => {
+                assert.equal(testGroups.length, 1);
+                const group = testGroups[0];
+                assert.equal(group.id(), insertedGroupId);
+                assert.equal(group.repetitionCount(), 2);
+                const requests = group.buildRequests();
+                assert.equal(requests.length, 4);
+
+                const set0 = requests[0].commitSet();
+                const set1 = requests[1].commitSet();
+                assert.deepEqual(Repository.sortByNamePreferringOnesWithURL(set0.repositories()), [webkit, macos]);
+                assert.deepEqual(set1.repositories(), [webkit]);
+                assert.equal(set0.revisionForRepository(webkit), '191622');
+                assert.equal(set0.revisionForRepository(macos), '15A284');
+                assert.equal(set1.revisionForRepository(webkit), '191623');
+                assert.equal(set1.revisionForRepository(macos), null);
+
+                const repositoryGroup0 = requests[0].repositoryGroup();
+                assert.equal(repositoryGroup0.name(), 'system-and-webkit');
+                assert.equal(repositoryGroup0, requests[2].repositoryGroup());
+                assert(repositoryGroup0.accepts(set0));
+                assert(!repositoryGroup0.accepts(set1));
+
+                const repositoryGroup1 = requests[1].repositoryGroup();
+                assert.equal(repositoryGroup1.name(), 'webkit-only');
+                assert.equal(repositoryGroup1, requests[3].repositoryGroup());
+                assert(!repositoryGroup1.accepts(set0));
+                assert(repositoryGroup1.accepts(set1));
             });
         });
     });
index 0e24f0f..91bb396 100644 (file)
@@ -16,21 +16,29 @@ MockData = {
         Test.clearStaticMap();
         TestGroup.clearStaticMap();
         Triggerable.clearStaticMap();
+        TriggerableRepositoryGroup.clearStaticMap();
     },
+    emptyTriggeragbleId() { return 1001; },
     someTestId() { return 200; },
     somePlatformId() { return 65; },
+    macosRepositoryId() { return 9; },
+    webkitRepositoryId() { return 11; },
+    gitWebkitRepositoryId() { return 111; },
     addMockData: function (db, statusList)
     {
         if (!statusList)
             statusList = ['pending', 'pending', 'pending', 'pending'];
         return Promise.all([
-            db.insert('build_triggerables', {id: 1, name: 'build-webkit'}),
+            db.insert('build_triggerables', {id: 1000, name: 'build-webkit'}),
             db.insert('build_slaves', {id: 20, name: 'sync-slave', password_hash: crypto.createHash('sha256').update('password').digest('hex')}),
-            db.insert('repositories', {id: 9, name: 'OS X'}),
-            db.insert('repositories', {id: 11, name: 'WebKit'}),
-            db.insert('commits', {id: 87832, repository: 9, revision: '10.11 15A284'}),
-            db.insert('commits', {id: 93116, repository: 11, revision: '191622', time: (new Date(1445945816878)).toISOString()}),
-            db.insert('commits', {id: 96336, repository: 11, revision: '192736', time: (new Date(1448225325650)).toISOString()}),
+            db.insert('repositories', {id: this.macosRepositoryId(), name: 'macOS'}),
+            db.insert('repositories', {id: this.webkitRepositoryId(), name: 'WebKit'}),
+            db.insert('triggerable_repository_groups', {id: 2001, name: 'webkit-svn', triggerable: 1000}),
+            db.insert('triggerable_repositories', {repository: this.macosRepositoryId(), group: 2001}),
+            db.insert('triggerable_repositories', {repository: this.webkitRepositoryId(), group: 2001}),
+            db.insert('commits', {id: 87832, repository: this.macosRepositoryId(), revision: '10.11 15A284'}),
+            db.insert('commits', {id: 93116, repository: this.webkitRepositoryId(), revision: '191622', time: (new Date(1445945816878)).toISOString()}),
+            db.insert('commits', {id: 96336, repository: this.webkitRepositoryId(), revision: '192736', time: (new Date(1448225325650)).toISOString()}),
             db.insert('platforms', {id: MockData.somePlatformId(), name: 'some platform'}),
             db.insert('tests', {id: MockData.someTestId(), name: 'some test'}),
             db.insert('test_metrics', {id: 300, test: 200, name: 'some metric'}),
@@ -43,22 +51,59 @@ MockData = {
             db.insert('commit_set_relationships', {set: 402, commit: 96336}),
             db.insert('analysis_tasks', {id: 500, platform: 65, metric: 300, name: 'some task'}),
             db.insert('analysis_test_groups', {id: 600, task: 500, name: 'some test group'}),
-            db.insert('build_requests', {id: 700, status: statusList[0], triggerable: 1, platform: 65, test: 200, group: 600, order: 0, commit_set: 401}),
-            db.insert('build_requests', {id: 701, status: statusList[1], triggerable: 1, platform: 65, test: 200, group: 600, order: 1, commit_set: 402}),
-            db.insert('build_requests', {id: 702, status: statusList[2], triggerable: 1, platform: 65, test: 200, group: 600, order: 2, commit_set: 401}),
-            db.insert('build_requests', {id: 703, status: statusList[3], triggerable: 1, platform: 65, test: 200, group: 600, order: 3, commit_set: 402}),
+            db.insert('build_requests', {id: 700, status: statusList[0], triggerable: 1000, repository_group: 2001, platform: 65, test: 200, group: 600, order: 0, commit_set: 401}),
+            db.insert('build_requests', {id: 701, status: statusList[1], triggerable: 1000, repository_group: 2001, platform: 65, test: 200, group: 600, order: 1, commit_set: 402}),
+            db.insert('build_requests', {id: 702, status: statusList[2], triggerable: 1000, repository_group: 2001, platform: 65, test: 200, group: 600, order: 2, commit_set: 401}),
+            db.insert('build_requests', {id: 703, status: statusList[3], triggerable: 1000, repository_group: 2001, platform: 65, test: 200, group: 600, order: 3, commit_set: 402}),
+        ]);
+    },
+    addEmptyTriggerable(db)
+    {
+        return Promise.all([
+            db.insert('build_triggerables', {id: this.emptyTriggeragbleId(), name: 'empty-triggerable'}),
+            db.insert('repositories', {id: this.macosRepositoryId(), name: 'macOS'}),
+            db.insert('repositories', {id: this.webkitRepositoryId(), name: 'WebKit'}),
+            db.insert('repositories', {id: this.gitWebkitRepositoryId(), name: 'Git-WebKit'}),
+            db.insert('platforms', {id: MockData.somePlatformId(), name: 'some platform'}),
+            db.insert('tests', {id: MockData.someTestId(), name: 'some test'}),
+        ]);
+    },
+    addMockTestGroupWithGitWebKit(db)
+    {
+        return Promise.all([
+            db.insert('repositories', {id: this.gitWebkitRepositoryId(), name: 'Git-WebKit'}),
+            db.insert('triggerable_repository_groups', {id: 2002, name: 'webkit-git', triggerable: 1000}),
+            db.insert('triggerable_repositories', {repository: this.macosRepositoryId(), group: 2002}),
+            db.insert('triggerable_repositories', {repository: this.gitWebkitRepositoryId(), group: 2002}),
+            db.insert('commits', {id: 193116, repository: this.gitWebkitRepositoryId(), revision: '2ceda45d3cd63cde58d0dbf5767714e03d902e43', time: (new Date(1445945816878)).toISOString()}),
+            db.insert('commits', {id: 196336, repository: this.gitWebkitRepositoryId(), revision: '8e294365a452a89785d6536ca7f0fc8a95fa152d', time: (new Date(1448225325650)).toISOString()}),
+            db.insert('commit_sets', {id: 1401}),
+            db.insert('commit_set_relationships', {set: 1401, commit: 87832}),
+            db.insert('commit_set_relationships', {set: 1401, commit: 193116}),
+            db.insert('commit_sets', {id: 1402}),
+            db.insert('commit_set_relationships', {set: 1402, commit: 87832}),
+            db.insert('commit_set_relationships', {set: 1402, commit: 196336}),
+            db.insert('analysis_test_groups', {id: 1600, task: 500, name: 'test group with git'}),
+            db.insert('build_requests', {id: 1700, status: 'pending', triggerable: 1000, repository_group: 2002, platform: 65, test: 200, group: 1600, order: 0, commit_set: 1401}),
+            db.insert('build_requests', {id: 1701, status: 'pending', triggerable: 1000, repository_group: 2002, platform: 65, test: 200, group: 1600, order: 1, commit_set: 1402}),
+            db.insert('build_requests', {id: 1702, status: 'pending', triggerable: 1000, repository_group: 2002, platform: 65, test: 200, group: 1600, order: 2, commit_set: 1401}),
+            db.insert('build_requests', {id: 1703, status: 'pending', triggerable: 1000, repository_group: 2002, platform: 65, test: 200, group: 1600, order: 3, commit_set: 1402}),
         ]);
     },
     addAnotherMockTestGroup: function (db, statusList, author)
     {
         if (!statusList)
             statusList = ['pending', 'pending', 'pending', 'pending'];
+        const test = MockData.someTestId();
+        const triggerable = 1000;
+        const platform = 65;
+        const repository_group = 2001;
         return Promise.all([
-            db.insert('analysis_test_groups', {id: 601, task: 500, name: 'another test group', author: author}),
-            db.insert('build_requests', {id: 713, status: statusList[3], triggerable: 1, platform: 65, test: 200, group: 601, order: 3, commit_set: 402}),
-            db.insert('build_requests', {id: 710, status: statusList[0], triggerable: 1, platform: 65, test: 200, group: 601, order: 0, commit_set: 401}),
-            db.insert('build_requests', {id: 712, status: statusList[2], triggerable: 1, platform: 65, test: 200, group: 601, order: 2, commit_set: 401}),
-            db.insert('build_requests', {id: 711, status: statusList[1], triggerable: 1, platform: 65, test: 200, group: 601, order: 1, commit_set: 402}),
+            db.insert('analysis_test_groups', {id: 601, task: 500, name: 'another test group', author}),
+            db.insert('build_requests', {id: 713, status: statusList[3], triggerable, repository_group, platform, test, group: 601, order: 3, commit_set: 402}),
+            db.insert('build_requests', {id: 710, status: statusList[0], triggerable, repository_group, platform, test, group: 601, order: 0, commit_set: 401}),
+            db.insert('build_requests', {id: 712, status: statusList[2], triggerable, repository_group, platform, test, group: 601, order: 2, commit_set: 401}),
+            db.insert('build_requests', {id: 711, status: statusList[1], triggerable, repository_group, platform, test, group: 601, order: 1, commit_set: 402}),
         ]);
     },
     mockTestSyncConfigWithSingleBuilder: function ()
@@ -66,16 +111,21 @@ MockData = {
         return {
             'triggerableName': 'build-webkit',
             'lookbackCount': 2,
+            'buildRequestArgument': 'build-request-id',
+            'repositoryGroups': {
+                'webkit-svn': {
+                    'repositories': ['WebKit', 'macOS'],
+                    'properties': {
+                        'os': '<macOS>',
+                        'wk': '<WebKit>',
+                    }
+                }
+            },
             'configurations': [
                 {
                     'platform': 'some platform',
                     'test': ['some test'],
                     'builder': 'some-builder-1',
-                    'arguments': {
-                        'wk': {'root': 'WebKit'},
-                        'os': {'root': 'OS X'},
-                    },
-                    'buildRequestArgument': 'build-request-id',
                 }
             ]
         }
@@ -85,26 +135,26 @@ MockData = {
         return {
             'triggerableName': 'build-webkit',
             'lookbackCount': 2,
+            'buildRequestArgument': 'build-request-id',
+            'repositoryGroups': {
+                'webkit-svn': {
+                    'repositories': ['WebKit', 'macOS'],
+                    'properties': {
+                        'os': '<macOS>',
+                        'wk': '<WebKit>',
+                    }
+                }
+            },
             'configurations': [
                 {
                     'platform': 'some platform',
                     'test': ['some test'],
                     'builder': 'some-builder-1',
-                    'arguments': {
-                        'wk': {'root': 'WebKit'},
-                        'os': {'root': 'OS X'},
-                    },
-                    'buildRequestArgument': 'build-request-id',
                 },
                 {
                     'platform': 'some platform',
                     'test': ['some test'],
                     'builder': 'some builder 2',
-                    'arguments': {
-                        'wk': {'root': 'WebKit'},
-                        'os': {'root': 'OS X'},
-                    },
-                    'buildRequestArgument': 'build-request-id',
                 }
             ]
         }
index 814b9aa..66f086b 100644 (file)
@@ -917,35 +917,53 @@ describe('BuildbotTriggerable', function () {
     });
 
     describe('updateTriggerables', () => {
+
+        function refetchManifest()
+        {
+            MockData.resetV3Models();
+            return TestServer.remoteAPI().getJSON('/api/manifest').then((content) => Manifest._didFetchManifest(content));
+        }
+
         it('should update available triggerables', () => {
             const db = TestServer.database();
+            let macos;
+            let webkit;
             return MockData.addMockData(db).then(() => {
                 return Manifest.fetch();
             }).then(() => {
+                macos = Repository.findById(9);
+                assert.equal(macos.name(), 'macOS');
+                webkit = Repository.findById(11);
+                assert.equal(webkit.name(), 'WebKit');
+
                 return db.selectAll('triggerable_configurations', 'test');
             }).then((configurations) => {
                 assert.equal(configurations.length, 0);
                 assert.equal(Triggerable.all().length, 1);
 
-                let triggerable = Triggerable.all()[0];
+                const triggerable = Triggerable.all()[0];
                 assert.equal(triggerable.name(), 'build-webkit');
-                assert.deepEqual(triggerable.acceptedRepositories(), []);
 
-                let test = Test.findById(MockData.someTestId());
-                let platform = Platform.findById(MockData.somePlatformId());
+                const test = Test.findById(MockData.someTestId());
+                const platform = Platform.findById(MockData.somePlatformId());
                 assert.equal(Triggerable.findByTestConfiguration(test, platform), null);
 
-                let config = MockData.mockTestSyncConfigWithSingleBuilder();
-                let logger = new MockLogger;
-                let slaveInfo = {name: 'sync-slave', password: 'password'};
-                let buildbotTriggerable = new BuildbotTriggerable(config, TestServer.remoteAPI(), MockRemoteAPI, slaveInfo, logger);
+                const groups = TriggerableRepositoryGroup.sortByName(triggerable.repositoryGroups());
+                assert.equal(groups.length, 1);
+                assert.equal(groups[0].name(), 'webkit-svn');
+                assert.deepEqual(groups[0].repositories(), [webkit, macos]);
+
+                const config = MockData.mockTestSyncConfigWithSingleBuilder();
+                config.repositoryGroups = [
+                    {name: 'system-only', repositories: ['macOS'], properties: {'os': '<macOS>'}},
+                    {name: 'system-and-webkit', repositories: ['WebKit', 'macOS'], properties: {'os': '<macOS>', 'wk': '<WebKit>'}},
+                ]
+
+                const logger = new MockLogger;
+                const slaveInfo = {name: 'sync-slave', password: 'password'};
+                const buildbotTriggerable = new BuildbotTriggerable(config, TestServer.remoteAPI(), MockRemoteAPI, slaveInfo, logger);
                 return buildbotTriggerable.updateTriggerable();
-            }).then(() => {
-                MockData.resetV3Models();
-                assert.equal(Triggerable.all().length, 0);
-                return TestServer.remoteAPI().getJSON('/api/manifest');
-            }).then((manifestContent) => {
-                Manifest._didFetchManifest(manifestContent);
+            }).then(() => refetchManifest()).then(() => {
                 return db.selectAll('triggerable_configurations', 'test');
             }).then((configurations) => {
                 assert.equal(configurations.length, 1);
@@ -958,7 +976,30 @@ describe('BuildbotTriggerable', function () {
                 let platform = Platform.findById(MockData.somePlatformId());
                 let triggerable = Triggerable.findByTestConfiguration(test, platform);
                 assert.equal(triggerable.name(), 'build-webkit');
-            });
+
+                const groups = TriggerableRepositoryGroup.sortByName(triggerable.repositoryGroups());
+                assert.equal(groups.length, 2);
+                assert.equal(groups[0].name(), 'system-and-webkit');
+                assert.deepEqual(groups[0].repositories(), [webkit, macos]);
+                assert.equal(groups[1].name(), 'system-only');
+                assert.deepEqual(groups[1].repositories(), [macos]);
+
+                const config = MockData.mockTestSyncConfigWithSingleBuilder();
+                config.repositoryGroups = [ ];
+
+                const logger = new MockLogger;
+                const slaveInfo = {name: 'sync-slave', password: 'password'};
+                const buildbotTriggerable = new BuildbotTriggerable(config, TestServer.remoteAPI(), MockRemoteAPI, slaveInfo, logger);
+                return buildbotTriggerable.updateTriggerable();
+            }).then(() => refetchManifest()).then(() => {
+                assert.equal(Triggerable.all().length, 1);
+                const groups = TriggerableRepositoryGroup.sortByName(Triggerable.all()[0].repositoryGroups());
+                assert.equal(groups.length, 2);
+                assert.equal(groups[0].name(), 'system-and-webkit');
+                assert.deepEqual(groups[0].repositories(), [webkit, macos]);
+                assert.equal(groups[1].name(), 'system-only');
+                assert.deepEqual(groups[1].repositories(), [macos]);
+            })
         });
     });
 
index 94b5a20..bf9ee0f 100644 (file)
@@ -57,14 +57,15 @@ class BuildbotBuildEntry {
 
 class BuildbotSyncer {
 
-    constructor(remote, object)
+    constructor(remote, object, commonConfigurations)
     {
         this._remote = remote;
         this._testConfigurations = [];
+        this._repositoryGroups = commonConfigurations.repositoryGroups;
+        this._slavePropertyName = commonConfigurations.slaveArgument;
+        this._buildRequestPropertyName = commonConfigurations.buildRequestArgument;
         this._builderName = object.builder;
-        this._slavePropertyName = object.slaveArgument;
         this._slaveList = object.slaveList;
-        this._buildRequestPropertyName = object.buildRequestArgument;
         this._entryList = null;
         this._slavesWithNewRequests = new Set;
     }
@@ -78,6 +79,7 @@ class BuildbotSyncer {
         this._testConfigurations.push({test, platform, propertiesTemplate});
     }
     testConfigurations() { return this._testConfigurations; }
+    repositoryGroups() { return this._repositoryGroups; }
 
     matchesConfiguration(request)
     {
@@ -200,34 +202,25 @@ class BuildbotSyncer {
         for (let repository of commitSet.repositories())
             repositoryByName[repository.name()] = repository;
 
-        let propertiesTemplate = null;
-        for (let config of this._testConfigurations) {
-            if (config.platform == buildRequest.platform() && config.test == buildRequest.test())
-                propertiesTemplate = config.propertiesTemplate;
-        }
-        assert(propertiesTemplate);
+        const matchingConfiguration = this._testConfigurations.find((config) => config.platform == buildRequest.platform() && config.test == buildRequest.test());
+        assert(matchingConfiguration, `Build request ${buildRequest.id()} does not match a configuration in the builder "${this._builderName}"`);
+        const propertiesTemplate = matchingConfiguration.propertiesTemplate;
+
+        const repositoryGroup = buildRequest.repositoryGroup();
+        assert(repositoryGroup.accepts(commitSet), `Build request ${buildRequest.id()} does not specify a commit set accepted by the repository group ${repositoryGroup.id()}`);
+
+        const repositoryGroupConfiguration = this._repositoryGroups[repositoryGroup.name()];
+        assert(repositoryGroupConfiguration, `Build request ${buildRequest.id()} uses an unsupported repository group "${repositoryGroup.name()}"`);
 
         const properties = {};
-        for (let key in propertiesTemplate) {
-            const value = propertiesTemplate[key];
-            if (typeof(value) != 'object')
-                properties[key] = value;
-            else if ('root' in value) {
-                const repositoryName = value['root'];
-                const repository = repositoryByName[repositoryName];
-                assert(repository, `"${repositoryName}" must be specified`);
-                properties[key] = commitSet.revisionForRepository(repository);
-            } else if ('rootOptions' in value) {
-                const filteredOptions = value['rootOptions'].filter((option) => option in repositoryByName);
-                assert.equal(filteredOptions.length, 1, `There should be exactly one valid root among "${value['rootOptions']}".`);
-                properties[key] = commitSet.revisionForRepository(repositoryByName[filteredOptions[0]]);
-            }
-            else if ('rootsExcluding' in value) {
-                const revisionSet = this._revisionSetFromCommitSetWithExclusionList(commitSet, value['rootsExcluding']);
-                properties[key] = JSON.stringify(revisionSet);
-            }
-        }
+        for (let propertyName in propertiesTemplate)
+            properties[propertyName] = propertiesTemplate[propertyName];
 
+        const repositoryGroupTemplate = repositoryGroupConfiguration.propertiesTemplate;
+        for (let propertyName in repositoryGroupTemplate) {
+            const value = repositoryGroupTemplate[propertyName];
+            properties[propertyName] = value instanceof Repository ? commitSet.revisionForRepository(value) : value;
+        }
         properties[this._buildRequestPropertyName] = buildRequest.id();
 
         return properties;
@@ -252,35 +245,87 @@ class BuildbotSyncer {
 
     static _loadConfig(remote, config)
     {
-        const shared = config['shared'] || {};
         const types = config['types'] || {};
         const builders = config['builders'] || {};
 
+        assert(config.buildRequestArgument, 'buildRequestArgument must specify the name of the property used to store the build request ID');
+
+        assert.equal(typeof(config.repositoryGroups), 'object', 'repositoryGroups must specify a dictionary from the name to its definition');
+
+        const repositoryGroups = {};
+        for (const name in config.repositoryGroups)
+            repositoryGroups[name] = this._parseRepositoryGroup(name, config.repositoryGroups[name]);
+
+        const commonConfigurations = {
+            repositoryGroups,
+            slaveArgument: config.slaveArgument,
+            buildRequestArgument: config.buildRequestArgument,
+        };
+
         let syncerByBuilder = new Map;
+        const expandedConfigurations = [];
         for (let entry of config['configurations']) {
-            const newConfig = {};
-            this._validateAndMergeConfig(newConfig, shared);
-            this._validateAndMergeConfig(newConfig, entry);
-
-            const expandedConfigurations = this._expandTypesAndPlatforms(newConfig);
-            for (let config of expandedConfigurations) {
-                if ('type' in config) {
-                    const type = config['type'];
-                    assert(type, `${type} is not a valid type in the configuration`);
-                    this._validateAndMergeConfig(config, types[type]);
-                }
-
-                const builder = entry['builder'];
-                if (builders[builder])
-                    this._validateAndMergeConfig(config, builders[builder]);
-
-                this._createTestConfiguration(remote, syncerByBuilder, config);
+            for (const expandedConfig of this._expandTypesAndPlatforms(entry))
+                expandedConfigurations.push(expandedConfig);
+        }
+
+        for (let entry of expandedConfigurations) {
+            const mergedConfig = {};
+            this._validateAndMergeConfig(mergedConfig, entry);
+
+            if ('type' in mergedConfig) {
+                const type = mergedConfig['type'];
+                assert(type, `${type} is not a valid type in the configuration`);
+                this._validateAndMergeConfig(mergedConfig, types[type]);
             }
+
+            const builder = entry['builder'];
+            if (builders[builder])
+                this._validateAndMergeConfig(mergedConfig, builders[builder]);
+
+            this._createTestConfiguration(remote, syncerByBuilder, mergedConfig, commonConfigurations);
         }
 
         return Array.from(syncerByBuilder.values());
     }
 
+    static _parseRepositoryGroup(name, group)
+    {
+        assert(Array.isArray(group.repositories), 'Each repository group must specify a list of repositories');
+        assert(group.repositories.length, 'Each repository group must specify a list of repositories');
+        assert(!('description' in group) || typeof(group['description']) == 'string', 'The description of a repository group must be a string');
+        assert.equal(typeof(group.properties), 'object', 'Each repository group must specify a dictionary of properties');
+
+        const repositoryByName = {};
+        const repositories = group.repositories.map((repositoryName) => {
+            const repository = Repository.findTopLevelByName(repositoryName);
+            assert(repository, `"${repositoryName}" is not a valid repository name`);
+            repositoryByName[repositoryName] = repository;
+            return repository;
+        });
+        const propertiesTemplate = {};
+        const usedRepositories = [];
+        for (const propertyName in group.properties) {
+            let value = group.properties[propertyName];
+            const match = value.match(/^<(.+)>$/);
+            if (match) {
+                const repositoryName = match[1];
+                value = repositoryByName[repositoryName];
+                assert(value, `Repository group "${name}" uses "${repositoryName}" in its property but does not list in the list of repositories`);
+                usedRepositories.push(value);
+            }
+            propertiesTemplate[propertyName] = value;
+        }
+        assert.equal(repositories.length, usedRepositories.length, `Repository group "${name}" does not use some of the listed repositories`);
+        return {
+            name: group.name,
+            description: group.description,
+            propertiesTemplate,
+            arguments: group.arguments,
+            repositories: repositories.map((repository) => repository.id()),
+        };
+    }
+
     static _expandTypesAndPlatforms(unresolvedConfig)
     {
         const typeExpanded = [];
@@ -302,13 +347,11 @@ class BuildbotSyncer {
         return configurations;
     }
 
-    static _createTestConfiguration(remote, syncerByBuilder, newConfig)
+    static _createTestConfiguration(remote, syncerByBuilder, newConfig, commonConfigurations)
     {
         assert('platform' in newConfig, 'configuration must specify a platform');
         assert('test' in newConfig, 'configuration must specify a test');
         assert('builder' in newConfig, 'configuration must specify a builder');
-        assert('properties' in newConfig, 'configuration must specify arguments to post on a builder');
-        assert('buildRequestArgument' in newConfig, 'configuration must specify buildRequestArgument');
 
         const test = Test.findByPath(newConfig.test);
         assert(test, `${newConfig.test} is not a valid test path`);
@@ -318,10 +361,10 @@ class BuildbotSyncer {
 
         let syncer = syncerByBuilder.get(newConfig.builder);
         if (!syncer) {
-            syncer = new BuildbotSyncer(remote, newConfig);
+            syncer = new BuildbotSyncer(remote, newConfig, commonConfigurations);
             syncerByBuilder.set(newConfig.builder, syncer);
         }
-        syncer.addTestConfiguration(test, platform, newConfig.properties);
+        syncer.addTestConfiguration(test, platform, newConfig.properties || {});
     }
 
     static _validateAndMergeConfig(config, valuesToMerge, excludedProperty)
@@ -341,7 +384,7 @@ class BuildbotSyncer {
                 break;
             case 'test': // Fallthrough
             case 'slaveList': // Fallthrough
-            case 'platforms':
+            case 'platforms': // Fallthrough
             case 'types':
                 assert(value instanceof Array, `${name} should be an array`);
                 assert(value.every(function (part) { return typeof part == 'string'; }), `${name} should be an array of strings`);
@@ -350,8 +393,6 @@ class BuildbotSyncer {
             case 'type': // Fallthrough
             case 'builder': // Fallthrough
             case 'platform': // Fallthrough
-            case 'slaveArgument': // Fallthrough
-            case 'buildRequestArgument':
                 assert.equal(typeof(value), 'string', `${name} should be of string type`);
                 config[name] = value;
                 break;
index 5d9f7e6..15d3899 100644 (file)
@@ -30,17 +30,21 @@ class BuildbotTriggerable {
     updateTriggerable()
     {
         const map = new Map;
+        let repositoryGroups = [];
         for (const syncer of this._syncers) {
             for (const config of syncer.testConfigurations()) {
                 const entry = {test: config.test.id(), platform: config.platform.id()};
                 map.set(entry.test + '-' + entry.platform, entry);
             }
+            // FIXME: Move BuildbotSyncer._loadConfig here and store repository groups directly.
+            repositoryGroups = syncer.repositoryGroups();
         }
-        return this._remote.postJSON(`/api/update-triggerable/`, {
+        return this._remote.postJSONWithStatus(`/api/update-triggerable/`, {
             'slaveName': this._slaveInfo.name,
             'slavePassword': this._slaveInfo.password,
             'triggerable': this._name,
-            'configurations': Array.from(map.values())});
+            'configurations': Array.from(map.values()),
+            'repositoryGroups': Object.keys(repositoryGroups).map((groupName) => repositoryGroups[groupName])});
     }
 
     syncOnce()
@@ -49,8 +53,9 @@ class BuildbotTriggerable {
         let buildReqeustsByGroup = new Map;
 
         this._logger.log(`Fetching build requests for ${this._name}...`);
+        let validRequests;
         return BuildRequest.fetchForTriggerable(this._name).then((buildRequests) => {
-            this._validateRequests(buildRequests);
+            validRequests = this._validateRequests(buildRequests);
             buildReqeustsByGroup = BuildbotTriggerable._testGroupMapForBuildRequests(buildRequests);
             return this._pullBuildbotOnAllSyncers(buildReqeustsByGroup);
         }).then((updates) => {
@@ -58,7 +63,10 @@ class BuildbotTriggerable {
             const promistList = [];
             const testGroupList = Array.from(buildReqeustsByGroup.values()).sort(function (a, b) { return a.groupOrder - b.groupOrder; });
             for (const group of testGroupList) {
-                const promise = this._scheduleNextRequestInGroupIfSlaveIsAvailable(group, updates);
+                const nextRequest = this._nextRequestInGroup(group, updates);
+                if (!validRequests.has(nextRequest))
+                    continue;
+                const promise = this._scheduleRequestIfSlaveIsAvailable(nextRequest, group.syncer, group.slaveName);
                 if (promise)
                     promistList.push(promise);
             }
@@ -78,14 +86,37 @@ class BuildbotTriggerable {
     _validateRequests(buildRequests)
     {
         const testPlatformPairs = {};
+        const validatedRequests = new Set;
         for (let request of buildRequests) {
             if (!this._syncers.some((syncer) => syncer.matchesConfiguration(request))) {
                 const key = request.platform().id + '-' + request.test().id();
                 if (!(key in testPlatformPairs))
-                    this._logger.error(`No matching configuration for "${request.test().fullName()}" on "${request.platform().name()}".`);                
+                    this._logger.error(`Build request ${request.id()} has no matching configuration: "${request.test().fullName()}" on "${request.platform().name()}".`);
                 testPlatformPairs[key] = true;
+                continue;
             }
+            const triggerable = request.triggerable();
+            if (!triggerable) {
+                this._logger.error(`Build request ${request.id()} does not specify a valid triggerable`);
+                continue;
+            }
+            assert(triggerable instanceof Triggerable, 'Must specify a valid triggerable');
+            assert.equal(triggerable.name(), this._name, 'Must specify the triggerable of this syncer');
+            const repositoryGroup = request.repositoryGroup();
+            if (!repositoryGroup) {
+                this._logger.error(`Build request ${request.id()} does not specify a repository group. Such a build request is no longer supported.`);
+                continue;
+            }
+            const acceptedGroups = triggerable.repositoryGroups();
+            if (!acceptedGroups.includes(repositoryGroup)) {
+                const acceptedNames = acceptedGroups.map((group) => group.name()).join(', ');
+                this._logger.error(`Build request ${request.id()} specifies ${repositoryGroup.name()} but triggerable ${this._name} only accepts ${acceptedNames}`);
+                continue;
+            }
+            validatedRequests.add(request);
         }
+
+        return validatedRequests;
     }
 
     _pullBuildbotOnAllSyncers(buildReqeustsByGroup)
@@ -129,24 +160,25 @@ class BuildbotTriggerable {
         }).then(() => updates);
     }
 
-    _scheduleNextRequestInGroupIfSlaveIsAvailable(groupInfo, pendingUpdates)
+    _nextRequestInGroup(groupInfo, pendingUpdates)
     {
-        let nextRequest = null;
         for (const request of groupInfo.requests) {
             if (request.isScheduled() || (request.id() in pendingUpdates && pendingUpdates[request.id()]['status'] == 'scheduled'))
-                break;
-            if (request.isPending() && !(request.id() in pendingUpdates)) {
-                nextRequest = request;
-                break;
-            }
+                return null;
+            if (request.isPending() && !(request.id() in pendingUpdates))
+                return request;
         }
+        return null;
+    }
+
+    _scheduleRequestIfSlaveIsAvailable(nextRequest, syncer, slaveName)
+    {
         if (!nextRequest)
             return null;
 
         if (!!nextRequest.order()) {
-            const syncer = groupInfo.syncer;
             if (syncer)
-                return this._scheduleRequestWithLog(syncer, nextRequest, groupInfo.slaveName);
+                return this._scheduleRequestWithLog(syncer, nextRequest, slaveName);
             this._logger.error(`Could not identify the syncer for ${nextRequest.id()}.`);
         }
 
index 23d9de6..76b4e37 100644 (file)
@@ -1,6 +1,7 @@
 'use strict';
 
-function importFromV3(file, name) {
+function importFromV3(file, name)
+{
     const modelsDirectory = '../../public/v3/';
 
     global[name] = require(modelsDirectory + file)[name];
@@ -24,6 +25,7 @@ importFromV3('models/platform.js', 'Platform');
 importFromV3('models/repository.js', 'Repository');
 importFromV3('models/commit-set.js', 'MeasurementCommitSet');
 importFromV3('models/commit-set.js', 'CommitSet');
+importFromV3('models/commit-set.js', 'CustomCommitSet');
 importFromV3('models/test.js', 'Test');
 importFromV3('models/test-group.js', 'TestGroup');
 importFromV3('models/time-series.js', 'TimeSeries');
index e93ec2f..f716e9e 100644 (file)
@@ -53,7 +53,7 @@ function sampleBuildRequestData()
     };
 }
 
-describe('TestGroup', function () {
+describe('BuildRequest', function () {
     MockModels.inject();
 
     describe('waitingTime', function () {
index cb6c45a..f2a561b 100644 (file)
@@ -12,16 +12,17 @@ let BuildbotSyncer = require('../tools/js/buildbot-syncer.js').BuildbotSyncer;
 function sampleiOSConfig()
 {
     return {
-        'shared':
-            {
-                'arguments': {
-                    'desired_image': {'root': 'iOS'},
-                    'opensource': {'rootOptions': ['WebKit-SVN', 'WebKit-Git']},
-                    'roots_dict': {'rootsExcluding': ['iOS']}
-                },
-                'slaveArgument': 'slavename',
-                'buildRequestArgument': 'build_request_id'
-            },
+        'slaveArgument': 'slavename',
+        'buildRequestArgument': 'build_request_id',
+        'repositoryGroups': {
+            'ios-svn-webkit': {
+                'repositories': ['WebKit', 'iOS'],
+                'properties': {
+                    'desired_image': '<iOS>',
+                    'opensource': '<WebKit>',
+                }
+            }
+        },
         'types': {
             'speedometer': {
                 'test': ['Speedometer'],
@@ -63,14 +64,8 @@ function sampleiOSConfigWithExpansions()
 {
     return {
         "triggerableName": "build-webkit-ios",
-        "shared":
-            {
-                "arguments": {
-                    "webkit-revision": {"root": "WebKit"},
-                    "os-version": {"root": "iOS"}
-                },
-                "buildRequestArgument": "build-request-id"
-            },
+        "buildRequestArgument": "build-request-id",
+        "repositoryGroups": { },
         "types": {
             "iphone-plt": {
                 "test": ["PLT-iPhone"],
@@ -110,35 +105,25 @@ function sampleiOSConfigWithExpansions()
     }
 }
 
-let sampleCommitSetData = {
-    'WebKit': {
-        'id': '111127',
-        'time': 1456955807334,
-        'repository': 'WebKit',
-        'revision': '197463',
-    },
-    'Shared': {
-        'id': '111237',
-        'time': 1456931874000,
-        'repository': 'Shared',
-        'revision': '80229',
-    },
-    'WebKit-Git': {
-        "id":"111239",
-        "time":1456931874000,
-        "repository":"WebKit-Git",
-        "revision":"9abcdef",
-    },
-};
-
 function smallConfiguration()
 {
     return {
-        'builder': 'some builder',
-        'platform': 'Some platform',
-        'test': ['Some test'],
-        'arguments': {},
-        'buildRequestArgument': 'id'};
+        'buildRequestArgument': 'id',
+        'repositoryGroups': {
+            'ios-svn-webkit': {
+                'repositories': ['iOS', 'WebKit'],
+                'properties': {
+                    'os': '<iOS>',
+                    'wk': '<WebKit>'
+                }
+            }
+        },
+        'configurations': [{
+            'builder': 'some builder',
+            'platform': 'Some platform',
+            'test': ['Some test']
+        }]
+    };
 }
 
 function smallPendingBuild()
@@ -210,12 +195,12 @@ function createSampleBuildRequest(platform, test)
     let commitSet = CommitSet.ensureSingleton('4197', {commits: [
         {'id': '111127', 'time': 1456955807334, 'repository': MockModels.webkit, 'revision': '197463'},
         {'id': '111237', 'time': 1456931874000, 'repository': MockModels.sharedRepository, 'revision': '80229'},
-        {'id': '111239', 'time': 1456931874000, 'repository': MockModels.webkitGit, 'revision': '9abcdef'},
         {'id': '88930', 'time': 0, 'repository': MockModels.ios, 'revision': '13A452'},
     ]});
 
-    let request = BuildRequest.ensureSingleton('16733-' + platform.id(), {'commitSet': commitSet, 'status': 'pending', 'platform': platform, 'test': test});
-    return request;
+    return BuildRequest.ensureSingleton('16733-' + platform.id(), {'triggerable': MockModels.triggerable,
+        repositoryGroup: MockModels.svnRepositoryGroup,
+        'commitSet': commitSet, 'status': 'pending', 'platform': platform, 'test': test});
 }
 
 function samplePendingBuild(buildRequestId, buildTime, slaveName)
@@ -229,11 +214,6 @@ function samplePendingBuild(buildRequestId, buildTime, slaveName)
             ['owner', '<unknown>', 'Force Build Form'],
             ['test_name', 'speedometer', 'Force Build Form'],
             ['reason', 'force build','Force Build Form'],
-            [
-                'roots_dict',
-                JSON.stringify(sampleCommitSetData),
-                'Force Build Form'
-            ],
             ['slavename', slaveName, ''],
             ['scheduler', 'ABTest-iPad-RunBenchmark-Tests-ForceScheduler', 'Scheduler']
         ],
@@ -280,7 +260,6 @@ function sampleInProgressBuild(slaveName)
             ['desired_image', '13A452', 'Force Build Form'],
             ['owner', '<unknown>', 'Force Build Form'],
             ['reason', 'force build', 'Force Build Form'],
-            ['roots_dict', JSON.stringify(sampleCommitSetData), 'Force Build Form'],
             ['scheduler', 'ABTest-iPad-RunBenchmark-Tests-ForceScheduler', 'Scheduler'],
             ['slavename', slaveName || 'ABTest-iPad-0', 'BuildSlave'],
         ],
@@ -356,7 +335,6 @@ function sampleFinishedBuild(buildRequestId, slaveName)
             ['desired_image', '13A452', 'Force Build Form'],
             ['owner', '<unknown>', 'Force Build Form'],
             ['reason', 'force build', 'Force Build Form'],
-            ['roots_dict', JSON.stringify(sampleCommitSetData), 'Force Build Form'],
             ['scheduler', 'ABTest-iPad-RunBenchmark-Tests-ForceScheduler', 'Scheduler'],
             ['slavename', slaveName || 'ABTest-iPad-0', 'BuildSlave'],
         ],
@@ -423,89 +401,68 @@ describe('BuildbotSyncer', () => {
     describe('_loadConfig', () => {
 
         it('should create BuildbotSyncer objects for a configuration that specify all required options', () => {
-            let syncers = BuildbotSyncer._loadConfig(MockRemoteAPI, {'configurations': [smallConfiguration()]});
-            assert.equal(syncers.length, 1);
+            assert.equal(BuildbotSyncer._loadConfig(MockRemoteAPI, smallConfiguration()).length, 1);
         });
 
         it('should throw when some required options are missing', () => {
             assert.throws(() => {
-                let config = smallConfiguration();
-                delete config['builder'];
-                BuildbotSyncer._loadConfig(MockRemoteAPI, {'configurations': [config]});
+                const config = smallConfiguration();
+                delete config.configurations[0].builder;
+                BuildbotSyncer._loadConfig(MockRemoteAPI, config);
             }, 'builder should be a required option');
             assert.throws(() => {
-                let config = smallConfiguration();
-                delete config['platform'];
-                BuildbotSyncer._loadConfig(MockRemoteAPI, {'configurations': [config]});
+                const config = smallConfiguration();
+                delete config.configurations[0].platform;
+                BuildbotSyncer._loadConfig(MockRemoteAPI, config);
             }, 'platform should be a required option');
             assert.throws(() => {
-                let config = smallConfiguration();
-                delete config['test'];
-                BuildbotSyncer._loadConfig(MockRemoteAPI, {'configurations': [config]});
+                const config = smallConfiguration();
+                delete config.configurations[0].test;
+                BuildbotSyncer._loadConfig(MockRemoteAPI, config);
             }, 'test should be a required option');
             assert.throws(() => {
-                let config = smallConfiguration();
-                delete config['arguments'];
-                BuildbotSyncer._loadConfig(MockRemoteAPI, {'configurations': [config]});
-            });
-            assert.throws(() => {
-                let config = smallConfiguration();
-                delete config['buildRequestArgument'];
-                BuildbotSyncer._loadConfig(MockRemoteAPI, {'configurations': [config]});
-            });
+                const config = smallConfiguration();
+                delete config.buildRequestArgument;
+                BuildbotSyncer._loadConfig(MockRemoteAPI, config);
+            }, 'buildRequestArgument should be required');
         });
 
         it('should throw when a test name is not an array of strings', () => {
             assert.throws(() => {
-                let config = smallConfiguration();
-                config.test = 'some test';
-                BuildbotSyncer._loadConfig(MockRemoteAPI, {'configurations': [config]});
+                const config = smallConfiguration();
+                config.configurations[0].test = 'some test';
+                BuildbotSyncer._loadConfig(MockRemoteAPI, config);
             });
             assert.throws(() => {
-                let config = smallConfiguration();
-                config.test = [1];
-                BuildbotSyncer._loadConfig(MockRemoteAPI, {'configurations': [config]});
+                const config = smallConfiguration();
+                config.configurations[0].test = [1];
+                BuildbotSyncer._loadConfig(MockRemoteAPI, config);
             });
         });
 
         it('should throw when arguments is not an object', () => {
             assert.throws(() => {
-                let config = smallConfiguration();
-                config.arguments = 'hello';
-                BuildbotSyncer._loadConfig(MockRemoteAPI, {'configurations': [config]});
+                const config = smallConfiguration();
+                config.configurations[0].arguments = 'hello';
+                BuildbotSyncer._loadConfig(MockRemoteAPI, config);
             });
         });
 
         it('should throw when arguments\'s values are malformed', () => {
             assert.throws(() => {
-                let config = smallConfiguration();
-                config.arguments = {'some': {'root': 'some root', 'rootsExcluding': ['other root']}};
-                BuildbotSyncer._loadConfig(RemoteAPI, {'configurations': [config]});
-            });
-            assert.throws(() => {
-                let config = smallConfiguration();
-                config.arguments = {'some': {'otherKey': 'some root'}};
-                BuildbotSyncer._loadConfig(RemoteAPI, {'configurations': [config]});
-            });
-            assert.throws(() => {
-                let config = smallConfiguration();
-                config.arguments = {'some': {'root': ['a', 'b']}};
-                BuildbotSyncer._loadConfig(RemoteAPI, {'configurations': [config]});
-            });
-            assert.throws(() => {
-                let config = smallConfiguration();
-                config.arguments = {'some': {'root': 1}};
-                BuildbotSyncer._loadConfig(RemoteAPI, {'configurations': [config]});
+                const config = smallConfiguration();
+                config.configurations[0].arguments = {'some': {'otherKey': 'some root'}};
+                BuildbotSyncer._loadConfig(RemoteAPI, config);
             });
             assert.throws(() => {
-                let config = smallConfiguration();
-                config.arguments = {'some': {'rootsExcluding': 'a'}};
-                BuildbotSyncer._loadConfig(RemoteAPI, {'configurations': [config]});
+                const config = smallConfiguration();
+                config.configurations[0].arguments = {'some': {'root': ['a', 'b']}};
+                BuildbotSyncer._loadConfig(RemoteAPI, config);
             });
             assert.throws(() => {
-                let config = smallConfiguration();
-                config.arguments = {'some': {'rootsExcluding': [1]}};
-                BuildbotSyncer._loadConfig(RemoteAPI, {'configurations': [config]});
+                const config = smallConfiguration();
+                config.configurations[0].arguments = {'some': {'root': 1}};
+                BuildbotSyncer._loadConfig(RemoteAPI, config);
             });
         });
 
@@ -565,13 +522,105 @@ describe('BuildbotSyncer', () => {
             assert.equal(configurations[1].platform, MockModels.ipad);
             assert.equal(configurations[1].test, MockModels.speedometer);
         });
+
+        it('should throw when repositoryGroups is not an object', () => {
+            assert.throws(() => {
+                const config = smallConfiguration();
+                config.repositoryGroups = 1;
+                BuildbotSyncer._loadConfig(MockRemoteAPI, config);
+            });
+            assert.throws(() => {
+                const config = smallConfiguration();
+                config.repositoryGroups = 'hello';
+                BuildbotSyncer._loadConfig(MockRemoteAPI, config);
+            });
+        });
+
+        it('should throw when a repository group does not specify a list of repository', () => {
+            assert.throws(() => {
+                const config = smallConfiguration();
+                config.repositoryGroups = {'some-group': {}};
+                BuildbotSyncer._loadConfig(MockRemoteAPI, config);
+            });
+            assert.throws(() => {
+                const config = smallConfiguration();
+                config.repositoryGroups = {'some-group': {'repositories': 1}};
+                BuildbotSyncer._loadConfig(MockRemoteAPI, config);
+            });
+        });
+
+        it('should throw when a repository group specifies an empty list of repository', () => {
+            assert.throws(() => {
+                const config = smallConfiguration();
+                config.repositoryGroups = {'some-group': {'repositories': []}};
+                BuildbotSyncer._loadConfig(MockRemoteAPI, config);
+            });
+        });
+
+        it('should throw when a repository group specifies a valid repository', () => {
+            assert.throws(() => {
+                const config = smallConfiguration();
+                config.repositoryGroups = {'some-group': {'repositories': ['InvalidRepositoryName']}};
+                BuildbotSyncer._loadConfig(MockRemoteAPI, config);
+            });
+        });
+
+        it('should throw when the description of a repository group is not a string', () => {
+            assert.throws(() => {
+                const config = smallConfiguration();
+                config.repositoryGroups = {'some-group': {'repositories': ['WebKit'], 'description': 1}};
+                BuildbotSyncer._loadConfig(MockRemoteAPI, config);
+            });
+            assert.throws(() => {
+                const config = smallConfiguration();
+                config.repositoryGroups = {'some-group': {'repositories': ['WebKit'], 'description': [1, 2]}};
+                BuildbotSyncer._loadConfig(MockRemoteAPI, config);
+            });
+        });
+
+        it('should throw when a repository group does not specify a dictionary of properties', () => {
+            assert.throws(() => {
+                const config = smallConfiguration();
+                config.repositoryGroups = {'some-group': {'repositories': ['WebKit'], properties: 1}};
+                BuildbotSyncer._loadConfig(MockRemoteAPI, config);
+            });
+            assert.throws(() => {
+                const config = smallConfiguration();
+                config.repositoryGroups = {'some-group': {'repositories': ['WebKit'], properties: 'hello'}};
+                BuildbotSyncer._loadConfig(MockRemoteAPI, config);
+            });
+        });
+
+        it('should throw when a repository group refers to a non-existent repository in the properties dictionary', () => {
+            assert.throws(() => {
+                const config = smallConfiguration();
+                config.repositoryGroups = {'some-group': {'repositories': ['WebKit'], properties: {'wk': '<InvalidRepository>'}}};
+                BuildbotSyncer._loadConfig(MockRemoteAPI, config);
+            });
+        });
+
+        it('should throw when a repository group refers to a repository in the properties dictionary which is not listed in the list of repositories', () => {
+            assert.throws(() => {
+                const config = smallConfiguration();
+                config.repositoryGroups = {'some-group': {'repositories': ['WebKit'], properties: {'os': '<iOS>'}}};
+                BuildbotSyncer._loadConfig(MockRemoteAPI, config);
+            });
+        });
+
+        it('should throw when a repository group does not use a lited repository', () => {
+            assert.throws(() => {
+                const config = smallConfiguration();
+                config.repositoryGroups = {'some-group': {'repositories': ['WebKit'], properties: {}}};
+                BuildbotSyncer._loadConfig(MockRemoteAPI, config);
+            });
+        });
     });
 
     describe('_propertiesForBuildRequest', () => {
         it('should include all properties specified in a given configuration', () => {
             let syncers = BuildbotSyncer._loadConfig(RemoteAPI, sampleiOSConfig());
             let properties = syncers[0]._propertiesForBuildRequest(createSampleBuildRequest(MockModels.iphone, MockModels.speedometer));
-            assert.deepEqual(Object.keys(properties), ['desired_image', 'opensource', 'roots_dict', 'test_name', 'forcescheduler', 'build_request_id']);
+            assert.deepEqual(Object.keys(properties).sort(), ['build_request_id', 'desired_image', 'forcescheduler', 'opensource', 'test_name']);
         });
 
         it('should preserve non-parametric property values', () => {
@@ -591,18 +640,6 @@ describe('BuildbotSyncer', () => {
             assert.equal(properties['desired_image'], '13A452');
         });
 
-        it('should resolve "rootOptions"', () => {
-            let syncers = BuildbotSyncer._loadConfig(RemoteAPI, sampleiOSConfig());
-            let properties = syncers[0]._propertiesForBuildRequest(createSampleBuildRequest(MockModels.iphone, MockModels.speedometer));
-            assert.equal(properties['roots_dict'], JSON.stringify(sampleCommitSetData));
-        });
-
-        it('should resolve "rootsExcluding"', () => {
-            let syncers = BuildbotSyncer._loadConfig(RemoteAPI, sampleiOSConfig());
-            let properties = syncers[0]._propertiesForBuildRequest(createSampleBuildRequest(MockModels.iphone, MockModels.speedometer));
-            assert.equal(properties['roots_dict'], JSON.stringify(sampleCommitSetData));
-        });
-
         it('should set the property for the build request id', () => {
             let syncers = BuildbotSyncer._loadConfig(RemoteAPI, sampleiOSConfig());
             let request = createSampleBuildRequest(MockModels.iphone, MockModels.speedometer);
@@ -887,11 +924,8 @@ describe('BuildbotSyncer', () => {
                 assert.deepEqual(requests[0].data, {
                     'build_request_id': '16733-' + MockModels.iphone.id(),
                     'desired_image': '13A452',
-                    "opensource": "9abcdef",
+                    "opensource": "197463",
                     'forcescheduler': 'ABTest-iPhone-RunBenchmark-Tests-ForceScheduler',
-                    'roots_dict': '{"WebKit":{"id":"111127","time":1456955807334,"repository":"WebKit","revision":"197463"},'
-                        + '"Shared":{"id":"111237","time":1456931874000,"repository":"Shared","revision":"80229"},'
-                        + '"WebKit-Git":{"id":"111239","time":1456931874000,"repository":"WebKit-Git","revision":"9abcdef"}}',
                     'slavename': 'some-slave',
                     'test_name': 'speedometer'
                 });
@@ -915,31 +949,31 @@ describe('BuildbotSyncer', () => {
         }
 
         it('should schedule a build if builder has no builds if slaveList is not specified', () => {
-            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, {'configurations': [smallConfiguration()]})[0];
+            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, smallConfiguration())[0];
 
             return pullBuildbotWithAssertion(syncer, [], {}).then(() => {
                 syncer.scheduleRequestInGroupIfAvailable(createSampleBuildRequest(MockModels.somePlatform, MockModels.someTest));
                 assert.equal(requests.length, 1);
                 assert.equal(requests[0].url, '/builders/some%20builder/force');
                 assert.equal(requests[0].method, 'POST');
-                assert.deepEqual(requests[0].data, {id: '16733-' + MockModels.somePlatform.id()});
+                assert.deepEqual(requests[0].data, {id: '16733-' + MockModels.somePlatform.id(), 'os': '13A452', 'wk': '197463'});
             });
         });
 
         it('should schedule a build if builder only has finished builds if slaveList is not specified', () => {
-            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, {'configurations': [smallConfiguration()]})[0];
+            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, smallConfiguration())[0];
 
             return pullBuildbotWithAssertion(syncer, [], {[-1]: smallFinishedBuild()}).then(() => {
                 syncer.scheduleRequestInGroupIfAvailable(createSampleBuildRequest(MockModels.somePlatform, MockModels.someTest));
                 assert.equal(requests.length, 1);
                 assert.equal(requests[0].url, '/builders/some%20builder/force');
                 assert.equal(requests[0].method, 'POST');
-                assert.deepEqual(requests[0].data, {id: '16733-' + MockModels.somePlatform.id()});
+                assert.deepEqual(requests[0].data, {id: '16733-' + MockModels.somePlatform.id(), 'os': '13A452', 'wk': '197463'});
             });
         });
 
         it('should not schedule a build if builder has a pending build if slaveList is not specified', () => {
-            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, {'configurations': [smallConfiguration()]})[0];
+            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, smallConfiguration())[0];
 
             return pullBuildbotWithAssertion(syncer, [smallPendingBuild()], {}).then(() => {
                 syncer.scheduleRequestInGroupIfAvailable(createSampleBuildRequest(MockModels.somePlatform, MockModels.someTest));
@@ -1041,7 +1075,7 @@ describe('BuildbotSyncer', () => {
         });
 
         it('should not schedule a build if a new request had been submitted to the same builder without slaveList', () => {
-            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, {'configurations': [smallConfiguration()]})[0];
+            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, smallConfiguration())[0];
 
             return pullBuildbotWithAssertion(syncer, [], {}).then(() => {
                 syncer.scheduleRequest(createSampleBuildRequest(MockModels.somePlatform, MockModels.someTest), null);
index b36a3df..657f07b 100644 (file)
@@ -13,6 +13,7 @@ var MockModels = {
             Test.clearStaticMap();
             TestGroup.clearStaticMap();
             BuildRequest.clearStaticMap();
+            Triggerable.clearStaticMap();
 
             MockModels.osx = Repository.ensureSingleton(9, {name: 'OS X'});
             MockModels.ios = Repository.ensureSingleton(22, {name: 'iOS'});
@@ -39,6 +40,23 @@ var MockModels = {
             MockModels.iPhonePLT = Test.ensureSingleton(1500, {name: 'PLT-iPhone'});
             MockModels.pltMean = Metric.ensureSingleton(1158, {name: 'Time', aggregator: 'Arithmetic', test: MockModels.plt});
             MockModels.elCapitan = Platform.ensureSingleton(31, {name: 'El Capitan', metrics: [MockModels.pltMean]});
+
+            MockModels.osRepositoryGroup = new TriggerableRepositoryGroup(31, {
+                name: 'ios',
+                repositories: [MockModels.ios]
+            });
+            MockModels.svnRepositoryGroup = new TriggerableRepositoryGroup(32, {
+                name: 'ios-svn-webkit',
+                repositories: [MockModels.ios, MockModels.webkit, MockModels.sharedRepository]
+            });
+            MockModels.gitRepositoryGroup = new TriggerableRepositoryGroup(33, {
+                name: 'ios-git-webkit',
+                repositories: [MockModels.ios, MockModels.webkitGit, MockModels.sharedRepository]
+            });
+            MockModels.triggerable = new Triggerable(3, {name: 'build-webkit',
+                repositoryGroups: [MockModels.osRepositoryGroup, MockModels.svnRepositoryGroup, MockModels.gitRepositoryGroup],
+                configurations: [{test: MockModels.iPhonePLT, platform: MockModels.iphone}]});
+
         });
     }
 }