run-benchmark should support SunSpider, Kraken and Octane
authorrniwa@webkit.org <rniwa@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 11 May 2015 21:40:52 +0000 (21:40 +0000)
committerrniwa@webkit.org <rniwa@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 11 May 2015 21:40:52 +0000 (21:40 +0000)
commit931bede8c60f3af02b9ae5221c06bf9a151ca9f3
treee044d44375d592ad03cc73d161cfefd6483679f9
parentfacb0fde6236218b8dc61c92eaa3bf49505e57c4
run-benchmark should support SunSpider, Kraken and Octane
https://bugs.webkit.org/show_bug.cgi?id=144840

Reviewed by Darin Adler.

Added the support for SunSpider, Kraken, and Octane. Because of the licensing issues, we can't commit the
Kraken source code into the WebKit repository as done for other benchmarks. Instead, we'll dynamically
download it from the remote server using newly added RemoteZipBenchmarkBuilder. We do the same for Octane
for simplicity. Use newly added --local-copy option to specify the location of a local copy if there is any.

Renamed "original_benchmark in the plan file to "local_copy" and added a new optional "remote_archive" to
specify the URL to a remote ZIP file. This optional field is used by Kraken and Octane benchmark plans.

In addition, generalized the ability to run a "create script" in JetStreamBenchmarkBuilder since it's also
needed for SunSpider and Kraken. This feature has now been folded into GenericBenchmarkBuilder.

* Scripts/webkitpy/benchmark_runner/benchmark_builder/benchmark_builders.json: Added
RemoteZipBenchmarkBuilder for Kraken and removed JetStreamBenchmarkBuilder since GenericBenchmarkBuilder
now has the ability to run an arbitrary "create script".

* Scripts/run-benchmark:
(main): Added an optional argument, --local-copy, to override the location of the benchmark's local copy.
This also overrides the remote archive URL specified in the plan.

* Scripts/webkitpy/benchmark_runner/README.md: Updated the description.

* Scripts/webkitpy/benchmark_runner/benchmark_builder/benchmark_builders.json: Deleted the entry for
JetStreamBenchmarkBuilder.

* Scripts/webkitpy/benchmark_runner/benchmark_builder/generic_benchmark_builder.py:
(GenericBenchmarkBuilder.prepare): Call _fetchRemoteArchive if local_copy is not specified in the plan or
by --local-copy option but remote_archive is specified. Also call self.clean() here instead of relying on
_runCreateScript and _applyPatch to clean after themselves.
(GenericBenchmarkBuilder._runCreateScript): Moved from JetStreamBenchmarkBuilder._runCreateScript since
JetStream, SunSpider, Kraken all use this feature.
(GenericBenchmarkBuilder._copyBenchmarkToTempDir): Use self.name as the destination location instead of
the leaf directory name since the latter is not available when a remote ZIP file is used.
(GenericBenchmarkBuilder._fetchRemoteArchive): Added.
(GenericBenchmarkBuilder._applyPatch): Apply the patch inside destination directory to avoid hard coding
the benchmark name in the patches.

* Scripts/webkitpy/benchmark_runner/benchmark_builder/jetstream_benchmark_builder.py: Removed.

* Scripts/webkitpy/benchmark_runner/benchmark_runner.py:
(BenchmarkRunner.__init__): Set self.planName. Also override 'local_copy' when --local-copy option is used.
(BenchmarkRunner.execute): Exit early if neither local_copy nor remote_archive is specified. Prefix the URL
the browser opens by planName as the plan file no longer contains that.

* Scripts/webkitpy/benchmark_runner/data/patches/JetStream.patch: Changed the path to be relative against
the top directory of JetStream instead of its parent.
* Scripts/webkitpy/benchmark_runner/data/patches/Kraken.patch: Added.
* Scripts/webkitpy/benchmark_runner/data/patches/Octane.patch: Added.
* Scripts/webkitpy/benchmark_runner/data/patches/Speedometer.patch: Changed the path to be relative against
the top directory of JetStream instead of its parent.
* Scripts/webkitpy/benchmark_runner/data/patches/SunSpider.patch: Added.
* Scripts/webkitpy/benchmark_runner/data/plans/jetstream.plan: Uses GenericBenchmarkBuilder and specifies
the script to run. The entry point was changed to the relative path from the top directory of JetStream as
done in JetStream.patch.
* Scripts/webkitpy/benchmark_runner/data/plans/kraken.plan: Added.
* Scripts/webkitpy/benchmark_runner/data/plans/octane.plan: Added.
* Scripts/webkitpy/benchmark_runner/data/plans/speedometer.plan: The entry point was changed to the relative
path from the top directory of JetStream as done in JetStream.patch.
* Scripts/webkitpy/benchmark_runner/data/plans/sunspider.plan: Added.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@184124 268f45cc-cd09-0410-ab3c-d52691b4dbfc
17 files changed:
Tools/ChangeLog
Tools/Scripts/run-benchmark
Tools/Scripts/webkitpy/benchmark_runner/README.md
Tools/Scripts/webkitpy/benchmark_runner/benchmark_builder/benchmark_builders.json
Tools/Scripts/webkitpy/benchmark_runner/benchmark_builder/generic_benchmark_builder.py
Tools/Scripts/webkitpy/benchmark_runner/benchmark_builder/jetstream_benchmark_builder.py [deleted file]
Tools/Scripts/webkitpy/benchmark_runner/benchmark_runner.py
Tools/Scripts/webkitpy/benchmark_runner/data/patches/JetStream.patch
Tools/Scripts/webkitpy/benchmark_runner/data/patches/Kraken.patch [new file with mode: 0644]
Tools/Scripts/webkitpy/benchmark_runner/data/patches/Octane.patch [new file with mode: 0644]
Tools/Scripts/webkitpy/benchmark_runner/data/patches/Speedometer.patch
Tools/Scripts/webkitpy/benchmark_runner/data/patches/SunSpider.patch [new file with mode: 0644]
Tools/Scripts/webkitpy/benchmark_runner/data/plans/jetstream.plan
Tools/Scripts/webkitpy/benchmark_runner/data/plans/kraken.plan [new file with mode: 0644]
Tools/Scripts/webkitpy/benchmark_runner/data/plans/octane.plan [new file with mode: 0644]
Tools/Scripts/webkitpy/benchmark_runner/data/plans/speedometer.plan
Tools/Scripts/webkitpy/benchmark_runner/data/plans/sunspider.plan [new file with mode: 0644]