Have a memory test where we can validate JSCs mini memory mode
authorsbarati@apple.com <sbarati@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 25 May 2018 19:05:25 +0000 (19:05 +0000)
committersbarati@apple.com <sbarati@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Fri, 25 May 2018 19:05:25 +0000 (19:05 +0000)
commit1661d6f93c3a7c55fc2251da70fc85dae11297a8
treed385ac35c02e656d6cd1239014d09582fb703eaf
parente89718fd3cbcabb2256c8999adcf62faed2f923b
Have a memory test where we can validate JSCs mini memory mode
https://bugs.webkit.org/show_bug.cgi?id=185932

Reviewed by Mark Lam.

PerformanceTests:

We add a directory here with the contents of the testmem benchmark.
To run it, use `Tools/Scripts/run-testmem`. To add new tests in the future,
you just need to add JS files to this directory.

* testmem: Added.
* testmem/air.js: Added.
* testmem/base64.js: Added.
* testmem/basic.js: Added.
* testmem/box2d.js: Added.
* testmem/crypto-md5.js: Added.
* testmem/date-format-tofte.js: Added.
* testmem/earley-boyer.js: Added.
* testmem/hash-map.js: Added.
* testmem/regex-dna.js: Added.
* testmem/splay.js: Added.
* testmem/tagcloud.js: Added.

Source/JavaScriptCore:

This patch adds the testmem CLI. It takes as input a file to run
and the number of iterations to run it (by default it runs it
20 times). Each iteration runs in a new JSContext. Each JSContext
belongs to a VM that is created once. When finished, the CLI dumps
out the peak memory usage of the process, the memory usage at the end
of running all the iterations of the process, and the total time it
took to run all the iterations.

* JavaScriptCore.xcodeproj/project.pbxproj:
* testmem: Added.
* testmem/testmem.mm: Added.
(description):
(Footprint::now):
(main):

Tools:

This patch adds the run-testmem script. This runs the new testmem
CLI against the contents in the testmem benchmark. The contents of testmem
contain tests from ARES-6, Sunspider, Octane, and JetStream.

One iteration of the benchmark comprises of running each testmem test for
a certain number of iterations. We collect the peak memory usage, the memory
usage after running all the iterations, and the time it takes to run all the
iterations. We then run the entire benchmark 3 times over. For each test, we
compute the arithmetic mean over each iteration for time, peak memory, and
end memory. The benchmark then computes a time score and memory score.

peak memory score = geomean({peak memory of each individual test})
end memory score = geomean({end memory of each individual test})
memory score = geomean({peak memory score, end memory score})
time score = geomean({time score of each individual test})

We want the memory score to weigh improvements to peak memory usage and
memory usage at the end equally.

We plan to tune this benchmark in JSC's mini mode by improving the memory
score while ensuring the time score is not more than 10-15% regressed.

* Scripts/run-testmem: Added.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@232193 268f45cc-cd09-0410-ab3c-d52691b4dbfc
17 files changed:
PerformanceTests/ChangeLog
PerformanceTests/testmem/air.js [new file with mode: 0644]
PerformanceTests/testmem/base64.js [new file with mode: 0644]
PerformanceTests/testmem/basic.js [new file with mode: 0644]
PerformanceTests/testmem/box2d.js [new file with mode: 0644]
PerformanceTests/testmem/crypto-md5.js [new file with mode: 0644]
PerformanceTests/testmem/date-format-tofte.js [new file with mode: 0644]
PerformanceTests/testmem/earley-boyer.js [new file with mode: 0644]
PerformanceTests/testmem/hash-map.js [new file with mode: 0644]
PerformanceTests/testmem/regex-dna.js [new file with mode: 0644]
PerformanceTests/testmem/splay.js [new file with mode: 0644]
PerformanceTests/testmem/tagcloud.js [new file with mode: 0644]
Source/JavaScriptCore/ChangeLog
Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj
Source/JavaScriptCore/testmem/testmem.mm [new file with mode: 0644]
Tools/ChangeLog
Tools/Scripts/run-testmem [new file with mode: 0644]