1 2014-09-15 Geoffrey Garen <ggaren@apple.com>
3 bmalloc: allocate small and medium objects using the same bump pointer class
4 https://bugs.webkit.org/show_bug.cgi?id=136843
6 Reviewed by Gavin Barraclough.
8 4% speedup on MallocBench.
10 Now that medium-sized objects have dedicated per-size allocators, they
11 don't need to use an arbitrary bump pointer allocator. This means that
12 every allocator knows how many objects it will allocate from the start,
13 and we don't need a post-processing step to adjust refcounts based on
14 real allocation count.
16 * bmalloc.xcodeproj/project.pbxproj: Renamed SmallAllocator to BumpAllocator
17 since it's used for small and medium objects now.
19 * bmalloc/Allocator.cpp:
20 (bmalloc::Allocator::Allocator): Updated to use new interface.
21 (bmalloc::Allocator::scavenge): To "retire" an allocator, we just need
22 to make sure that we finish allocating all the objects in it.
24 (bmalloc::Allocator::allocateMedium):
25 (bmalloc::Allocator::allocateSlowCase):
26 (bmalloc::Allocator::retire): Deleted.
27 (bmalloc::Allocator::processSmallAllocatorLog): Deleted.
28 (bmalloc::Allocator::processMediumAllocatorLog): Deleted.
29 * bmalloc/Allocator.h:
30 (bmalloc::Allocator::allocateFastCase): Removed abstractions and data
31 used to post-process an allocator based on how many objects it allocated.
33 * bmalloc/BumpAllocator.h: Copied from Source/bmalloc/bmalloc/SmallAllocator.h.
34 (bmalloc::BumpAllocator::BumpAllocator):
35 (bmalloc::BumpAllocator::init):
36 (bmalloc::BumpAllocator::line):
37 (bmalloc::BumpAllocator::validate):
38 (bmalloc::BumpAllocator::allocate):
39 (bmalloc::BumpAllocator::refill):
40 (bmalloc::BumpAllocator::clear): Updated these functions to be agnostic
41 about the kinds of lines they allocate into. In some cases, the line
42 type must be provided as a template parameter by the caller.
44 (bmalloc::SmallAllocator::SmallAllocator): Deleted.
45 (bmalloc::SmallAllocator::line): Deleted.
46 (bmalloc::SmallAllocator::allocate): Deleted.
47 (bmalloc::SmallAllocator::objectCount): Deleted.
48 (bmalloc::SmallAllocator::derefCount): Deleted.
49 (bmalloc::SmallAllocator::refill): Deleted.
50 (bmalloc::SmallAllocator::clear): Deleted.
52 * bmalloc/ObjectType.h:
55 * bmalloc/SmallAllocator.h:
56 (bmalloc::SmallAllocator::isNull): Deleted.
57 (bmalloc::SmallAllocator::canAllocate): Deleted.
58 (bmalloc::SmallAllocator::SmallAllocator): Deleted.
59 (bmalloc::SmallAllocator::line): Deleted.
60 (bmalloc::SmallAllocator::allocate): Deleted.
61 (bmalloc::SmallAllocator::objectCount): Deleted.
62 (bmalloc::SmallAllocator::derefCount): Deleted.
63 (bmalloc::SmallAllocator::refill): Deleted.
64 (bmalloc::SmallAllocator::clear): Deleted.
66 2014-09-12 Geoffrey Garen <ggaren@apple.com>
68 Fixed a goof in bmalloc Vector sizing
69 https://bugs.webkit.org/show_bug.cgi?id=136795
71 Reviewed by Gavin Barraclough and Sam Weinig.
73 We want our minimum vector to be page-sized since the OS will give us
74 a page no matter what -- but we want that many bytes, and not enough
75 bytes to store that many elements.
77 * bmalloc/Vector.h: Math is hard.
79 2014-09-11 Geoffrey Garen <ggaren@apple.com>
81 bmalloc should segregate medium-sized objects by line like it does for small-sized objects
82 https://bugs.webkit.org/show_bug.cgi?id=136693
84 Reviewed by Gavin Barraclough.
86 4% reduction in heap size on the MallocBench *_memory_warning benchmarks.
90 We keep an array of medium allocators, just like our array of small
93 In future, we can simplify the allocation fast path by merging the small
94 and medium allocator arrays. For now, this is the simplest change that
97 * bmalloc/Allocator.cpp:
98 (bmalloc::Allocator::Allocator):
99 (bmalloc::Allocator::scavenge):
100 (bmalloc::Allocator::allocateMedium):
101 * bmalloc/Allocator.h:
103 (bmalloc::Sizes::mediumSizeClassFor):
105 2014-09-11 Geoffrey Garen <ggaren@apple.com>
107 Reviewed by Sam Weinig.
109 Renamed log => retire for clarity.
111 * bmalloc/Allocator.cpp:
112 (bmalloc::Allocator::scavenge):
113 (bmalloc::Allocator::retire):
114 (bmalloc::Allocator::allocateMedium):
115 (bmalloc::Allocator::allocateSlowCase):
116 (bmalloc::Allocator::log): Deleted.
117 * bmalloc/Allocator.h:
119 2014-09-11 Geoffrey Garen <ggaren@apple.com>
121 bmalloc: eager scavenge leaves behind a bogus allocator
122 https://bugs.webkit.org/show_bug.cgi?id=136743
124 Reviewed by Sam Weinig.
126 Be sure to clear the allocator after logging it in the eager scavenge
127 case, so that we don't later try to allocate out of the lines that we
130 We didn't need to do this previously because scavenge would only happen
131 at thread exit time, after which no further allocation from the per-thread
132 cache would take place.
134 * bmalloc/Allocator.cpp:
135 (bmalloc::Allocator::scavenge):
136 * bmalloc/MediumAllocator.h:
137 (bmalloc::MediumAllocator::clear):
138 * bmalloc/SmallAllocator.h:
139 (bmalloc::SmallAllocator::clear):
141 2014-09-05 Geoffrey Garen <ggaren@apple.com>
143 bmalloc should honor the FastMalloc statistics API
144 https://bugs.webkit.org/show_bug.cgi?id=136592
146 Reviewed by Gavin Barraclough.
148 We do this by tracking "size" and "capacity" in the VM heap.
150 The VM heap's "capacity" is all the VM we ever allocated.
152 The VM heap's "size" the subset of VM currently held onto by the
153 VM heap (and therefore not in use by the regular heap).
155 Somewhat ironically, reducing the process's memory footprint, increases
156 the size of the VM heap, since the VM heap holds the pages that are
157 purely virtual and not physical.
160 (bmalloc::Heap::size):
161 (bmalloc::Heap::capacity):
163 * bmalloc/VMHeap.cpp:
164 (bmalloc::VMHeap::VMHeap):
165 (bmalloc::VMHeap::allocateSmallChunk):
166 (bmalloc::VMHeap::allocateMediumChunk):
167 (bmalloc::VMHeap::allocateLargeChunk):
169 (bmalloc::VMHeap::size):
170 (bmalloc::VMHeap::capacity):
171 (bmalloc::VMHeap::allocateSmallPage):
172 (bmalloc::VMHeap::allocateMediumPage):
173 (bmalloc::VMHeap::allocateLargeRange):
174 (bmalloc::VMHeap::deallocateSmallPage):
175 (bmalloc::VMHeap::deallocateMediumPage):
176 (bmalloc::VMHeap::deallocateLargeRange):
178 (bmalloc::api::heapSize):
179 (bmalloc::api::heapCapacity):
181 2014-09-02 Geoffrey Garen <ggaren@apple.com>
183 bmalloc crashes on the EWS bots (due to bad large object allocation)
184 https://bugs.webkit.org/show_bug.cgi?id=136469
186 Reviewed by Andreas Kling.
188 It's possible to convince bmalloc to perform a bad large object allocation,
191 (1) Insert object A into freelist F0.
193 (2) Split, merge and split again A's neighbors such that object B is
194 inserted into freelist F0, with boundary tag and size equal to object A,
195 but pointer not completely equal to object A. Put object B at the head of F0.
197 (3) Allocate some other object from F0, swapping its position in the
198 freelist with object B, such that object A is now ahead of object B.
200 --> Now, the next allocation for size A/B will allocate object A, which
201 has a slightly wrong idea about where the object actually begins.
202 Immediately, you'll corrupt a little memory, and over time, you'll also
203 corrupt boundary tag metadata.
205 The solution is to store the begin pointer in the boundary tag. Luckily,
206 this doesn't make the tag any bigger, and it's not a noticeable slowdown
209 * bmalloc/Algorithm.h:
210 (bmalloc::rightShift):
211 * bmalloc/BeginTag.h:
212 (bmalloc::BeginTag::isInFreeList): This is the bug fix. Make sure to
213 validate the start pointer when popping off the free list. Through a
214 very uncommon set of steps, it is possible to have an item in the free
215 list that is valid by all accounts except for its start pointer.
217 * bmalloc/BoundaryTag.h:
218 (bmalloc::BoundaryTag::compactBegin):
219 (bmalloc::BoundaryTag::setRange):
220 (bmalloc::BoundaryTag::setSize): Deleted. Record a compact version of the
221 start pointer. We don't need the whole pointer -- just the offset, in
222 largeAlignment increments, into the relevant boundary tag bucket.
224 * bmalloc/BoundaryTagInlines.h:
225 (bmalloc::validateNext):
226 (bmalloc::BoundaryTag::init):
227 (bmalloc::BoundaryTag::mergeLarge):
228 (bmalloc::BoundaryTag::splitLarge):
229 * bmalloc/SegregatedFreeList.cpp:
230 (bmalloc::SegregatedFreeList::insert):
231 (bmalloc::SegregatedFreeList::takeGreedy):
232 (bmalloc::SegregatedFreeList::take): Provide the whole range instead of
233 the size when establishing a boundary tag, as required by the new
238 2014-08-14 Geoffrey Garen <ggaren@apple.com>
240 Fixed a bmalloc crash seen on the EWS bot
241 https://bugs.webkit.org/show_bug.cgi?id=135955
243 Reviewed by Andreas Kling.
245 * bmalloc/Syscall.h: Some CG APIs vm_copy their input buffers. If the
246 input buffer is a malloc region, that region will get marked Copy-On-Write
247 by the kernel. Calls to madvise() for COW regions fail and return EINVAL
248 on older OS X's. In 10.10, they still fail, but they do not return
251 So, we can only ASSERT that our syscalls succeed starting with 10.10.
253 2014-08-14 Geoffrey Garen <ggaren@apple.com>
255 Fixed the bmalloc build
256 https://bugs.webkit.org/show_bug.cgi?id=135953
258 Reviewed by Andreas Kling.
260 * bmalloc.xcodeproj/project.pbxproj: Marked a few headers as private.
261 These headers are used, so they must be available outside the project.
263 2014-08-13 Daniel Bates <dabates@apple.com>
265 Attempt to fix the build following <http://trac.webkit.org/changeset/172576>
266 (https://bugs.webkit.org/show_bug.cgi?id=135895)
268 Substitute PerThreadStorage<T>::initSharedKeyIfNeeded() for initSharedKeyIfNeeded() in
269 implementation of PerThread<T>::getFastCase().
271 * bmalloc/PerThread.h:
272 (bmalloc::PerThread<T>::getFastCase):
274 2014-08-13 Daniel Bates <dabates@apple.com>
276 Make bmalloc::PerThread work without C++ thread local storage
277 https://bugs.webkit.org/show_bug.cgi?id=135895
279 Reviewed by Geoffrey Garen.
281 Implement support for building bmalloc without C++ thread local storage.
283 * bmalloc/BPlatform.h: Remove macro define BPLATFORM_IOS_SIMULATOR. Added macro function
284 BCOMPILER_SUPPORTS() and macro define BCOMPILER_SUPPORTS_CXX_THREAD_LOCAL that can be used
285 to determine whether the compiler supports C++ thread local storage.
286 * bmalloc/PerThread.h:
287 (bmalloc::PerThreadStorage::get): Modified to call pthread_getspecific() when building
288 without C++ thread local storage.
289 (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Added.
290 (bmalloc::PerThreadStorage::init): Moved logic to initialize shared Pthread key from here to
291 PerThreadStorage::initSharedKeyIfNeeded().
292 (bmalloc::PerThread<T>::getFastCase): Modified to call PerThreadStorage::initSharedKeyIfNeeded()
293 before querying PerThreadStorage::get() when building without C++ thread local storage so as to
294 ensure that the shared key has been initialized.
295 (_pthread_setspecific_direct): Deleted.
296 (_pthread_getspecific_direct): Deleted.
298 2014-08-13 Daniel Bates <dabates@apple.com>
300 [iOS] Make JavaScriptCore and bmalloc build with the public SDK
301 https://bugs.webkit.org/show_bug.cgi?id=135848
303 Reviewed by Geoffrey Garen.
305 * bmalloc/BPlatform.h: Added macro BPLATFORM_IOS_SIMULATOR, which evaluates to true
306 when building for the iOS Simulator.
307 * bmalloc/PerThread.h: Use pthread_machdep.h code path when building for iOS Simulator
308 using the public SDK.
309 (_pthread_setspecific_direct): Added; only defined when building for the iOS Simulator
310 using the public SDK.
311 (_pthread_getspecific_direct): Added; only defined when building for the iOS Simulator
312 using the public SDK.
314 2014-08-12 Daniel Bates <dabates@apple.com>
316 BPLATFORM(IOS) always evaluates to false
317 https://bugs.webkit.org/show_bug.cgi?id=135843
319 Reviewed by Geoffrey Garen.
321 Fix typo in definition of BPLATFORM() and include system header TargetConditionals.h
322 (when building on an Apple platform) so that BPLATFORM(X) evaluates to true when
323 building for platform X. In particular, so that BPLATFORM(IOS) evaluates to true when
326 As a side effect of this change, the change made in <http://trac.webkit.org/changeset/167289>
327 will be honored and iOS will assume a VM page size of 16kB (again) instead of 4kB.
329 * bmalloc/BPlatform.h:
331 2014-08-11 Andy Estes <aestes@apple.com>
333 [iOS] Get rid of iOS.xcconfig
334 https://bugs.webkit.org/show_bug.cgi?id=135809
336 Reviewed by Joseph Pecoraro.
338 All iOS.xcconfig did was include AspenFamily.xcconfig, so there's no need for the indirection.
340 * Configurations/Base.xcconfig:
341 * Configurations/iOS.xcconfig: Removed.
342 * bmalloc.xcodeproj/project.pbxproj:
344 2014-05-01 Dan Bernstein <mitz@apple.com>
346 Fixed production builds for the iOS Simulator.
347 <rdar://problem/16792221>
349 * Configurations/bmalloc.xcconfig: Include INSTALL_PATH_PREFIX in
350 PRIVATE_HEADERS_FOLDER_PATH when installing.
352 2014-04-20 Geoffrey Garen <ggaren@apple.com>
354 bmalloc: Segregate pages by objects size
355 https://bugs.webkit.org/show_bug.cgi?id=131909
357 Reviewed by Andreas Kling.
359 2% reduction in memory-at-end on the Membuster memory_warning benchmarks.
361 * bmalloc/Allocator.cpp:
362 (bmalloc::Allocator::allocateSlowCase):
363 * bmalloc/Allocator.h:
364 (bmalloc::Allocator::allocateFastCase):
365 (bmalloc::Allocator::smallAllocatorFor): Use the new shared helper
366 function for size class calculation.
368 * bmalloc/Deallocator.cpp:
369 (bmalloc::Deallocator::Deallocator):
370 (bmalloc::Deallocator::scavenge):
371 (bmalloc::Deallocator::deallocateSmallLine):
372 (bmalloc::Deallocator::allocateSmallLine):
373 * bmalloc/Deallocator.h: Keep a cache for every size class, since the
374 cache can't be shared anymore.
377 (bmalloc::Heap::allocateSmallLineSlowCase):
379 (bmalloc::Heap::deallocateSmallLine): Ditto.
381 (bmalloc::Heap::allocateSmallLine): Check size class in addition to
382 page refcount when allocating a line because we might have deallocated
383 the page and the recycled it for another size class.
385 (bmalloc::Heap::deallocateMediumLine):
386 (bmalloc::Heap::allocateMediumLine):
388 (bmalloc::Line::refCount):
390 (bmalloc::Page::refCount):
391 (bmalloc::Page::smallSizeClass):
392 (bmalloc::Page::setSmallSizeClass):
393 (bmalloc::Page<Traits>::refCount): Deleted.
395 (bmalloc::Sizes::smallSizeClassFor): New shared API for computing
396 an index into an array from a size.
398 2014-04-19 Geoffrey Garen <ggaren@apple.com>
400 bmalloc: Improved alignment in LargeChunk
401 https://bugs.webkit.org/show_bug.cgi?id=131895
403 Reviewed by Andreas Kling.
406 * bmalloc/LargeChunk.h: Align to vmPageSize just like Chunk does.
407 Technically, the previous alignment was harmless, but I would prefer,
408 dear reader, not to have to explain the interlocking set of
409 circumstances that made it so.
411 2014-04-19 Geoffrey Garen <ggaren@apple.com>
413 Rolled out r167502 because it caused a crash on the facebook benchmark.
417 bmalloc: Added an XSmall line size
418 https://bugs.webkit.org/show_bug.cgi?id=131851
420 Reviewed by Sam Weinig.
422 2014-04-19 Geoffrey Garen <ggaren@apple.com>
424 bmalloc: Mutex should be harder to use wrong
425 https://bugs.webkit.org/show_bug.cgi?id=131879
427 Reviewed by Andreas Kling.
429 Mutex now has a proper constructor, so you can't deadlock by forgetting
432 * bmalloc.xcodeproj/project.pbxproj:
433 * bmalloc/Allocator.cpp:
434 (bmalloc::Allocator::processXSmallAllocatorLog):
435 (bmalloc::Allocator::processSmallAllocatorLog):
436 (bmalloc::Allocator::processMediumAllocatorLog):
437 (bmalloc::Allocator::allocateLarge):
438 (bmalloc::Allocator::allocateXLarge): Global replace Mutex => StaticMutex,
439 since the Heap mutex is a static.
441 * bmalloc/AsyncTask.h:
442 (bmalloc::Function>::AsyncTask): Use Mutex, since we're not static. No
443 need for explicit initialization anymore.
445 * bmalloc/Deallocator.cpp:
446 (bmalloc::Deallocator::scavenge):
447 (bmalloc::Deallocator::deallocateLarge):
448 (bmalloc::Deallocator::deallocateXLarge):
449 (bmalloc::Deallocator::processObjectLog):
450 (bmalloc::Deallocator::deallocateSmallLine):
451 (bmalloc::Deallocator::deallocateXSmallLine):
452 (bmalloc::Deallocator::allocateSmallLine):
453 (bmalloc::Deallocator::allocateXSmallLine):
454 (bmalloc::Deallocator::deallocateMediumLine):
455 (bmalloc::Deallocator::allocateMediumLine):
456 * bmalloc/Deallocator.h:
459 (bmalloc::Heap::Heap):
460 (bmalloc::Heap::concurrentScavenge):
461 (bmalloc::Heap::scavenge):
462 (bmalloc::Heap::scavengeSmallPages):
463 (bmalloc::Heap::scavengeXSmallPages):
464 (bmalloc::Heap::scavengeMediumPages):
465 (bmalloc::Heap::scavengeLargeRanges):
466 (bmalloc::Heap::allocateXSmallLineSlowCase):
467 (bmalloc::Heap::allocateSmallLineSlowCase):
468 (bmalloc::Heap::allocateMediumLineSlowCase):
469 (bmalloc::Heap::allocateXLarge):
470 (bmalloc::Heap::deallocateXLarge):
471 (bmalloc::Heap::allocateLarge):
472 (bmalloc::Heap::deallocateLarge):
474 (bmalloc::Heap::deallocateXSmallLine):
475 (bmalloc::Heap::allocateXSmallLine):
476 (bmalloc::Heap::deallocateSmallLine):
477 (bmalloc::Heap::allocateSmallLine):
478 (bmalloc::Heap::deallocateMediumLine):
479 (bmalloc::Heap::allocateMediumLine):
481 (bmalloc::Line<Traits>::deref):
482 * bmalloc/Mutex.cpp: Removed.
484 (bmalloc::Mutex::Mutex):
485 (bmalloc::Mutex::init): Deleted.
486 (bmalloc::Mutex::try_lock): Deleted.
487 (bmalloc::Mutex::lock): Deleted.
488 (bmalloc::Mutex::unlock): Deleted.
490 (bmalloc::Page<Traits>::ref):
491 (bmalloc::Page<Traits>::deref):
492 (bmalloc::Page<Traits>::refCount):
493 * bmalloc/PerProcess.h:
494 (bmalloc::PerProcess::mutex):
495 (bmalloc::PerProcess<T>::getSlowCase):
496 * bmalloc/StaticMutex.cpp: Added.
497 (bmalloc::StaticMutex::lockSlowCase):
498 * bmalloc/StaticMutex.h: Added.
499 (bmalloc::StaticMutex::init):
500 (bmalloc::StaticMutex::try_lock):
501 (bmalloc::StaticMutex::lock):
502 (bmalloc::StaticMutex::unlock):
504 (bmalloc::VMHeap::deallocateXSmallPage):
505 (bmalloc::VMHeap::deallocateSmallPage):
506 (bmalloc::VMHeap::deallocateMediumPage):
507 (bmalloc::VMHeap::deallocateLargeRange):
509 (bmalloc::api::scavenge): Global replace Mutex => StaticMutex,
510 since the Heap mutex is a static.
512 2014-04-18 Geoffrey Garen <ggaren@apple.com>
514 bmalloc: AsyncTask should use Mutex instead of std::mutex
515 https://bugs.webkit.org/show_bug.cgi?id=131865
517 Reviewed by Gavin Barraclough.
519 std::mutex is so slow that it makes parallelizing simple tasks through
520 AsyncTask a net regression. Mutex fixes this.
522 * bmalloc/AsyncTask.h:
523 (bmalloc::Function>::AsyncTask):
524 (bmalloc::Function>::join):
525 (bmalloc::Function>::runSlowCase):
526 (bmalloc::Function>::entryPoint):
528 (bmalloc::Mutex::init):
530 2014-04-18 Geoffrey Garen <ggaren@apple.com>
532 bmalloc: Added an XSmall line size
533 https://bugs.webkit.org/show_bug.cgi?id=131851
535 Reviewed by Sam Weinig.
537 Reduces malloc footprint on Membuster recordings by 10%.
539 This is a throughput regression, but we're still way ahead of TCMalloc.
540 I have some ideas for how to recover the regression -- but I wanted to
541 get this win in first.
543 Full set of benchmark results:
545 bmalloc> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks --measure-heap nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
549 reddit_memory_warning 7,896kB 7,532kB ^ 1.05x smaller
550 flickr_memory_warning 12,968kB 12,324kB ^ 1.05x smaller
551 theverge_memory_warning 16,672kB 15,200kB ^ 1.1x smaller
553 <geometric mean> 11,952kB 11,216kB ^ 1.07x smaller
554 <arithmetic mean> 12,512kB 11,685kB ^ 1.07x smaller
555 <harmonic mean> 11,375kB 10,726kB ^ 1.06x smaller
558 reddit_memory_warning 7,320kB 6,856kB ^ 1.07x smaller
559 flickr_memory_warning 10,848kB 9,692kB ^ 1.12x smaller
560 theverge_memory_warning 16,380kB 14,872kB ^ 1.1x smaller
562 <geometric mean> 10,916kB 9,961kB ^ 1.1x smaller
563 <arithmetic mean> 11,516kB 10,473kB ^ 1.1x smaller
564 <harmonic mean> 10,350kB 9,485kB ^ 1.09x smaller
566 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
570 churn 127ms 151ms ! 1.19x slower
571 list_allocate 130ms 164ms ! 1.26x slower
572 tree_allocate 109ms 127ms ! 1.17x slower
573 tree_churn 115ms 120ms ! 1.04x slower
574 facebook 240ms 259ms ! 1.08x slower
575 fragment 91ms 131ms ! 1.44x slower
576 fragment_iterate 105ms 106ms ! 1.01x slower
577 message_one 260ms 259ms ^ 1.0x faster
578 message_many 149ms 154ms ! 1.03x slower
579 medium 194ms 248ms ! 1.28x slower
580 big 157ms 160ms ! 1.02x slower
582 <geometric mean> 144ms 163ms ! 1.13x slower
583 <arithmetic mean> 152ms 171ms ! 1.12x slower
584 <harmonic mean> 137ms 156ms ! 1.14x slower
586 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
590 churn 126ms 148ms ! 1.17x slower
591 churn --parallel 62ms 76ms ! 1.23x slower
592 list_allocate 130ms 164ms ! 1.26x slower
593 list_allocate --parallel 120ms 175ms ! 1.46x slower
594 tree_allocate 111ms 127ms ! 1.14x slower
595 tree_allocate --parallel 95ms 135ms ! 1.42x slower
596 tree_churn 115ms 124ms ! 1.08x slower
597 tree_churn --parallel 107ms 126ms ! 1.18x slower
598 facebook 240ms 276ms ! 1.15x slower
599 facebook --parallel 802ms 1,088ms ! 1.36x slower
600 fragment 92ms 130ms ! 1.41x slower
601 fragment --parallel 66ms 124ms ! 1.88x slower
602 fragment_iterate 109ms 127ms ! 1.17x slower
603 fragment_iterate --parallel 55ms 64ms ! 1.16x slower
604 message_one 260ms 260ms
605 message_many 170ms 238ms ! 1.4x slower
606 medium 185ms 250ms ! 1.35x slower
607 medium --parallel 210ms 334ms ! 1.59x slower
608 big 150ms 169ms ! 1.13x slower
609 big --parallel 138ms 144ms ! 1.04x slower
611 <geometric mean> 135ms 170ms ! 1.26x slower
612 <arithmetic mean> 167ms 214ms ! 1.28x slower
613 <harmonic mean> 117ms 148ms ! 1.26x slower
615 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
619 reddit_memory_warning 13,836kB 13,436kB ^ 1.03x smaller
620 flickr_memory_warning 24,868kB 25,188kB ! 1.01x bigger
621 theverge_memory_warning 24,504kB 26,636kB ! 1.09x bigger
623 <geometric mean> 20,353kB 20,812kB ! 1.02x bigger
624 <arithmetic mean> 21,069kB 21,753kB ! 1.03x bigger
625 <harmonic mean> 19,570kB 19,780kB ! 1.01x bigger
628 reddit_memory_warning 8,656kB 10,016kB ! 1.16x bigger
629 flickr_memory_warning 11,844kB 13,784kB ! 1.16x bigger
630 theverge_memory_warning 18,516kB 22,748kB ! 1.23x bigger
632 <geometric mean> 12,382kB 14,644kB ! 1.18x bigger
633 <arithmetic mean> 13,005kB 15,516kB ! 1.19x bigger
634 <harmonic mean> 11,813kB 13,867kB ! 1.17x bigger
636 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
640 churn 416ms 148ms ^ 2.81x faster
641 list_allocate 463ms 164ms ^ 2.82x faster
642 tree_allocate 292ms 127ms ^ 2.3x faster
643 tree_churn 157ms 120ms ^ 1.31x faster
644 facebook 327ms 276ms ^ 1.18x faster
645 fragment 335ms 129ms ^ 2.6x faster
646 fragment_iterate 344ms 108ms ^ 3.19x faster
647 message_one 386ms 258ms ^ 1.5x faster
648 message_many 410ms 154ms ^ 2.66x faster
649 medium 391ms 245ms ^ 1.6x faster
650 big 261ms 167ms ^ 1.56x faster
652 <geometric mean> 332ms 164ms ^ 2.02x faster
653 <arithmetic mean> 344ms 172ms ^ 1.99x faster
654 <harmonic mean> 317ms 157ms ^ 2.02x faster
656 * bmalloc.xcodeproj/project.pbxproj:
657 * bmalloc/Allocator.cpp:
658 (bmalloc::Allocator::Allocator): Don't assume that each allocator's
659 index corresponds with its size. Instead, use the size selection function
660 explicitly. Now that we have XSmall, some small allocator entries are
663 (bmalloc::Allocator::scavenge):
664 (bmalloc::Allocator::log):
665 (bmalloc::Allocator::processXSmallAllocatorLog):
666 (bmalloc::Allocator::allocateSlowCase):
667 * bmalloc/Allocator.h:
668 (bmalloc::Allocator::xSmallAllocatorFor):
669 (bmalloc::Allocator::allocateFastCase):
671 * bmalloc/Deallocator.cpp:
672 (bmalloc::Deallocator::scavenge):
673 (bmalloc::Deallocator::processObjectLog):
674 (bmalloc::Deallocator::deallocateSlowCase):
675 (bmalloc::Deallocator::deallocateXSmallLine):
676 (bmalloc::Deallocator::allocateXSmallLine):
677 * bmalloc/Deallocator.h:
678 (bmalloc::Deallocator::deallocateFastCase):
680 (bmalloc::Heap::scavenge):
681 (bmalloc::Heap::scavengeXSmallPages):
682 (bmalloc::Heap::allocateXSmallLineSlowCase):
684 (bmalloc::Heap::deallocateXSmallLine):
685 (bmalloc::Heap::allocateXSmallLine):
686 * bmalloc/LargeChunk.h:
687 (bmalloc::LargeChunk::get):
688 (bmalloc::LargeChunk::endTag):
690 * bmalloc/MediumAllocator.h:
691 (bmalloc::MediumAllocator::allocate):
692 (bmalloc::MediumAllocator::refill):
693 * bmalloc/ObjectType.cpp:
694 (bmalloc::objectType):
695 * bmalloc/ObjectType.h:
700 (bmalloc::isSmallOrMedium): Deleted.
701 * bmalloc/SegregatedFreeList.h: I boiler-plate copied existing code for
702 handling small objects. There's probably a reasonable way to share this
703 code in the future -- I'll look into that once it's stopped changing.
705 * bmalloc/Sizes.h: Tweaked size classes to make Membuster happy. This
706 is the main reason things got slower.
708 * bmalloc/SmallAllocator.h:
709 (bmalloc::SmallAllocator::allocate):
710 * bmalloc/SmallTraits.h:
711 * bmalloc/VMHeap.cpp:
712 (bmalloc::VMHeap::allocateXSmallChunk):
714 (bmalloc::VMHeap::allocateXSmallPage):
715 (bmalloc::VMHeap::deallocateXSmallPage):
716 * bmalloc/XSmallAllocator.h: Added.
717 (bmalloc::XSmallAllocator::isNull):
718 (bmalloc::XSmallAllocator::canAllocate):
719 (bmalloc::XSmallAllocator::XSmallAllocator):
720 (bmalloc::XSmallAllocator::line):
721 (bmalloc::XSmallAllocator::allocate):
722 (bmalloc::XSmallAllocator::objectCount):
723 (bmalloc::XSmallAllocator::derefCount):
724 (bmalloc::XSmallAllocator::refill):
725 * bmalloc/XSmallChunk.h: Added.
726 * bmalloc/XSmallLine.h: Added.
727 * bmalloc/XSmallPage.h: Added.
728 * bmalloc/XSmallTraits.h: Added.
730 (bmalloc::api::realloc): Boiler-plate copy, as above.
732 2014-04-14 Geoffrey Garen <ggaren@apple.com>
734 MallocBench should scavenge explicitly instead of waiting
735 https://bugs.webkit.org/show_bug.cgi?id=131661
737 Reviewed by Andreas Kling.
739 Added explicit scavenge support to bmalloc. This isn't a memory win,
740 since bmalloc's per-thread cache is so small. But it makes testing
743 * bmalloc/Allocator.cpp:
744 (bmalloc::Allocator::~Allocator):
745 (bmalloc::Allocator::scavenge):
746 * bmalloc/Allocator.h:
748 (bmalloc::Cache::operator new):
749 (bmalloc::Cache::operator delete):
750 (bmalloc::Cache::Cache):
751 (bmalloc::Cache::scavenge):
753 * bmalloc/Deallocator.cpp:
754 (bmalloc::Deallocator::~Deallocator):
755 (bmalloc::Deallocator::scavenge):
756 * bmalloc/Deallocator.h: Factored existing scavenging code into helper
757 functions, for reuse.
761 (bmalloc::Heap::concurrentScavenge):
762 (bmalloc::Heap::scavenge):
763 (bmalloc::Heap::scavengeSmallPages):
764 (bmalloc::Heap::scavengeMediumPages):
765 (bmalloc::Heap::scavengeLargeRanges):
766 * bmalloc/Heap.h: Made scavenge sleep duration a parameter. Forced
767 scavenging -- in response to a benchmark or a low memory warning --
768 wants to complete as soon as possible, so its sleep duration is 0.
771 (bmalloc::api::scavenge):
772 * bmalloc/mbmalloc.cpp: Exported the scavenge API for MallocBench's use.
774 2014-04-14 Geoffrey Garen <ggaren@apple.com>
777 https://bugs.webkit.org/show_bug.cgi?id=131658
779 Reviewed by Sam Weinig.
781 This reduces memory use a lot on Membuster:
785 reddit_memory_warning 18ms 17ms ^ 1.06x faster
786 flickr_memory_warning 34ms 36ms ! 1.06x slower
787 theverge_memory_warning 39ms 41ms ! 1.05x slower
789 <geometric mean> 29ms 29ms ! 1.02x slower
790 <arithmetic mean> 30ms 31ms ! 1.03x slower
791 <harmonic mean> 27ms 27ms ^ 1.0x faster
794 reddit_memory_warning 16,412kB 16,436kB ! 1.0x bigger
795 flickr_memory_warning 30,120kB 30,184kB ! 1.0x bigger
796 theverge_memory_warning 33,408kB 33,420kB ! 1.0x bigger
798 <geometric mean> 25,466kB 25,499kB ! 1.0x bigger
799 <arithmetic mean> 26,647kB 26,680kB ! 1.0x bigger
800 <harmonic mean> 24,181kB 24,214kB ! 1.0x bigger
803 reddit_memory_warning 2,404kB 1,920kB ^ 1.25x smaller
804 flickr_memory_warning 3,764kB 3,072kB ^ 1.23x smaller
805 theverge_memory_warning 3,648kB 3,132kB ^ 1.16x smaller
807 <geometric mean> 3,208kB 2,644kB ^ 1.21x smaller
808 <arithmetic mean> 3,272kB 2,708kB ^ 1.21x smaller
809 <harmonic mean> 3,139kB 2,574kB ^ 1.22x smaller
812 * bmalloc.xcodeproj/project.pbxproj:
813 * bmalloc/BPlatform.h: Added.
814 * bmalloc/VMAllocate.h: Only use 16kB pages on iOS because the page size
817 2014-04-14 Alexey Proskuryakov <ap@apple.com>
819 Fixed svn:ignore on bmalloc.xcodeproj, it had erroneous leading spaces.
821 * bmalloc.xcodeproj: Modified property svn:ignore.
823 2014-04-13 Geoffrey Garen <ggaren@apple.com>
825 Fixed some mbmalloc exports
826 https://bugs.webkit.org/show_bug.cgi?id=131599
828 Reviewed by Ryosuke Niwa.
830 * bmalloc.xcodeproj/project.pbxproj: Made some headers a private part
831 of the project, so we can call them from API.
833 * bmalloc/mbmalloc.cpp: Marked the mbmalloc functions with default
834 visibility, so they show up as exported in the .dylib.
836 2014-04-09 Geoffrey Garen <ggaren@apple.com>
838 Put bmalloc headers in the right place
839 https://bugs.webkit.org/show_bug.cgi?id=131464
841 Reviewed by Mark Rowe.
843 * Configurations/bmalloc.xcconfig: Set PRIVATE_HEADERS_FOLDER_PATH to
844 specify that we don't just want to dump all of our generically-named
845 headers into /usr/local/include.
847 2014-04-08 Geoffrey Garen <ggaren@apple.com>
849 Made bmalloc more #include friendly
850 https://bugs.webkit.org/show_bug.cgi?id=131386
852 Reviewed by Andreas Kling.
854 Marked a bunch of headers private so they can be used from client code
855 that #includes bmalloc.h.
857 Renamed ASSERT macros to BASSERT. This matches their header, which already
858 had to be renamed, and fixes conflicts with WTF's ASSERT macros.
860 * bmalloc.xcodeproj/project.pbxproj:
861 * bmalloc/Allocator.cpp:
862 (bmalloc::Allocator::allocateSlowCase):
863 * bmalloc/AsyncTask.h:
864 (bmalloc::Function>::runSlowCase):
866 * bmalloc/BoundaryTag.h:
867 (bmalloc::BoundaryTag::setSize):
868 * bmalloc/BoundaryTagInlines.h:
870 (bmalloc::BoundaryTag::init):
871 (bmalloc::BoundaryTag::deallocate):
872 (bmalloc::BoundaryTag::splitLarge):
873 (bmalloc::BoundaryTag::allocate):
875 * bmalloc/Deallocator.cpp:
876 (bmalloc::Deallocator::processObjectLog):
877 (bmalloc::Deallocator::deallocateSlowCase):
878 * bmalloc/Deallocator.h:
879 (bmalloc::Deallocator::deallocateFastCase):
880 * bmalloc/FixedVector.h:
881 (bmalloc::Capacity>::operator):
882 (bmalloc::Capacity>::push):
883 (bmalloc::Capacity>::pop):
884 (bmalloc::Capacity>::shrink):
886 (bmalloc::Heap::allocateLarge):
887 * bmalloc/LargeChunk.h:
888 (bmalloc::LargeChunk::get):
889 (bmalloc::LargeChunk::endTag):
891 (bmalloc::Line<Traits>::concurrentRef):
892 (bmalloc::Line<Traits>::deref):
893 * bmalloc/MediumAllocator.h:
894 (bmalloc::MediumAllocator::allocate):
895 * bmalloc/ObjectType.h:
898 (bmalloc::Page<Traits>::ref):
899 (bmalloc::Page<Traits>::deref):
900 * bmalloc/PerThread.h:
901 (bmalloc::PerThread<T>::getSlowCase):
902 * bmalloc/SegregatedFreeList.cpp:
903 (bmalloc::SegregatedFreeList::SegregatedFreeList):
904 (bmalloc::SegregatedFreeList::insert):
905 * bmalloc/SmallAllocator.h:
906 (bmalloc::SmallAllocator::allocate):
907 (bmalloc::SmallAllocator::refill):
909 * bmalloc/VMAllocate.h:
910 (bmalloc::vmValidate):
911 (bmalloc::vmAllocate):
912 (bmalloc::vmDeallocatePhysicalPagesSloppy):
914 (bmalloc::Vector<T>::operator):
915 (bmalloc::Vector<T>::pop):
916 (bmalloc::Vector<T>::shrink):
917 * bmalloc/XLargeChunk.h:
918 (bmalloc::XLargeChunk::range):
919 (bmalloc::XLargeChunk::size):
921 2014-04-08 Geoffrey Garen <ggaren@apple.com>
923 Removed an unused file.
927 * bmalloc/AsyncTask.cpp: Removed.
929 2014-04-07 Geoffrey Garen <ggaren@apple.com>
932 https://bugs.webkit.org/show_bug.cgi?id=131333
934 Reviewed by Mark Rowe.
936 * Makefile: Added. For make clients.
938 These files are required for building any project in WebKit. I copied
940 * Configurations: Added.
941 * Configurations/Base.xcconfig: Added.
942 * Configurations/DebugRelease.xcconfig: Added.
943 * Configurations/bmalloc.xcconfig: Added.
944 * Configurations/iOS.xcconfig: Added.
945 * Configurations/mbmalloc.xcconfig: Added.
947 * bmalloc.xcodeproj/project.pbxproj: I removed per-project-file stuff
948 from here because everything is in .xcconfig files now.
950 I had to fix a bunch of minor warnings, since they're enabled in our
953 * bmalloc/AsyncTask.h:
954 (bmalloc::Function>::AsyncTask):
956 * bmalloc/BoundaryTagInlines.h:
959 (bmalloc::Heap::Heap):
960 (bmalloc::Heap::allocateLarge):
961 (bmalloc::Heap::deallocateLarge):
963 (bmalloc::Mutex::Mutex): Deleted.
964 * bmalloc/VMAllocate.h:
965 (bmalloc::vmValidate):
966 * bmalloc/mbmalloc.cpp:
968 2014-04-07 Geoffrey Garen <ggaren@apple.com>
970 bmalloc: Fixed a leak in the per-thread cache
971 https://bugs.webkit.org/show_bug.cgi?id=131330
973 Reviewed by Andreas Kling.
975 Remember to deallocate our line caches upon thread exit.
977 * bmalloc/Deallocator.cpp:
978 (bmalloc::Deallocator::~Deallocator):
980 2014-04-07 Geoffrey Garen <ggaren@apple.com>
982 bmalloc: rolled out the tryLock experiment
983 https://bugs.webkit.org/show_bug.cgi?id=131328
985 Reviewed by Andreas Kling.
989 * bmalloc.xcodeproj/project.pbxproj:
990 * bmalloc/Allocator.cpp:
991 (bmalloc::Allocator::processSmallAllocatorLog):
992 (bmalloc::Allocator::processMediumAllocatorLog):
993 * bmalloc/Deallocator.cpp:
994 (bmalloc::Deallocator::processObjectLog):
995 (bmalloc::Deallocator::deallocateSlowCase):
996 (bmalloc::Deallocator::deallocateSmallLine):
997 (bmalloc::Deallocator::deallocateMediumLine):
998 * bmalloc/Deallocator.h:
999 (bmalloc::Deallocator::deallocateFastCase):
1001 (bmalloc::Heap::deallocateSmallLine):
1002 (bmalloc::Heap::deallocateMediumLine):
1004 (bmalloc::Line<Traits>::deref):
1006 (bmalloc::Page<Traits>::deref):
1008 2014-04-07 Geoffrey Garen <ggaren@apple.com>
1011 https://bugs.webkit.org/show_bug.cgi?id=131170
1013 Reviewed by Andreas Kling.
1018 * bmalloc.xcodeproj: Added.
1019 * bmalloc.xcodeproj/project.pbxproj: Added.
1020 * bmalloc/Algorithm.h: Added.
1025 (bmalloc::roundUpToMultipleOf):
1026 (bmalloc::roundDownToMultipleOf):
1028 (bmalloc::bitCount):
1029 (bmalloc::isPowerOfTwo):
1030 * bmalloc/Allocator.cpp: Added.
1031 (bmalloc::Allocator::Allocator):
1032 (bmalloc::Allocator::~Allocator):
1033 (bmalloc::Allocator::log):
1034 (bmalloc::Allocator::processSmallAllocatorLog):
1035 (bmalloc::Allocator::processMediumAllocatorLog):
1036 (bmalloc::Allocator::allocateLarge):
1037 (bmalloc::Allocator::allocateXLarge):
1038 (bmalloc::Allocator::allocateMedium):
1039 (bmalloc::Allocator::allocateSlowCase):
1040 * bmalloc/Allocator.h: Added.
1041 (bmalloc::Allocator::smallAllocatorFor):
1042 (bmalloc::Allocator::allocateFastCase):
1043 (bmalloc::Allocator::allocate):
1044 * bmalloc/AsyncTask.cpp: Added.
1045 (bmalloc::AsyncTask<Function>::runSlowCase):
1046 (bmalloc::AsyncTask<Function>::pthreadEntryPoint):
1047 (bmalloc::AsyncTask<Function>::entryPoint):
1048 * bmalloc/AsyncTask.h: Added.
1049 (bmalloc::Function>::AsyncTask):
1050 (bmalloc::Function>::join):
1051 (bmalloc::Function>::run):
1052 (bmalloc::Function>::runSlowCase):
1053 (bmalloc::Function>::pthreadEntryPoint):
1054 (bmalloc::Function>::entryPoint):
1055 * bmalloc/BAssert.h: Added.
1056 * bmalloc/BeginTag.h: Added.
1057 (bmalloc::BeginTag::isInFreeList):
1058 * bmalloc/BoundaryTag.h: Added.
1059 (bmalloc::BoundaryTag::isXLarge):
1060 (bmalloc::BoundaryTag::setXLarge):
1061 (bmalloc::BoundaryTag::isFree):
1062 (bmalloc::BoundaryTag::setFree):
1063 (bmalloc::BoundaryTag::isEnd):
1064 (bmalloc::BoundaryTag::setEnd):
1065 (bmalloc::BoundaryTag::hasPhysicalPages):
1066 (bmalloc::BoundaryTag::setHasPhysicalPages):
1067 (bmalloc::BoundaryTag::isNull):
1068 (bmalloc::BoundaryTag::clear):
1069 (bmalloc::BoundaryTag::size):
1070 (bmalloc::BoundaryTag::setSize):
1071 (bmalloc::BoundaryTag::prev):
1072 (bmalloc::BoundaryTag::next):
1073 * bmalloc/BoundaryTagInlines.h: Added.
1074 (bmalloc::validate):
1075 (bmalloc::validatePrev):
1076 (bmalloc::validateNext):
1077 (bmalloc::BoundaryTag::init):
1078 (bmalloc::BoundaryTag::mergeLargeLeft):
1079 (bmalloc::BoundaryTag::mergeLargeRight):
1080 (bmalloc::BoundaryTag::mergeLarge):
1081 (bmalloc::BoundaryTag::deallocate):
1082 (bmalloc::BoundaryTag::splitLarge):
1083 (bmalloc::BoundaryTag::allocate):
1084 * bmalloc/Cache.cpp: Added.
1085 (bmalloc::Cache::operator new):
1086 (bmalloc::Cache::operator delete):
1087 (bmalloc::Cache::Cache):
1088 (bmalloc::Cache::allocateSlowCase):
1089 (bmalloc::Cache::allocateSlowCaseNullCache):
1090 (bmalloc::Cache::deallocateSlowCase):
1091 (bmalloc::Cache::deallocateSlowCaseNullCache):
1092 * bmalloc/Cache.h: Added.
1093 (bmalloc::Cache::allocator):
1094 (bmalloc::Cache::deallocator):
1095 (bmalloc::Cache::allocateFastCase):
1096 (bmalloc::Cache::deallocateFastCase):
1097 (bmalloc::Cache::allocate):
1098 (bmalloc::Cache::deallocate):
1099 * bmalloc/Chunk.h: Added.
1100 (bmalloc::Chunk::begin):
1101 (bmalloc::Chunk::end):
1102 (bmalloc::Chunk::lines):
1103 (bmalloc::Chunk::pages):
1104 * bmalloc/Deallocator.cpp: Added.
1105 (bmalloc::Deallocator::Deallocator):
1106 (bmalloc::Deallocator::~Deallocator):
1107 (bmalloc::Deallocator::deallocateLarge):
1108 (bmalloc::Deallocator::deallocateXLarge):
1109 (bmalloc::Deallocator::processObjectLog):
1110 (bmalloc::Deallocator::deallocateSlowCase):
1111 (bmalloc::Deallocator::deallocateSmallLine):
1112 (bmalloc::Deallocator::allocateSmallLine):
1113 (bmalloc::Deallocator::deallocateMediumLine):
1114 (bmalloc::Deallocator::allocateMediumLine):
1115 * bmalloc/Deallocator.h: Added.
1116 (bmalloc::Deallocator::deallocateFastCase):
1117 (bmalloc::Deallocator::deallocate):
1118 * bmalloc/EndTag.h: Added.
1119 (bmalloc::EndTag::operator=):
1120 * bmalloc/FixedVector.h: Added.
1121 (bmalloc::FixedVector::begin):
1122 (bmalloc::FixedVector::end):
1123 (bmalloc::FixedVector::size):
1124 (bmalloc::FixedVector::capacity):
1125 (bmalloc::FixedVector::clear):
1126 (bmalloc::FixedVector::isEmpty):
1127 (bmalloc::Capacity>::FixedVector):
1128 (bmalloc::Capacity>::operator):
1129 (bmalloc::Capacity>::push):
1130 (bmalloc::Capacity>::pop):
1131 (bmalloc::Capacity>::shrink):
1132 * bmalloc/Heap.cpp: Added.
1134 (bmalloc::Heap::Heap):
1135 (bmalloc::Heap::concurrentScavenge):
1136 (bmalloc::Heap::scavengeSmallPages):
1137 (bmalloc::Heap::scavengeMediumPages):
1138 (bmalloc::Heap::scavengeLargeRanges):
1139 (bmalloc::Heap::allocateSmallLineSlowCase):
1140 (bmalloc::Heap::allocateMediumLineSlowCase):
1141 (bmalloc::Heap::allocateXLarge):
1142 (bmalloc::Heap::deallocateXLarge):
1143 (bmalloc::Heap::allocateLarge):
1144 (bmalloc::Heap::deallocateLarge):
1145 * bmalloc/Heap.h: Added.
1146 (bmalloc::Heap::deallocateSmallLine):
1147 (bmalloc::Heap::allocateSmallLine):
1148 (bmalloc::Heap::deallocateMediumLine):
1149 (bmalloc::Heap::allocateMediumLine):
1150 * bmalloc/Inline.h: Added.
1151 * bmalloc/LargeChunk.h: Added.
1152 (bmalloc::LargeChunk::begin):
1153 (bmalloc::LargeChunk::end):
1154 (bmalloc::LargeChunk::create):
1155 (bmalloc::LargeChunk::get):
1156 (bmalloc::LargeChunk::beginTag):
1157 (bmalloc::LargeChunk::endTag):
1158 * bmalloc/Line.h: Added.
1159 (bmalloc::Line<Traits>::begin):
1160 (bmalloc::Line<Traits>::end):
1161 (bmalloc::Line<Traits>::concurrentRef):
1162 (bmalloc::Line<Traits>::deref):
1163 * bmalloc/MediumAllocator.h: Added.
1164 (bmalloc::MediumAllocator::isNull):
1165 (bmalloc::MediumAllocator::MediumAllocator):
1166 (bmalloc::MediumAllocator::line):
1167 (bmalloc::MediumAllocator::allocate):
1168 (bmalloc::MediumAllocator::derefCount):
1169 (bmalloc::MediumAllocator::refill):
1170 * bmalloc/MediumChunk.h: Added.
1171 * bmalloc/MediumLine.h: Added.
1172 * bmalloc/MediumPage.h: Added.
1173 * bmalloc/MediumTraits.h: Added.
1174 * bmalloc/Mutex.cpp: Added.
1175 (bmalloc::Mutex::lockSlowCase):
1176 * bmalloc/Mutex.h: Added.
1177 (bmalloc::Mutex::Mutex):
1178 (bmalloc::Mutex::try_lock):
1179 (bmalloc::Mutex::lock):
1180 (bmalloc::Mutex::unlock):
1181 * bmalloc/ObjectType.cpp: Added.
1182 (bmalloc::objectType):
1183 * bmalloc/ObjectType.h: Added.
1184 (bmalloc::isSmallOrMedium):
1186 * bmalloc/Page.h: Added.
1187 (bmalloc::Page<Traits>::ref):
1188 (bmalloc::Page<Traits>::deref):
1189 (bmalloc::Page<Traits>::refCount):
1190 * bmalloc/PerProcess.h: Added.
1191 (bmalloc::PerProcess::mutex):
1192 (bmalloc::PerProcess<T>::getFastCase):
1193 (bmalloc::PerProcess<T>::get):
1194 (bmalloc::PerProcess<T>::getSlowCase):
1195 * bmalloc/PerThread.h: Added.
1196 (bmalloc::PerThreadStorage<Cache>::get):
1197 (bmalloc::PerThreadStorage<Cache>::init):
1198 (bmalloc::PerThreadStorage::get):
1199 (bmalloc::PerThreadStorage::init):
1200 (bmalloc::PerThread<T>::getFastCase):
1201 (bmalloc::PerThread<T>::get):
1202 (bmalloc::PerThread<T>::destructor):
1203 (bmalloc::PerThread<T>::getSlowCase):
1204 * bmalloc/Range.h: Added.
1205 (bmalloc::Range::Range):
1206 (bmalloc::Range::begin):
1207 (bmalloc::Range::end):
1208 (bmalloc::Range::size):
1209 (bmalloc::Range::operator!):
1210 (bmalloc::Range::operator<):
1211 * bmalloc/SegregatedFreeList.cpp: Added.
1212 (bmalloc::SegregatedFreeList::SegregatedFreeList):
1213 (bmalloc::SegregatedFreeList::insert):
1214 (bmalloc::SegregatedFreeList::takeGreedy):
1215 (bmalloc::SegregatedFreeList::take):
1216 * bmalloc/SegregatedFreeList.h: Added.
1217 * bmalloc/Sizes.h: Added.
1218 * bmalloc/SmallAllocator.h: Added.
1219 (bmalloc::SmallAllocator::isNull):
1220 (bmalloc::SmallAllocator::canAllocate):
1221 (bmalloc::SmallAllocator::SmallAllocator):
1222 (bmalloc::SmallAllocator::line):
1223 (bmalloc::SmallAllocator::allocate):
1224 (bmalloc::SmallAllocator::objectCount):
1225 (bmalloc::SmallAllocator::derefCount):
1226 (bmalloc::SmallAllocator::refill):
1227 * bmalloc/SmallChunk.h: Added.
1228 * bmalloc/SmallLine.h: Added.
1229 * bmalloc/SmallPage.h: Added.
1230 * bmalloc/SmallTraits.h: Added.
1231 * bmalloc/Syscall.h: Added.
1232 * bmalloc/VMAllocate.h: Added.
1234 (bmalloc::vmValidate):
1235 (bmalloc::vmAllocate):
1236 (bmalloc::vmDeallocate):
1237 (bmalloc::vmDeallocatePhysicalPages):
1238 (bmalloc::vmAllocatePhysicalPages):
1239 (bmalloc::vmDeallocatePhysicalPagesSloppy):
1240 (bmalloc::vmAllocatePhysicalPagesSloppy):
1241 * bmalloc/VMHeap.cpp: Added.
1242 (bmalloc::VMHeap::VMHeap):
1243 (bmalloc::VMHeap::allocateSmallChunk):
1244 (bmalloc::VMHeap::allocateMediumChunk):
1245 (bmalloc::VMHeap::allocateLargeChunk):
1246 * bmalloc/VMHeap.h: Added.
1247 (bmalloc::VMHeap::allocateSmallPage):
1248 (bmalloc::VMHeap::allocateMediumPage):
1249 (bmalloc::VMHeap::allocateLargeRange):
1250 (bmalloc::VMHeap::deallocateSmallPage):
1251 (bmalloc::VMHeap::deallocateMediumPage):
1252 (bmalloc::VMHeap::deallocateLargeRange):
1253 * bmalloc/Vector.h: Added.
1254 (bmalloc::Vector::begin):
1255 (bmalloc::Vector::end):
1256 (bmalloc::Vector::size):
1257 (bmalloc::Vector::capacity):
1258 (bmalloc::Vector::last):
1259 (bmalloc::Vector::pop):
1260 (bmalloc::Vector<T>::Vector):
1261 (bmalloc::Vector<T>::~Vector):
1262 (bmalloc::Vector<T>::operator):
1263 (bmalloc::Vector<T>::push):
1264 (bmalloc::Vector<T>::pop):
1265 (bmalloc::Vector<T>::shrink):
1266 (bmalloc::Vector<T>::reallocateBuffer):
1267 (bmalloc::Vector<T>::shrinkCapacity):
1268 (bmalloc::Vector<T>::growCapacity):
1269 * bmalloc/XLargeChunk.h: Added.
1270 (bmalloc::XLargeChunk::get):
1271 (bmalloc::XLargeChunk::begin):
1272 (bmalloc::XLargeChunk::XLargeChunk):
1273 (bmalloc::XLargeChunk::create):
1274 (bmalloc::XLargeChunk::destroy):
1275 (bmalloc::XLargeChunk::range):
1276 (bmalloc::XLargeChunk::size):
1277 * bmalloc/bmalloc.h: Added.
1278 (bmalloc::api::malloc):
1279 (bmalloc::api::free):
1280 (bmalloc::api::realloc):
1281 * bmalloc/mbmalloc.cpp: Added.