fdac3724d0f18eb5481607c5090e3df6db99916d
[WebKit-https.git] / Source / bmalloc / ChangeLog
1 2015-02-27  Geoffrey Garen  <ggaren@apple.com>
2
3         bmalloc: Pathological madvise churn on the free(malloc(x)) benchmark
4         https://bugs.webkit.org/show_bug.cgi?id=142058
5
6         Reviewed by Andreas Kling.
7
8         The churn was caused by repeatedly splitting an object with physical
9         pages from an object without, and then merging them back together again.
10         The merge would conservatively forget that we had physical pages, forcing
11         a new call to madvise on the next allocation.
12
13         This patch more strictly segregates objects in the heap from objects in
14         the VM heap, with these changes:
15
16         (1) Objects in the heap are not allowed to merge with objects in the VM
17         heap, and vice versa -- since that would erase our precise knowledge of
18         which physical pages had been allocated.
19
20         (2) The VM heap is exclusively responsible for allocating and deallocating
21         physical pages.
22
23         (3) The heap free list must consider entries for objects that are in the
24         VM heap to be invalid, and vice versa. (This condition can arise
25         because the free list does not eagerly remove items.)
26
27         With these changes, we can know that any valid object in the heap's free
28         list already has physical pages, and does not need to call madvise.
29
30         Note that the VM heap -- as before -- might sometimes contain ranges
31         or pieces of ranges that have physical pages, since we allow splitting
32         of ranges at granularities smaller than the VM page size. These ranges
33         can eventually merge with ranges in the heap during scavenging.
34
35         * bmalloc.xcodeproj/project.pbxproj:
36
37         * bmalloc/BoundaryTag.h:
38         (bmalloc::BoundaryTag::owner):
39         (bmalloc::BoundaryTag::setOwner):
40         (bmalloc::BoundaryTag::initSentinel):
41         (bmalloc::BoundaryTag::hasPhysicalPages): Deleted.
42         (bmalloc::BoundaryTag::setHasPhysicalPages): Deleted. Replaced the concept
43         of "has physical pages" with a bit indicating which heap owns the large
44         object. This is a more precise concept, since the old bit was really a
45         Yes / Maybe bit.
46
47         * bmalloc/Deallocator.cpp:
48
49         * bmalloc/FreeList.cpp: Adopt
50         (bmalloc::FreeList::takeGreedy):
51         (bmalloc::FreeList::take):
52         (bmalloc::FreeList::removeInvalidAndDuplicateEntries):
53         * bmalloc/FreeList.h:
54         (bmalloc::FreeList::push): Added API for considering the owner when
55         deciding if a free list entry is valid.
56
57         * bmalloc/Heap.cpp:
58         (bmalloc::Heap::Heap): Adopt new API.
59
60         (bmalloc::Heap::scavengeLargeRanges): Scavenge all ranges with no minimum,
61         since some ranges might be able to merge with ranges in the VM heap, and
62         they won't be allowed to until we scavenge them.
63
64         (bmalloc::Heap::allocateSmallPage):
65         (bmalloc::Heap::allocateMediumPage):
66         (bmalloc::Heap::allocateLarge): New VM heap API makes this function
67         simpler, since we always get back physical pages now.
68
69         * bmalloc/Heap.h:
70         * bmalloc/LargeObject.h:
71         (bmalloc::LargeObject::end):
72         (bmalloc::LargeObject::owner):
73         (bmalloc::LargeObject::setOwner):
74         (bmalloc::LargeObject::isValidAndFree):
75         (bmalloc::LargeObject::merge): Do not merge objects across heaps since
76         that causes madvise churn.
77         (bmalloc::LargeObject::validateSelf):
78         (bmalloc::LargeObject::init):
79         (bmalloc::LargeObject::hasPhysicalPages): Deleted.
80         (bmalloc::LargeObject::setHasPhysicalPages): Deleted. Propogate the Owner API.
81
82         * bmalloc/Owner.h: Added.
83
84         * bmalloc/SegregatedFreeList.cpp:
85         (bmalloc::SegregatedFreeList::SegregatedFreeList):
86         (bmalloc::SegregatedFreeList::insert):
87         (bmalloc::SegregatedFreeList::takeGreedy):
88         (bmalloc::SegregatedFreeList::take):
89         * bmalloc/SegregatedFreeList.h: Propogate the owner API.
90
91         * bmalloc/VMAllocate.h:
92         (bmalloc::vmDeallocatePhysicalPagesSloppy):
93         (bmalloc::vmAllocatePhysicalPagesSloppy): Clarified these functions and
94         removed an edge case.
95
96         * bmalloc/VMHeap.cpp:
97         (bmalloc::VMHeap::VMHeap):
98         * bmalloc/VMHeap.h:
99         (bmalloc::VMHeap::allocateSmallPage):
100         (bmalloc::VMHeap::allocateMediumPage):
101         (bmalloc::VMHeap::allocateLargeObject):
102         (bmalloc::VMHeap::deallocateLargeObject): Be sure to give each object
103         a new chance to merge, since it might have been prohibited from merging
104         before by virtue of not being in the VM heap.
105
106         (bmalloc::VMHeap::allocateLargeRange): Deleted.
107         (bmalloc::VMHeap::deallocateLargeRange): Deleted.
108
109 2015-02-26  Geoffrey Garen  <ggaren@apple.com>
110
111         bmalloc: Large object free list can grow infinitely
112         https://bugs.webkit.org/show_bug.cgi?id=142055
113
114         Reviewed by Andreas Kling.
115
116         By design, we don't eagerly remove large objects from the free list.
117         This creates two simple pathologies:
118
119             (1) If you free and then allocate the same object repeatedly, it will
120             duplicate itself in the free list repeatedly. Since it is never
121             invalid at the time of allocation, it will never be removed.
122
123             (2) If you split and then merge the same object repeatedly, it will
124             duplicate its split sibling in the free list repeatedly. If its
125             sibling is in a separate free list size class, it will never be
126             consulted at the time of allocation, so it will never be removed.
127
128         So, a simple "while (1) { free(malloc(x)); }" causes infinite memory
129         use in the free list.
130
131         The solution in this patch is a simple helper to remove garbage from the
132         free list if it grows too large. This pathology is not common, so the
133         cost is OK.
134
135         Long-term, perhaps we should rethink the laziness of these free lists.
136
137         * bmalloc/BoundaryTag.h:
138         (bmalloc::BoundaryTag::isMarked):
139         (bmalloc::BoundaryTag::setMarked): New bit, used by free list GC.
140
141         * bmalloc/FreeList.cpp:
142         (bmalloc::FreeList::removeInvalidAndDuplicateEntries): The GC algorithm.
143
144         * bmalloc/FreeList.h:
145         (bmalloc::FreeList::FreeList):
146         (bmalloc::FreeList::push): Invoke the GC if we're getting huge.
147
148         * bmalloc/LargeObject.h:
149         (bmalloc::LargeObject::isMarked):
150         (bmalloc::LargeObject::setMarked):
151         (bmalloc::LargeObject::validateSelf): Expose the new bit.
152
153         * bmalloc/Sizes.h: New constant to control GC frequency.
154
155 2015-02-26  Csaba Osztrogonác  <ossy@webkit.org>
156
157         URTBF after r180693.
158
159         * CMakeLists.txt:
160
161 2015-02-26  Geoffrey Garen  <ggaren@apple.com>
162
163         Try to fix the Mac build.
164
165         Unreviewed.
166
167         * bmalloc.xcodeproj/project.pbxproj: Make FreeList.h available.
168
169 2015-02-26  Geoffrey Garen  <ggaren@apple.com>
170
171         bmalloc: Refactored SegregatedFreeList and BoundaryTag::init
172         https://bugs.webkit.org/show_bug.cgi?id=142049
173
174         Reviewed by Anders Carlsson.
175
176         Split out a FreeList class from SegregatedFreeList. This will make it
177         easier to add behaviors on free list insertion and removal -- and it's
178         probably how I should have designed things at the start.
179
180         Moved BoundaryTag::init into LargeObject, since all the related logic
181         lives in LargeObject now too, and this allows us to remove BoundaryTagInlines.h.
182
183         * bmalloc.xcodeproj/project.pbxproj:
184         * bmalloc/BoundaryTagInlines.h: Removed.
185         * bmalloc/FreeList.cpp: Copied from Source/bmalloc/bmalloc/SegregatedFreeList.cpp.
186         (bmalloc::FreeList::takeGreedy):
187         (bmalloc::FreeList::take):
188         (bmalloc::SegregatedFreeList::SegregatedFreeList): Deleted.
189         (bmalloc::SegregatedFreeList::insert): Deleted.
190         (bmalloc::SegregatedFreeList::takeGreedy): Deleted.
191         (bmalloc::SegregatedFreeList::take): Deleted.
192         * bmalloc/FreeList.h: Copied from Source/bmalloc/bmalloc/SegregatedFreeList.h.
193         (bmalloc::FreeList::push):
194         * bmalloc/LargeObject.h:
195         (bmalloc::LargeObject::init):
196         * bmalloc/SegregatedFreeList.cpp:
197         (bmalloc::SegregatedFreeList::SegregatedFreeList):
198         (bmalloc::SegregatedFreeList::insert):
199         (bmalloc::SegregatedFreeList::takeGreedy):
200         (bmalloc::SegregatedFreeList::take):
201         * bmalloc/SegregatedFreeList.h:
202         * bmalloc/Sizes.h:
203         * bmalloc/VMHeap.cpp:
204         (bmalloc::VMHeap::grow):
205
206 2015-02-26  Geoffrey Garen  <ggaren@apple.com>
207
208         bmalloc: free up a bit in BoundaryTag
209         https://bugs.webkit.org/show_bug.cgi?id=142048
210
211         Reviewed by Brady Eidson.
212
213         We were wasting a bit by accident, and I need one now.
214
215         * bmalloc/Algorithm.h:
216         (bmalloc::rightShift): Deleted. Not needed, now that I've simplified
217         the math.
218
219         * bmalloc/BoundaryTag.h: Since each boundary tag bucket is 1024 bytes
220         long, the maximum offset into a bucket is 1023.
221
222         You need 5 bits to count up to 1024, but only 4 to count up to 1023.
223         
224         Math is hard.
225
226         (bmalloc::BoundaryTag::compactBegin): Switched to division because it
227         is simpler, and easier to match up with our ASSERT. The compiler will
228         turn division by constant power of two into a shift for us.
229
230         (bmalloc::BoundaryTag::setRange): Added an ASSERT for compactBegin
231         because we do encode it, so we should ASSERT that encoding did not
232         lose information.
233
234         * bmalloc/Sizes.h: Shifting is no longer used since we use division
235         instead.
236
237 2015-02-24  Stephanie Lewis  <slewis@apple.com>
238
239         Rolling out http://trac.webkit.org/changeset/180430 as it causes the PLT to crash.
240         <rdar://problem/19948015>
241
242         Unreviewed.
243
244         * bmalloc/VMHeap.cpp:
245         (bmalloc::VMHeap::grow):
246         * bmalloc/VMHeap.h:
247         * bmalloc/Zone.cpp:
248         (bmalloc::Zone::Zone):
249         (bmalloc::Zone::size): Deleted.
250         * bmalloc/Zone.h:
251
252 2015-02-24  Geoffrey Garen  <ggaren@apple.com>
253
254         bmalloc: Added a little more abstraction for large objects
255         https://bugs.webkit.org/show_bug.cgi?id=141978
256
257         Reviewed by Sam Weinig.
258
259         Previously, each client needed to manage the boundary tags of
260         a large object using free functions. This patch introduces a LargeObject
261         class that does things a little more automatically.
262
263         * bmalloc.xcodeproj/project.pbxproj:
264
265         * bmalloc/Allocator.cpp:
266         (bmalloc::Allocator::reallocate): Use the new LargeObject class.
267
268         * bmalloc/BeginTag.h:
269         (bmalloc::BeginTag::isInFreeList): Deleted. Moved this logic into the
270         LargeObject class.
271
272         * bmalloc/BoundaryTag.h:
273         (bmalloc::BoundaryTag::isSentinel):
274         (bmalloc::BoundaryTag::compactBegin):
275         (bmalloc::BoundaryTag::setRange):
276         (bmalloc::BoundaryTag::initSentinel): Added an explicit API for sentinels,
277         which we used to create and test for implicitly.
278
279         * bmalloc/BoundaryTagInlines.h:
280         (bmalloc::BoundaryTag::init):
281         (bmalloc::validate): Deleted.
282         (bmalloc::validatePrev): Deleted.
283         (bmalloc::validateNext): Deleted.
284         (bmalloc::BoundaryTag::mergeLeft): Deleted.
285         (bmalloc::BoundaryTag::mergeRight): Deleted.
286         (bmalloc::BoundaryTag::merge): Deleted.
287         (bmalloc::BoundaryTag::deallocate): Deleted.
288         (bmalloc::BoundaryTag::split): Deleted.
289         (bmalloc::BoundaryTag::allocate): Deleted. Moved this logic into the
290         LargeObject class.
291
292         * bmalloc/EndTag.h:
293         (bmalloc::EndTag::init):
294         (bmalloc::EndTag::operator=): Deleted. Re-reading this code, I found
295         special behavior in the assignment operator to be a surprising API.
296         So, I replaced the assignment operation with an explicit initializing
297         function.
298
299         * bmalloc/Heap.cpp:
300         (bmalloc::Heap::scavengeLargeRanges):
301         (bmalloc::Heap::allocateXLarge):
302         (bmalloc::Heap::findXLarge):
303         (bmalloc::Heap::deallocateXLarge):
304         (bmalloc::Heap::allocateLarge):
305         (bmalloc::Heap::deallocateLarge):
306         * bmalloc/Heap.h: No behavior changes here -- just adopting the
307         LargeObject interface.
308
309         * bmalloc/LargeObject.h: Added.
310         (bmalloc::LargeObject::operator!):
311         (bmalloc::LargeObject::begin):
312         (bmalloc::LargeObject::size):
313         (bmalloc::LargeObject::range):
314         (bmalloc::LargeObject::LargeObject):
315         (bmalloc::LargeObject::setFree):
316         (bmalloc::LargeObject::isFree):
317         (bmalloc::LargeObject::hasPhysicalPages):
318         (bmalloc::LargeObject::setHasPhysicalPages):
319         (bmalloc::LargeObject::isValidAndFree):
320         (bmalloc::LargeObject::merge):
321         (bmalloc::LargeObject::split):
322         (bmalloc::LargeObject::validateSelf):
323         (bmalloc::LargeObject::validate): Moved this code into a class, out of
324         BoundaryTag free functions.
325
326         New to the class are these features:
327
328             (1) Every reference to an object is validated upon creation and use.
329
330             (2) There's an explicit API for "This is a reference to an object
331             that might be stale (the DoNotValidate API)".
332
333             (3) The begin and end tags are kept in sync automatically.
334
335         * bmalloc/SegregatedFreeList.cpp:
336         (bmalloc::SegregatedFreeList::insert):
337         (bmalloc::SegregatedFreeList::takeGreedy):
338         (bmalloc::SegregatedFreeList::take):
339         * bmalloc/SegregatedFreeList.h: Adopt the LargeObject interface.
340
341         * bmalloc/VMHeap.cpp:
342         (bmalloc::VMHeap::grow):
343         * bmalloc/VMHeap.h:
344         (bmalloc::VMHeap::allocateLargeRange):
345         (bmalloc::VMHeap::deallocateLargeRange): Adopt the LargeObject interface.
346
347 2015-02-20  Geoffrey Garen  <ggaren@apple.com>
348
349         bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
350         https://bugs.webkit.org/show_bug.cgi?id=141802
351
352         Reviewed by Andreas Kling.
353
354         Rolling back in with a fix for a crash seen while using GuardMalloc.
355
356         * bmalloc/VMHeap.cpp:
357         (bmalloc::VMHeap::grow):
358         * bmalloc/VMHeap.h:
359         * bmalloc/Zone.cpp: Re-land the old patch.
360
361         (bmalloc::Zone::size): Be sure to implement the size() function since
362         it's accessible indirectly via the malloc_zone_from_ptr public API --
363         and GuardMalloc calls it all the time.
364
365         (bmalloc::Zone::Zone):
366         * bmalloc/Zone.h: Re-land the old patch.
367
368 2015-02-19  Commit Queue  <commit-queue@webkit.org>
369
370         Unreviewed, rolling out r180363.
371         https://bugs.webkit.org/show_bug.cgi?id=141814
372
373         Caused >50 crashes when running LayoutTests in GuardMalloc or
374         ASAN modes. (Requested by jernoble on #webkit).
375
376         Reverted changeset:
377
378         "bmalloc should implement malloc introspection (to stop false-
379         positive leaks when MallocStackLogging is off)"
380         https://bugs.webkit.org/show_bug.cgi?id=141802
381         http://trac.webkit.org/changeset/180363
382
383 2015-02-19  Geoffrey Garen  <ggaren@apple.com>
384
385         bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
386         https://bugs.webkit.org/show_bug.cgi?id=141802
387
388         Reviewed by Andreas Kling.
389
390         Fixed a last-minute type.
391
392         The macro is OS, not PLATFORM.
393
394         * bmalloc/VMHeap.cpp:
395         (bmalloc::VMHeap::grow):
396         * bmalloc/VMHeap.h:
397         * bmalloc/Zone.h:
398
399 2015-02-19  Geoffrey Garen  <ggaren@apple.com>
400
401         bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
402         https://bugs.webkit.org/show_bug.cgi?id=141802
403
404         Reviewed by Andreas Kling.
405
406         This patch does the bare minimum to stop false positive leaks from
407         being reported by the Darwin leaks tool. We register each super chunk
408         as a single object, and then request that the leaks tool scan it.
409
410         * bmalloc.xcodeproj/project.pbxproj: Added an abstraction for the malloc
411         zone introspection API.
412
413         * bmalloc/Algorithm.h: Missing #include.
414
415         * bmalloc/VMHeap.cpp:
416         (bmalloc::VMHeap::grow):
417         * bmalloc/VMHeap.h: Adopt the new abstraction.
418
419         * bmalloc/Zone.cpp: Added.
420         (bmalloc::remoteRead): Helper for reading an object out of another process.
421         (bmalloc::Zone::enumerator):
422         (bmalloc::Zone::Zone): Register a malloc zone so that we will participate
423         in introspection.
424
425         * bmalloc/Zone.h: Added.
426         (bmalloc::Zone::superChunks):
427         (bmalloc::Zone::addSuperChunk): Use a non-dynamically-allocated vector
428         since our dynamic allocations will not be scanned by leaks since they
429         will have the malloc VM tag.
430
431 2015-02-18  Geoffrey Garen  <ggaren@apple.com>
432
433         bmalloc: VMHeap should keep a record of all of its VM ranges (for malloc introspection)
434         https://bugs.webkit.org/show_bug.cgi?id=141759
435
436         Reviewed by Andreas Kling.
437
438         * bmalloc.xcodeproj/project.pbxproj:
439         * bmalloc/SuperChunk.h: Added.
440         (bmalloc::SuperChunk::create):
441         (bmalloc::SuperChunk::SuperChunk):
442         (bmalloc::SuperChunk::smallChunk):
443         (bmalloc::SuperChunk::mediumChunk):
444         (bmalloc::SuperChunk::largeChunk): Factored out super chunk creation
445         into a separate class, for clarity and type safety.
446
447         * bmalloc/VMHeap.cpp:
448         (bmalloc::VMHeap::grow):
449         (bmalloc::VMHeap::allocateSuperChunk): Renamed "allocateSuperChunk" to
450         "grow" because Andreas found "allocateSuperChunk" to be unclear.
451
452         * bmalloc/VMHeap.h: Track all our VM ranges. We will use this information
453         for malloc introspection.
454
455         (bmalloc::VMHeap::allocateSmallPage):
456         (bmalloc::VMHeap::allocateMediumPage):
457         (bmalloc::VMHeap::allocateLargeRange): Updated for renames.
458
459 2015-02-18  Zan Dobersek  <zdobersek@igalia.com>
460
461         Build bmalloc through CMake as a static library. It's then linked either
462         into the WTF library (if built as a shared library) or into the JSC and
463         WebKit2 libraries. There's no need to build it as a standalone shared library.
464
465         Rubber-stamped by Carlos Garcia Campos.
466
467         * CMakeLists.txt:
468
469 2015-02-13  Gyuyoung Kim  <gyuyoung.kim@samsung.com>
470
471         [BMalloc] Add a FIXME comment for memory alignas
472         https://bugs.webkit.org/show_bug.cgi?id=141556
473
474         Reviewed by Csaba Osztrogonác.
475
476         * bmalloc/Chunk.h: Add a FIXME comment.
477         * bmalloc/LargeChunk.h: ditto.
478
479 2015-02-11  Csaba Osztrogonác  <ossy@webkit.org>
480
481         bmalloc buildfix on 32 bit Linux (x86/ARM)
482         https://bugs.webkit.org/show_bug.cgi?id=141472
483
484         Reviewed by Gyuyoung Kim.
485
486         * bmalloc/Algorithm.h:
487         (bmalloc::roundUpToMultipleOf):
488         * bmalloc/FixedVector.h:
489         (bmalloc::FixedVector::clear):
490         * bmalloc/Sizes.h:
491         (bmalloc::Sizes::sizeClass):
492
493 2015-02-11  Gyuyoung Kim  <gyuyoung.kim@samsung.com>
494
495         [EFL][GTK] Use bmalloc instead of tcmalloc
496         https://bugs.webkit.org/show_bug.cgi?id=140162
497
498         Reviewed by Carlos Garcia Campos.
499
500         Support to use bmalloc on EFL and GTK ports.
501
502         * CMakeLists.txt: Added.
503         * bmalloc/Allocator.cpp:
504         (bmalloc::Allocator::allocate):
505             Fix unused return value caused by posix_memalign().
506         * bmalloc/AsyncTask.h:
507         * bmalloc/BoundaryTag.h:
508         (bmalloc::BoundaryTag::clear):
509         * bmalloc/Chunk.h:
510             Change Traits::Page with Traits::PageType in order to fix
511             -fpermitive build error on EFL and GTK port. 
512         * bmalloc/EndTag.h:
513         (bmalloc::EndTag::operator=):
514         * bmalloc/Line.h: ditto.
515         * bmalloc/MediumTraits.h:
516         * bmalloc/Page.h: ditto.
517         * bmalloc/PerThread.h:
518             EFL port doesn't support __has_include definition yet.
519             Define HAVE_PTHREAD_MACHDEP_H according to check if __has_include is supported.
520         * bmalloc/SmallTraits.h: ditto.
521         * bmalloc/VMAllocate.h:
522         (bmalloc::vmDeallocatePhysicalPages):
523         (bmalloc::vmAllocatePhysicalPages):
524         * bmalloc/Vector.h:
525         (bmalloc::Vector<T>::push):
526         (bmalloc::Vector<T>::reallocateBuffer):
527
528 2015-01-31  Sam Weinig  <sam@webkit.org>
529
530         Remove even more Mountain Lion support
531         https://bugs.webkit.org/show_bug.cgi?id=141124
532
533         Reviewed by Alexey Proskuryakov.
534
535         * Configurations/Base.xcconfig:
536         * Configurations/DebugRelease.xcconfig:
537
538 2015-01-30  Geoffrey Garen  <ggaren@apple.com>
539
540         GC marking threads should clear malloc caches
541         https://bugs.webkit.org/show_bug.cgi?id=141097
542
543         Reviewed by Andreas Kling.
544
545         Split the scavenging API into per-thread vs global, so that you can
546         request to scavenge your own thread without scavenging the whole heap.
547
548         * bmalloc/Cache.cpp:
549         (bmalloc::Cache::scavenge):
550         * bmalloc/bmalloc.h:
551         (bmalloc::api::scavengeThisThread):
552         (bmalloc::api::scavenge):
553
554 2015-01-28  Dana Burkart  <dburkart@apple.com>
555
556         Move ASan flag settings from DebugRelease.xcconfig to Base.xcconfig
557         https://bugs.webkit.org/show_bug.cgi?id=136765
558
559         Reviewed by Alexey Proskuryakov.
560
561         * Configurations/Base.xcconfig:
562         * Configurations/DebugRelease.xcconfig:
563
564 2015-01-21  Geoffrey Garen  <ggaren@apple.com>
565
566         bmalloc: support aligned allocation
567         https://bugs.webkit.org/show_bug.cgi?id=140732
568
569         Reviewed by Andreas Kling.
570
571         * bmalloc/Allocator.cpp:
572         (bmalloc::Allocator::allocate): New function for aligned allocation.
573
574         Small and medium requests just allocate and free until they find an
575         aligned pointer. This is slightly inefficient in the worst case, but
576         still constant-time with little-to-no space overhead.
577
578         Large requests use a new API that requires the client to specify both
579         its ideal size and alignment, and the worst-case size you would have to
580         allocate in order to produce some interior pointer of the requested size
581         and alignment. We put the burden of this calculation on the client
582         because it simplifies things if we guarantee that allocation won't fail.
583
584         XLarge requests are easy: we just forward them to vmAllocate, which
585         already supported aligned requests.
586
587         * bmalloc/BoundaryTag.h:
588         * bmalloc/BoundaryTagInlines.h:
589         (bmalloc::BoundaryTag::mergeLeft):
590         (bmalloc::BoundaryTag::mergeRight):
591         (bmalloc::BoundaryTag::merge):
592         (bmalloc::BoundaryTag::deallocate):
593         (bmalloc::BoundaryTag::split):
594         (bmalloc::BoundaryTag::allocate): No behavior change here. I just
595         refactored the interface to remove some reference out parameters in
596         order to clarify what changes and what doesn't.
597
598         * bmalloc/Heap.cpp:
599         (bmalloc::Heap::allocateXLarge): Added an alignment API.
600
601         (bmalloc::Heap::allocateLarge):
602         * bmalloc/Heap.h: Added an alignment API. I split out allocateLarge into
603         a few variants, so aligned and unaligned allocation could share some code.
604
605         * bmalloc/SegregatedFreeList.cpp:
606         (bmalloc::SegregatedFreeList::take):
607         * bmalloc/SegregatedFreeList.h: Changed to use a separate, explicit API
608         for aligned allocation. It turns out that the aligned path is pretty
609         different, since it ends up searching for two potential ways to satisfy
610         an allocation: either large enough and aligned, or large enough to split
611         into something not aligned and something large enough and aligned.
612
613         * bmalloc/VMAllocate.h:
614         (bmalloc::vmAllocate): Switched alignment to come before size because
615         that's how the memalign API specifies it.
616
617         * bmalloc/VMHeap.h:
618         (bmalloc::VMHeap::allocateLargeRange): Added an alignment API.
619
620 2015-01-20  Geoffrey Garen  <ggaren@apple.com>
621
622         bmalloc: a little bit of cleanup
623         https://bugs.webkit.org/show_bug.cgi?id=140687
624
625         Reviewed by Anders Carlsson.
626
627         * bmalloc/Algorithm.h:
628         (bmalloc::isPowerOfTwo): Added a check for 0, since 0 would break a lot
629         of code.
630
631         * bmalloc/BoundaryTag.h:
632         * bmalloc/BoundaryTagInlines.h:
633         (bmalloc::BoundaryTag::mergeLeft):
634         (bmalloc::BoundaryTag::mergeRight):
635         (bmalloc::BoundaryTag::merge):
636         (bmalloc::BoundaryTag::deallocate):
637         (bmalloc::BoundaryTag::split):
638         (bmalloc::BoundaryTag::allocate):
639         (bmalloc::BoundaryTag::mergeLargeLeft): Deleted.
640         (bmalloc::BoundaryTag::mergeLargeRight): Deleted.
641         (bmalloc::BoundaryTag::mergeLarge): Deleted.
642         (bmalloc::BoundaryTag::splitLarge): Deleted. Removed the word "Large"
643         from all these functions, since boundary tags always pertain to large
644         objects, and putting the word "Large" everywhere wasn't helping to
645         explain that.
646
647         * bmalloc/Heap.cpp:
648         (bmalloc::Heap::allocateXLarge):
649         (bmalloc::Heap::findXLarge):
650         (bmalloc::Heap::deallocateXLarge):
651         * bmalloc/Heap.h:
652         * bmalloc/VMHeap.h:
653         (bmalloc::VMHeap::allocateXLarge): Deleted.
654         (bmalloc::VMHeap::findXLarge): Deleted.
655         (bmalloc::VMHeap::deallocateXLarge): Deleted. Moved XLarge allocation
656         from VMHeap to Heap. Since the purpose of the VMHeap is to cache VM
657         ranges, and the VMHeap never caches any XLarge ranges, it doesn't
658         really make sense for the VMHeap to be involved.
659
660 2015-01-16  Geoffrey Garen  <ggaren@apple.com>
661
662         bmalloc: refactored XLarge allocation for better alignment
663         https://bugs.webkit.org/show_bug.cgi?id=140582
664
665         Reviewed by Andreas Kling.
666
667         XLarge objects used to be Large objects with an extra bit of metadata
668         that said "actually, I'm not large -- I'm extra large".
669
670         The metadata header in an XLarge allocation made it impossible for the
671         XLarge object to honor a very large alignment request.
672
673         The solution is to stop using a metadata header for XLarge objects, and
674         instead to store explicit metadata on the side.
675
676         This is a bit less astonishing, which is also nice.
677
678         Finding XLarge metadata is now a linear search. That's probably OK, since
679         it was always so in TCMalloc, and the usual number of XLarge allocations
680         in a process is 0.
681
682         This design makes it possible for the heap to cache XLarge allocations
683         with and/or without physical pages. I haven't actually done that yet
684         because the tradeoffs are subtle, so I don't want to do anything without
685         a motivating test case.
686
687         * bmalloc.xcodeproj/project.pbxproj:
688         * bmalloc/Allocator.cpp:
689         (bmalloc::Allocator::reallocate): Removed the concept of an XLargeChunk,
690         since an XLarge allocation is now just a naked buffer without a header.
691
692         (bmalloc::Allocator::allocateXLarge): Added an explicit qualifier for
693         XLarge alignment, since XLargeChunk won't give this to us implicitly
694         anymore.
695
696         * bmalloc/BoundaryTag.h:
697         (bmalloc::BoundaryTag::setRange):
698         (bmalloc::BoundaryTag::isXLarge): Deleted.
699         (bmalloc::BoundaryTag::setXLarge): Deleted.
700         * bmalloc/BoundaryTagInlines.h:
701         (bmalloc::validate):
702         (bmalloc::BoundaryTag::deallocate): Removed the XLarge hacks from Large allocations.
703
704         * bmalloc/Deallocator.cpp:
705         (bmalloc::Deallocator::deallocateXLarge):
706         (bmalloc::Deallocator::deallocateSlowCase):
707         * bmalloc/Heap.cpp:
708         (bmalloc::Heap::findXLarge):
709         (bmalloc::Heap::allocateXLarge):
710         (bmalloc::Heap::deallocateXLarge):
711         * bmalloc/Heap.h: Updated for interface changes.
712
713         * bmalloc/ObjectType.cpp:
714         (bmalloc::objectType):
715         * bmalloc/ObjectType.h:
716         (bmalloc::isXLarge): We can now tell if a pointer is XLarge just by
717         examining its bit pattern -- just like we do for other kinds of
718         allocations -- which is nice.
719
720         * bmalloc/Sizes.h:
721         * bmalloc/VMHeap.h:
722         (bmalloc::VMHeap::allocateXLarge):
723         (bmalloc::VMHeap::findXLarge):
724         (bmalloc::VMHeap::deallocateXLarge): Keep an explicit vector of metadata
725         for XLarge allocations.
726
727         * bmalloc/XLargeChunk.h: Removed.
728
729 2015-01-16  Geoffrey Garen  <ggaren@apple.com>
730
731         bmalloc: added some infrastructure for aligned allocation
732         https://bugs.webkit.org/show_bug.cgi?id=140572
733
734         Reviewed by Andreas Kling.
735
736         * bmalloc/Algorithm.h:
737         (bmalloc::isPowerOfTwo):
738         (bmalloc::roundUpToMultipleOf):
739         (bmalloc::roundDownToMultipleOf): Refactored some duplicate code to use our
740         isPowerOfTwo helper function.
741
742         * bmalloc/Allocator.cpp:
743         (bmalloc::Allocator::allocate):
744         * bmalloc/Allocator.h: Stubbed out an implementation of aligned allocation.
745         Doesn't do anything yet, but does correctly forward to system malloc
746         when bmalloc is disabled.
747
748         * bmalloc/Cache.cpp:
749         (bmalloc::Cache::allocateSlowCaseNullCache):
750         * bmalloc/Cache.h:
751         (bmalloc::Cache::allocate):
752         * bmalloc/bmalloc.h:
753         (bmalloc::api::memalign):
754         * bmalloc/mbmalloc.cpp: Stubbed out an API for aligned allocation.
755
756 2015-01-13  Geoffrey Garen  <ggaren@apple.com>
757
758         Consider alignment when allocating from a SegregatedFreeList
759         https://bugs.webkit.org/show_bug.cgi?id=140408
760
761         Reviewed by Sam Weinig.
762
763         In preparation for supporting aligned allocation.
764
765         No performance change.
766
767         Since this is just one extra branch in an already expensive function,
768         I decided not to duplicate the function just to avoid the branch in
769         the un-aligned case.
770
771         * bmalloc/SegregatedFreeList.cpp:
772         (bmalloc::SegregatedFreeList::take):
773         * bmalloc/SegregatedFreeList.h:
774
775 2015-01-13  Geoffrey Garen  <ggaren@apple.com>
776
777         Renamed minimum to size in SegregatedFreeList
778         https://bugs.webkit.org/show_bug.cgi?id=140406
779
780         Reviewed by Sam Weinig.
781
782         In preparation for supporting aligned allocation.
783
784         * bmalloc/SegregatedFreeList.cpp:
785         (bmalloc::SegregatedFreeList::takeGreedy):
786         (bmalloc::SegregatedFreeList::take): Every size passed to malloc is
787         really just a minimum. Let's not imply that this value is special.
788
789 2015-01-11  Dan Bernstein  <mitz@apple.com>
790
791         Geoff is organized, but he is not an organization.
792
793         Rubber-stamped by Anders Carlsson.
794
795         * bmalloc.xcodeproj/project.pbxproj: Removed the ORGANIZATIONNAME project attribute.
796
797 2015-01-07  Geoffrey Garen  <ggaren@apple.com>
798
799         Make bmalloc work with ASan
800         https://bugs.webkit.org/show_bug.cgi?id=140194
801
802         Reviewed by Mark Lam.
803
804         * bmalloc/BPlatform.h: Added a way to detect Darwin OSes, since we need
805         an OS-specific API to test for loaded runtime libraries.
806
807         * bmalloc/Environment.cpp:
808         (bmalloc::isASanEnabled):
809         (bmalloc::Environment::computeIsBmallocEnabled): Disabled bmalloc if
810         ASan is enabled, since system malloc has the Asan hooks we need.
811
812         You could check for the ASan compile-time flag instead, but doing this
813         check at runtime prepares bmalloc for a world where it is a dynamic
814         library that might be loaded into projects it did not compile with.
815
816 2015-01-05  Geoffrey Garen  <ggaren@apple.com>
817
818         Fix up bmalloc's PerThread for use on Linux
819         https://bugs.webkit.org/show_bug.cgi?id=139804
820
821         Reviewed by Anders Carlsson.
822
823         The previous implementation was a bit slow.
824
825         * bmalloc/PerThread.h:
826         (bmalloc::PerThreadStorage<Cache>::get):
827         (bmalloc::PerThreadStorage::get):
828         (bmalloc::PerThreadStorage::init): Added a catch-all cross-platform Unix
829         way to do fast per-thread access without taking a lock every time. This
830         probably works on all the platforms we care about, and it matches other
831         techniques we use elsewhere in WebKit.
832
833         (bmalloc::PerThread<T>::getFastCase): Removed the conditional from
834         this class because PerThreadStorage now encapsulates everything that
835         needs to be conditional.
836
837         (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Deleted.
838
839 2014-12-26  Dan Bernstein  <mitz@apple.com>
840
841         <rdar://problem/19348208> REGRESSION (r177027): iOS builds use the wrong toolchain
842         https://bugs.webkit.org/show_bug.cgi?id=139950
843
844         Reviewed by David Kilzer.
845
846         * Configurations/Base.xcconfig: Only define TOOLCHAINS when building for OS X, doing so
847         in a manner that works with Xcode 5.1.1.
848
849 2014-12-15  Geoffrey Garen  <ggaren@apple.com>
850
851         Safari crashes when you set Malloc environment variables
852         https://bugs.webkit.org/show_bug.cgi?id=139656
853
854         Reviewed by Michael Saboff.
855
856         I forgot to cover the realloc() case. Whoops. (OoPS?)
857
858         This time around, I ran the full MallocBench test suite in Malloc=1
859         mode, and it passed.
860
861         * bmalloc/Allocator.cpp:
862         (bmalloc::Allocator::reallocate):
863         * bmalloc/Allocator.h: Pushed realloc() logic down into the allocator.
864         It needs to be down there so that we can do the short-circuiting check
865         for whether bmalloc is enabled first.
866
867         Also added the check.
868
869         * bmalloc/Cache.cpp:
870         (bmalloc::Cache::scavenge):
871         (bmalloc::Cache::Cache):
872         (bmalloc::Cache::reallocateSlowCaseNullCache):
873         * bmalloc/Cache.h:
874         (bmalloc::Cache::deallocator):
875         (bmalloc::Cache::reallocate): Ditto.
876
877         * bmalloc/bmalloc.h:
878         (bmalloc::api::free):
879         (bmalloc::api::realloc): Ditto.
880
881         (bmalloc::api::scavenge): Pushed this down into Cache to match the
882         surrounding functions.
883
884 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
885
886         bmalloc should support system memory analysis tools (part 2)
887         https://bugs.webkit.org/show_bug.cgi?id=139565
888
889         Reviewed by Mark Lam.
890
891         This patch actually queries the environment to see if memory analysis
892         tools have been enabled.
893
894         * bmalloc/Deallocator.cpp:
895         (bmalloc::Deallocator::scavenge): Don't process the object log if
896         we've disabled bmalloc because it will be full of invalid nullptrs.
897
898         * bmalloc/Environment.cpp:
899         (bmalloc::isMallocEnvironmentVariableSet): Test for the list of known
900         Malloc debugging flags. I also added a plain "Malloc" catch-all for
901         when you want to disable bmalloc without enabling any kind of funny
902         business.
903
904         It would be slightly nicer just to iterate the list of environment
905         variables and strstr them, but getenv is the more portable option,
906         and performance here doesn't really matter.
907
908         (bmalloc::isLibgmallocEnabled): Test for the libgmalloc insertion
909         environment variable.
910
911         (bmalloc::Environment::computeIsBmallocEnabled):
912
913 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
914
915         Try to fix the iOS simulator build.
916
917         #include the declaration of malloc / free.
918
919         * bmalloc/Allocator.cpp:
920         * bmalloc/Deallocator.cpp:
921
922 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
923
924         Try to fix the build.
925
926         * bmalloc.xcodeproj/project.pbxproj: Marked a header exported.
927
928 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
929
930         bmalloc should support system memory analysis tools (part 1)
931         https://bugs.webkit.org/show_bug.cgi?id=139559
932
933         Reviewed by Mark Lam.
934
935         This patch adds the hooks to disable bmalloc at runtime if certain
936         environment variables are set, but doesn't actually read from the
937         environment yet.
938
939         No performance change.
940
941         * bmalloc.xcodeproj/project.pbxproj: Added the Environment class, which
942         we'll use to read environment variables and see if memory analysis tools
943         have been enabled.
944
945         * bmalloc/Allocator.cpp:
946         (bmalloc::Allocator::Allocator):
947         (bmalloc::Allocator::allocateSlowCase): Added a hook to disable bmalloc
948         on the allocation path. We cache the setting to make the check fast.
949
950         * bmalloc/Allocator.h: Interface changes.
951
952         * bmalloc/Cache.cpp:
953         (bmalloc::Cache::Cache): Pass a heap pointer through to our allocator
954         and deallocator. This main purpose is to enable them to query the
955         environment for whether bmalloc is enabled; but this is also a slightly
956         cleaner way to guarantee to them that the Heap has been pre-initialized.
957
958         * bmalloc/Deallocator.cpp:
959         (bmalloc::Deallocator::Deallocator): If bmalloc is disable, artificially
960         fill the object log to force us to take the slow path on all deallocations.
961
962         (bmalloc::Deallocator::deallocateSlowCase): Do the disabled check.
963
964         * bmalloc/Deallocator.h: Interface changes.
965
966         * bmalloc/Environment.cpp: Added.
967         (bmalloc::Environment::Environment):
968         (bmalloc::Environment::computeIsBmallocEnabled):
969         * bmalloc/Environment.h: Added.
970         (bmalloc::Environment::isBmallocEnabled): This is the class that will
971         encapsulate looking for environment variables that turn on heap
972         analysis tools.
973
974         * bmalloc/Heap.h:
975         (bmalloc::Heap::environment):
976
977         * bmalloc/Mutex.h:
978         (bmalloc::Mutex::Mutex):
979         * bmalloc/StaticMutex.h: A little refactoring to clarify these comments,
980         since I got super confused about them while writing this patch.
981
982         * bmalloc/VMHeap.cpp: Fixed an #include.
983
984 2014-12-09  David Kilzer  <ddkilzer@apple.com>
985
986         Switch from using PLATFORM_NAME to SDK selectors in ANGLE, bmalloc, gtest, JavaScriptCore, WTF
987         <http://webkit.org/b/139212>
988
989         Reviewed by Joseph Pecoraro.
990
991         * Configurations/Base.xcconfig:
992         - Only set GCC_ENABLE_OBJC_GC, GCC_MODEL_TUNING and TOOLCHAINS
993           on OS X.
994         * Configurations/DebugRelease.xcconfig:
995         - Only set MACOSX_DEPLOYMENT_TARGET and SDKROOT on OS X.
996
997 2014-11-07  Geoffrey Garen  <ggaren@apple.com>
998
999         bmalloc uses 8X more virtual memory than necessary
1000         https://bugs.webkit.org/show_bug.cgi?id=138495
1001
1002         Reviewed by Mark Lam.
1003
1004         iOS has a per-process virtual memory cap around 1GB, so there's some
1005         value to not going totally ham with virtual memory.
1006
1007         We currently use about 8X the necessary amount:
1008             - 2X to align our VM allocation
1009             - 4X to reserve small / medium / (2) large chunk VM ranges per superchunk
1010
1011         We can cut that down:
1012             - Return the unaligned portion of our VM allocation (-2X)
1013             - Use all the chunks in a superchunk, instead of allocating one
1014               chunk per superchunk (-4X)
1015
1016         * bmalloc/Algorithm.h:
1017         (bmalloc::roundUpToMultipleOf): Added a non-constant version of this
1018         function so we can call it with getpagesize() at runtime.
1019
1020         * bmalloc/Chunk.h:
1021         * bmalloc/LargeChunk.h:
1022         (bmalloc::LargeChunk::create): Deleted. Instead of each chunk allocating
1023         its own VM, VMHeap allocates the superchunk and all the chunks in it at a time.
1024
1025         * bmalloc/VMAllocate.h:
1026         (bmalloc::vmValidate):
1027         (bmalloc::vmAllocate): ASSERT that mmap succeeds to make crashes clearer
1028         if it does not succeed. Allocate precisely, and give back the extra.
1029
1030         * bmalloc/VMHeap.cpp:
1031         (bmalloc::VMHeap::allocateSuperChunk):
1032         (bmalloc::VMHeap::allocateSmallChunk): Deleted.
1033         (bmalloc::VMHeap::allocateMediumChunk): Deleted.
1034         (bmalloc::VMHeap::allocateLargeChunk): Deleted. Use all the chunks
1035         in a superchunk, instead of just one.
1036
1037         * bmalloc/VMHeap.h:
1038         (bmalloc::VMHeap::allocateSmallPage):
1039         (bmalloc::VMHeap::allocateMediumPage):
1040         (bmalloc::VMHeap::allocateLargeRange):
1041         * bmalloc/XLargeChunk.h:
1042         (bmalloc::XLargeChunk::create): Updated to match changes above.
1043
1044 2014-11-01  David Kilzer  <ddkilzer@apple.com>
1045
1046         JavaScriptCore is missing debug info for bmalloc because libbmalloc.a is stripped
1047         <https://webkit.org/b/138286>
1048         <rdar://problem/18847087>
1049
1050         Reviewed by Dan Bernstein.
1051
1052         * Configurations/bmalloc.xcconfig: Set STRIP_INSTALLED_PRODUCT
1053         to NO for the target that produces libbmalloc.a so that the
1054         debug symbols will be linked into JavaScriptCore and end up in
1055         its dSYM file.
1056
1057 2014-10-30  Dana Burkart  <dburkart@apple.com>
1058
1059         <rdar://problem/18821260> Prepare for the mysterious future
1060
1061         Reviewed by Lucas Forschler.
1062
1063         * Configurations/Base.xcconfig:
1064         * Configurations/DebugRelease.xcconfig:
1065
1066 2014-09-24  Geoffrey Garen  <ggaren@apple.com>
1067
1068         bmalloc: cleaned up fast path vs slow path
1069         https://bugs.webkit.org/show_bug.cgi?id=137081
1070
1071         Reviewed by Sam Weinig.
1072
1073         Might be a 1% speedup on MallocBench. Also cleans up the code a bit.
1074
1075         * bmalloc/Allocator.cpp:
1076         (bmalloc::Allocator::Allocator): Merged the small and medium range
1077         caches, just like the small and medium allocators. Ranges are abstract
1078         objects that don't really care whether they hold small or medium objects,
1079         so they don't need to be segregated.
1080
1081         (bmalloc::Allocator::scavenge): Ditto.
1082
1083         (bmalloc::Allocator::allocateBumpRangeSlowCase):
1084         (bmalloc::Allocator::allocateBumpRange): Same thing here, except that
1085         we do care a tiny bit, because we need to specify small vs medium when
1086         allocating new ranges from the heap, to ensure that the heap allocates
1087         from the right segment of VM.
1088
1089         (bmalloc::Allocator::allocateLarge):
1090         (bmalloc::Allocator::allocateXLarge): NO_INLINE because this was clouding
1091         up the fast path. Large allocation performance is dominated by allocation
1092         logic and initialization, so inlining it doesn't help.
1093
1094         (bmalloc::Allocator::allocateSlowCase): Slow path got a bit cleaner since
1095         it doesn't need to distinguish small vs medium objects.
1096
1097         (bmalloc::Allocator::allocateSmallBumpRange): Deleted.
1098         (bmalloc::Allocator::allocateMediumBumpRange): Deleted.
1099
1100         * bmalloc/Allocator.h:
1101         * bmalloc/BumpRange.h:
1102
1103         * bmalloc/Cache.cpp:
1104         (bmalloc::Cache::allocateSlowCase): Deleted.
1105         (bmalloc::Cache::deallocateSlowCase): Deleted.
1106         * bmalloc/Cache.h:
1107         (bmalloc::Cache::allocate):
1108         (bmalloc::Cache::deallocate):
1109         (bmalloc::Cache::allocateFastCase): Deleted.
1110         (bmalloc::Cache::deallocateFastCase): Deleted. Removed the Cache slow
1111         paths. The downside to this change is that the fast path branches to two
1112         distinct failure cases instead of one. The upside is that the slow path
1113         doesn't need to re-read the segment register, which is not as cheap as a
1114         normal register, and it doesn't need to do an extra level of function 
1115         call. Seems to be worth it.
1116
1117         * bmalloc/Deallocator.h:
1118         * bmalloc/Heap.cpp:
1119         (bmalloc::Heap::refillSmallBumpRangeCache):
1120         (bmalloc::Heap::refillMediumBumpRangeCache):
1121         * bmalloc/Heap.h: Updated for interface changes.
1122
1123         * bmalloc/Sizes.h: The most ranges a cache will hold is the number of
1124         small lines in a page / 2, since any other free lines will coalesce
1125         with their neighbors.
1126
1127 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
1128
1129         Rolled out r173346.
1130
1131             bmalloc should honor the FastMalloc statistics API
1132             https://bugs.webkit.org/show_bug.cgi?id=136592
1133
1134         This didn't really work. Because we allow ranges with and without
1135         physical pages to merge, and we allow double-committing and
1136         double-decommitting, we can't rely on commit actions to track memory
1137         footprint.
1138
1139         * bmalloc/Heap.cpp:
1140         (bmalloc::Heap::size): Deleted.
1141         (bmalloc::Heap::capacity): Deleted.
1142         * bmalloc/Heap.h:
1143         * bmalloc/VMHeap.cpp:
1144         (bmalloc::VMHeap::VMHeap):
1145         (bmalloc::VMHeap::allocateSmallChunk):
1146         (bmalloc::VMHeap::allocateMediumChunk):
1147         (bmalloc::VMHeap::allocateLargeChunk):
1148         * bmalloc/VMHeap.h:
1149         (bmalloc::VMHeap::allocateSmallPage):
1150         (bmalloc::VMHeap::allocateMediumPage):
1151         (bmalloc::VMHeap::allocateLargeRange):
1152         (bmalloc::VMHeap::deallocateSmallPage):
1153         (bmalloc::VMHeap::deallocateMediumPage):
1154         (bmalloc::VMHeap::deallocateLargeRange):
1155         (bmalloc::VMHeap::size): Deleted.
1156         (bmalloc::VMHeap::capacity): Deleted.
1157         * bmalloc/bmalloc.h:
1158         (bmalloc::api::heapSize): Deleted.
1159         (bmalloc::api::heapCapacity): Deleted.
1160
1161 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
1162
1163         bmalloc: Allocation should be more precise
1164         https://bugs.webkit.org/show_bug.cgi?id=136993
1165
1166         Reviewed by Gavin Barraclough.
1167
1168         13% reduction in heap size on the MallocBench *_memory_warning benchmarks.
1169
1170         This patch teaches the allocator to merge adjacent free lines into a
1171         single allocatable range. This allows us to shrink the size of an
1172         individual line without increasing fragmentation or the rate of allocator
1173         slow paths.
1174
1175         We'll only take more slow paths when available memory is sparse, which
1176         is exactly when it's worth it. When available memory is dense, we'll
1177         take fewer slow paths.
1178
1179         * bmalloc.xcodeproj/project.pbxproj:
1180         * bmalloc/Algorithm.h:
1181         (bmalloc::divideRoundingUp):
1182
1183         * bmalloc/Allocator.cpp:
1184         (bmalloc::Allocator::Allocator): Updated for interface changes.
1185
1186         (bmalloc::Allocator::scavenge): Scavenge by object instead of by line.
1187         Now that we merge lines, it's not convenient to scavenge by line.
1188
1189         (bmalloc::Allocator::allocateSmallBumpRange):
1190         (bmalloc::Allocator::allocateMediumBumpRange): Allocate whole ranges
1191         instead of individual lines.
1192
1193         (bmalloc::Allocator::allocateSlowCase):
1194         (bmalloc::Allocator::allocateSmallLine): Deleted.
1195         (bmalloc::Allocator::allocateMediumLine): Deleted.
1196         (bmalloc::Allocator::allocateMedium): Deleted.
1197         * bmalloc/Allocator.h:
1198         (bmalloc::Allocator::allocateFastCase): Folded medium allocations
1199         into the standard fast path with small allocations. Since a BumpAllocator
1200         just allocates out of an arbitrary range, it doesn't need to distinguish
1201         between small and medium lines.
1202
1203         * bmalloc/BumpAllocator.h:
1204         (bmalloc::BumpAllocator::size):
1205         (bmalloc::BumpAllocator::BumpAllocator):
1206         (bmalloc::BumpAllocator::init):
1207         (bmalloc::BumpAllocator::refill):
1208         (bmalloc::BumpAllocator::line): Deleted. No need to track line information
1209         anymore: the heap just gives us a pointer and a pre-computed number of
1210         objects, and we allocate them.
1211
1212         * bmalloc/Deallocator.cpp:
1213         (bmalloc::Deallocator::processObjectLog): Updated for interface changes.
1214
1215         * bmalloc/Heap.cpp:
1216         (bmalloc::Heap::Heap):
1217         (bmalloc::Heap::initializeLineMetadata): Pre-compute precise metadata
1218         detailing where all objects will lie in memory. After we merge two lines,
1219         we might allocate an object that spans from one line to the next. This
1220         metadata details which bits of memory overlap in that way, and how they
1221         overlap.
1222
1223         (bmalloc::Heap::refillSmallBumpRangeCache):
1224         (bmalloc::Heap::refillMediumBumpRangeCache): Scan a whole page at a time,
1225         and merge adjacent free lines into BumpRanges.
1226
1227         (bmalloc::Heap::allocateSmallPage):
1228         (bmalloc::Heap::allocateMediumPage):
1229         (bmalloc::Heap::deallocateSmallLine):
1230         (bmalloc::Heap::deallocateMediumLine): Track pages rather than lines,
1231         since we scan for free memory a page at a time.
1232
1233         (bmalloc::Heap::allocateSmallLineSlowCase): Deleted.
1234         (bmalloc::Heap::allocateMediumLineSlowCase): Deleted. Folded into the
1235         fast path.
1236
1237         * bmalloc/Heap.h:
1238         (bmalloc::Heap::derefSmallLine):
1239         (bmalloc::Heap::derefMediumLine):
1240         (bmalloc::Heap::deallocateSmallLine): Deleted.
1241         (bmalloc::Heap::allocateSmallLine): Deleted.
1242         (bmalloc::Heap::deallocateMediumLine): Deleted.
1243         (bmalloc::Heap::allocateMediumLine): Deleted. Updated for interface changes.
1244
1245         * bmalloc/Line.h:
1246         (bmalloc::Line<Traits>::ref):
1247         (bmalloc::Line<Traits>::deref):
1248         (bmalloc::Line<Traits>::concurrentRef): Deleted. We don't pass a derefCount
1249         anymore, since we only ever deref by 1 now.
1250
1251         * bmalloc/MediumAllocator.h:
1252         (bmalloc::MediumAllocator::isNull): Deleted.
1253         (bmalloc::MediumAllocator::MediumAllocator): Deleted.
1254         (bmalloc::MediumAllocator::line): Deleted.
1255         (bmalloc::MediumAllocator::allocate): Deleted.
1256         (bmalloc::MediumAllocator::derefCount): Deleted.
1257         (bmalloc::MediumAllocator::refill): Deleted.
1258         (bmalloc::MediumAllocator::clear): Deleted. Deleted some code that's
1259         been dead for a while, since it doesn't build anymore with this patch.
1260
1261         * bmalloc/Page.h:
1262         (bmalloc::Page::sizeClass):
1263         (bmalloc::Page::setSizeClass):
1264         (bmalloc::Page::smallSizeClass): Deleted.
1265         (bmalloc::Page::setSmallSizeClass): Deleted. Renamed setSmallSizeClass
1266         to sizeClass, since we use it for medium sizes too.
1267
1268         * bmalloc/Sizes.h:
1269         (bmalloc::Sizes::sizeClass):
1270         (bmalloc::Sizes::objectSize): Shrank line sizes to save memory.
1271
1272         (bmalloc::Sizes::smallSizeClassFor): Deleted.
1273         (bmalloc::Sizes::mediumSizeClassFor): Deleted.
1274
1275         * bmalloc/bmalloc.h:
1276         (bmalloc::api::realloc): Now that we have precise objects sizes, realloc
1277         can be a bit more precise. It also has to be, since we can't guarantee
1278         that an object ends at the end of a line anymore.
1279
1280 2014-09-19  Daniel Bates  <dabates@apple.com>
1281
1282         Always assume internal SDK when building configuration Production
1283         https://bugs.webkit.org/show_bug.cgi?id=136925
1284         <rdar://problem/18362399>
1285
1286         Reviewed by Dan Bernstein.
1287
1288         * Configurations/Base.xcconfig:
1289
1290 2014-09-16  Geoffrey Garen  <ggaren@apple.com>
1291
1292         bmalloc: moved line caches from the deallocator to the allocator
1293         https://bugs.webkit.org/show_bug.cgi?id=136868
1294
1295         Reviewed by Gavin Barraclough.
1296
1297         I did this mostly as a simplification, to make it easier to change the
1298         allocation strategy.
1299
1300         No throughput change on MallocBench. Saves about 50kB.
1301
1302         Since the deallocator needs to lock the heap when freeing lines anyway,
1303         there isn't much benefit to giving the deallocator a local cache of
1304         deallocated lines.
1305
1306         We still give the allocator a local cache of lines because that does
1307         reduce the frequency at which it needs to lock the heap in order to
1308         acquire more lines.
1309
1310         * bmalloc/Allocator.cpp:
1311         (bmalloc::Allocator::scavenge):
1312         (bmalloc::Allocator::allocateSmallLine):
1313         (bmalloc::Allocator::allocateMediumLine):
1314         (bmalloc::Allocator::allocateMedium):
1315         (bmalloc::Allocator::allocateSlowCase):
1316         * bmalloc/Allocator.h:
1317         * bmalloc/Deallocator.cpp:
1318         (bmalloc::Deallocator::Deallocator):
1319         (bmalloc::Deallocator::scavenge):
1320         (bmalloc::Deallocator::processObjectLog):
1321         (bmalloc::Deallocator::deallocateSmallLine): Deleted.
1322         (bmalloc::Deallocator::allocateSmallLine): Deleted.
1323         (bmalloc::Deallocator::deallocateMediumLine): Deleted.
1324         (bmalloc::Deallocator::allocateMediumLine): Deleted.
1325         * bmalloc/Deallocator.h:
1326
1327         * bmalloc/Sizes.h:
1328         * bmalloc/VMAllocate.h: Took the opportunity to make the line cache size
1329         exactly one page in size. That's about what we were shooting for anyway,
1330         and it may make it easier to switch to per-page allocation in future.
1331
1332 2014-09-15  Geoffrey Garen  <ggaren@apple.com>
1333
1334         bmalloc: allocate small and medium objects using the same bump pointer class
1335         https://bugs.webkit.org/show_bug.cgi?id=136843
1336
1337         Reviewed by Gavin Barraclough.
1338
1339         4% speedup on MallocBench.
1340
1341         Now that medium-sized objects have dedicated per-size allocators, they
1342         don't need to use an arbitrary bump pointer allocator. This means that
1343         every allocator knows how many objects it will allocate from the start,
1344         and we don't need a post-processing step to adjust refcounts based on
1345         real allocation count.
1346
1347         * bmalloc.xcodeproj/project.pbxproj: Renamed SmallAllocator to BumpAllocator
1348         since it's used for small and medium objects now.
1349
1350         * bmalloc/Allocator.cpp:
1351         (bmalloc::Allocator::Allocator): Updated to use new interface.
1352         (bmalloc::Allocator::scavenge): To "retire" an allocator, we just need
1353         to make sure that we finish allocating all the objects in it.
1354
1355         (bmalloc::Allocator::allocateMedium):
1356         (bmalloc::Allocator::allocateSlowCase):
1357         (bmalloc::Allocator::retire): Deleted.
1358         (bmalloc::Allocator::processSmallAllocatorLog): Deleted.
1359         (bmalloc::Allocator::processMediumAllocatorLog): Deleted.
1360         * bmalloc/Allocator.h:
1361         (bmalloc::Allocator::allocateFastCase): Removed abstractions and data
1362         used to post-process an allocator based on how many objects it allocated.
1363
1364         * bmalloc/BumpAllocator.h: Copied from Source/bmalloc/bmalloc/SmallAllocator.h.
1365         (bmalloc::BumpAllocator::BumpAllocator):
1366         (bmalloc::BumpAllocator::init):
1367         (bmalloc::BumpAllocator::line):
1368         (bmalloc::BumpAllocator::validate):
1369         (bmalloc::BumpAllocator::allocate):
1370         (bmalloc::BumpAllocator::refill):
1371         (bmalloc::BumpAllocator::clear): Updated these functions to be agnostic
1372         about the kinds of lines they allocate into. In some cases, the line
1373         type must be provided as a template parameter by the caller.
1374
1375         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
1376         (bmalloc::SmallAllocator::line): Deleted.
1377         (bmalloc::SmallAllocator::allocate): Deleted.
1378         (bmalloc::SmallAllocator::objectCount): Deleted.
1379         (bmalloc::SmallAllocator::derefCount): Deleted.
1380         (bmalloc::SmallAllocator::refill): Deleted.
1381         (bmalloc::SmallAllocator::clear): Deleted.
1382
1383         * bmalloc/ObjectType.h:
1384         (bmalloc::isMedium):
1385
1386         * bmalloc/SmallAllocator.h:
1387         (bmalloc::SmallAllocator::isNull): Deleted.
1388         (bmalloc::SmallAllocator::canAllocate): Deleted.
1389         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
1390         (bmalloc::SmallAllocator::line): Deleted.
1391         (bmalloc::SmallAllocator::allocate): Deleted.
1392         (bmalloc::SmallAllocator::objectCount): Deleted.
1393         (bmalloc::SmallAllocator::derefCount): Deleted.
1394         (bmalloc::SmallAllocator::refill): Deleted.
1395         (bmalloc::SmallAllocator::clear): Deleted.
1396
1397 2014-09-12  Geoffrey Garen  <ggaren@apple.com>
1398
1399         Fixed a goof in bmalloc Vector sizing
1400         https://bugs.webkit.org/show_bug.cgi?id=136795
1401
1402         Reviewed by Gavin Barraclough and Sam Weinig.
1403
1404         We want our minimum vector to be page-sized since the OS will give us
1405         a page no matter what -- but we want that many bytes, and not enough
1406         bytes to store that many elements.
1407
1408         * bmalloc/Vector.h: Math is hard.
1409
1410 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
1411
1412         bmalloc should segregate medium-sized objects by line like it does for small-sized objects
1413         https://bugs.webkit.org/show_bug.cgi?id=136693
1414
1415         Reviewed by Gavin Barraclough.
1416
1417         4% reduction in heap size on the MallocBench *_memory_warning benchmarks.
1418
1419         No throughput change.
1420
1421         We keep an array of medium allocators, just like our array of small
1422         allocators.
1423
1424         In future, we can simplify the allocation fast path by merging the small
1425         and medium allocator arrays. For now, this is the simplest change that
1426         gets the win.
1427
1428         * bmalloc/Allocator.cpp:
1429         (bmalloc::Allocator::Allocator):
1430         (bmalloc::Allocator::scavenge):
1431         (bmalloc::Allocator::allocateMedium):
1432         * bmalloc/Allocator.h:
1433         * bmalloc/Sizes.h:
1434         (bmalloc::Sizes::mediumSizeClassFor):
1435
1436 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
1437
1438         Reviewed by Sam Weinig.
1439
1440         Renamed log => retire for clarity.
1441
1442         * bmalloc/Allocator.cpp:
1443         (bmalloc::Allocator::scavenge):
1444         (bmalloc::Allocator::retire):
1445         (bmalloc::Allocator::allocateMedium):
1446         (bmalloc::Allocator::allocateSlowCase):
1447         (bmalloc::Allocator::log): Deleted.
1448         * bmalloc/Allocator.h:
1449
1450 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
1451
1452         bmalloc: eager scavenge leaves behind a bogus allocator
1453         https://bugs.webkit.org/show_bug.cgi?id=136743
1454
1455         Reviewed by Sam Weinig.
1456
1457         Be sure to clear the allocator after logging it in the eager scavenge
1458         case, so that we don't later try to allocate out of the lines that we
1459         have thrown away.
1460
1461         We didn't need to do this previously because scavenge would only happen
1462         at thread exit time, after which no further allocation from the per-thread
1463         cache would take place.
1464
1465         * bmalloc/Allocator.cpp:
1466         (bmalloc::Allocator::scavenge):
1467         * bmalloc/MediumAllocator.h:
1468         (bmalloc::MediumAllocator::clear):
1469         * bmalloc/SmallAllocator.h:
1470         (bmalloc::SmallAllocator::clear):
1471
1472 2014-09-05  Geoffrey Garen  <ggaren@apple.com>
1473
1474         bmalloc should honor the FastMalloc statistics API
1475         https://bugs.webkit.org/show_bug.cgi?id=136592
1476
1477         Reviewed by Gavin Barraclough.
1478
1479         We do this by tracking "size" and "capacity" in the VM heap.
1480
1481         The VM heap's "capacity" is all the VM we ever allocated.
1482
1483         The VM heap's "size" the subset of VM currently held onto by the
1484         VM heap (and therefore not in use by the regular heap).
1485
1486         Somewhat ironically, reducing the process's memory footprint, increases
1487         the size of the VM heap, since the VM heap holds the pages that are
1488         purely virtual and not physical.
1489
1490         * bmalloc/Heap.cpp:
1491         (bmalloc::Heap::size):
1492         (bmalloc::Heap::capacity):
1493         * bmalloc/Heap.h:
1494         * bmalloc/VMHeap.cpp:
1495         (bmalloc::VMHeap::VMHeap):
1496         (bmalloc::VMHeap::allocateSmallChunk):
1497         (bmalloc::VMHeap::allocateMediumChunk):
1498         (bmalloc::VMHeap::allocateLargeChunk):
1499         * bmalloc/VMHeap.h:
1500         (bmalloc::VMHeap::size):
1501         (bmalloc::VMHeap::capacity):
1502         (bmalloc::VMHeap::allocateSmallPage):
1503         (bmalloc::VMHeap::allocateMediumPage):
1504         (bmalloc::VMHeap::allocateLargeRange):
1505         (bmalloc::VMHeap::deallocateSmallPage):
1506         (bmalloc::VMHeap::deallocateMediumPage):
1507         (bmalloc::VMHeap::deallocateLargeRange):
1508         * bmalloc/bmalloc.h:
1509         (bmalloc::api::heapSize):
1510         (bmalloc::api::heapCapacity):
1511
1512 2014-09-02  Geoffrey Garen  <ggaren@apple.com>
1513
1514         bmalloc crashes on the EWS bots (due to bad large object allocation)
1515         https://bugs.webkit.org/show_bug.cgi?id=136469
1516
1517         Reviewed by Andreas Kling.
1518
1519         It's possible to convince bmalloc to perform a bad large object allocation,
1520         through these steps:
1521
1522         (1) Insert object A into freelist F0.
1523
1524         (2) Split, merge and split again A's neighbors such that object B is
1525         inserted into freelist F0, with boundary tag and size equal to object A,
1526         but pointer not completely equal to object A. Put object B at the head of F0.
1527
1528         (3) Allocate some other object from F0, swapping its position in the
1529         freelist with object B, such that object A is now ahead of object B.
1530
1531         --> Now, the next allocation for size A/B will allocate object A, which
1532         has a slightly wrong idea about where the object actually begins.
1533         Immediately, you'll corrupt a little memory, and over time, you'll also
1534         corrupt boundary tag metadata.
1535
1536         The solution is to store the begin pointer in the boundary tag. Luckily,
1537         this doesn't make the tag any bigger, and it's not a noticeable slowdown
1538         on MallocBench.
1539
1540         * bmalloc/Algorithm.h:
1541         (bmalloc::rightShift):
1542         * bmalloc/BeginTag.h:
1543         (bmalloc::BeginTag::isInFreeList): This is the bug fix. Make sure to
1544         validate the start pointer when popping off the free list. Through a
1545         very uncommon set of steps, it is possible to have an item in the free
1546         list that is valid by all accounts except for its start pointer.
1547
1548         * bmalloc/BoundaryTag.h:
1549         (bmalloc::BoundaryTag::compactBegin):
1550         (bmalloc::BoundaryTag::setRange):
1551         (bmalloc::BoundaryTag::setSize): Deleted. Record a compact version of the
1552         start pointer. We don't need the whole pointer -- just the offset, in
1553         largeAlignment increments, into the relevant boundary tag bucket.
1554
1555         * bmalloc/BoundaryTagInlines.h:
1556         (bmalloc::validateNext):
1557         (bmalloc::BoundaryTag::init):
1558         (bmalloc::BoundaryTag::mergeLarge):
1559         (bmalloc::BoundaryTag::splitLarge):
1560         * bmalloc/SegregatedFreeList.cpp:
1561         (bmalloc::SegregatedFreeList::insert):
1562         (bmalloc::SegregatedFreeList::takeGreedy):
1563         (bmalloc::SegregatedFreeList::take): Provide the whole range instead of
1564         the size when establishing a boundary tag, as required by the new
1565         interface.
1566
1567         * bmalloc/Sizes.h:
1568
1569 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
1570
1571         Fixed a bmalloc crash seen on the EWS bot
1572         https://bugs.webkit.org/show_bug.cgi?id=135955
1573
1574         Reviewed by Andreas Kling.
1575
1576         * bmalloc/Syscall.h: Some CG APIs vm_copy their input buffers. If the
1577         input buffer is a malloc region, that region will get marked Copy-On-Write
1578         by the kernel. Calls to madvise() for COW regions fail and return EINVAL
1579         on older OS X's. In 10.10, they still fail, but they do not return
1580         EINVAL.
1581
1582         So, we can only ASSERT that our syscalls succeed starting with 10.10.
1583
1584 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
1585
1586         Fixed the bmalloc build
1587         https://bugs.webkit.org/show_bug.cgi?id=135953
1588
1589         Reviewed by Andreas Kling.
1590
1591         * bmalloc.xcodeproj/project.pbxproj: Marked a few headers as private.
1592         These headers are used, so they must be available outside the project.
1593
1594 2014-08-13  Daniel Bates  <dabates@apple.com>
1595
1596         Attempt to fix the build following <http://trac.webkit.org/changeset/172576>
1597         (https://bugs.webkit.org/show_bug.cgi?id=135895)
1598
1599         Substitute PerThreadStorage<T>::initSharedKeyIfNeeded() for initSharedKeyIfNeeded() in
1600         implementation of PerThread<T>::getFastCase().
1601
1602         * bmalloc/PerThread.h:
1603         (bmalloc::PerThread<T>::getFastCase):
1604
1605 2014-08-13  Daniel Bates  <dabates@apple.com>
1606
1607         Make bmalloc::PerThread work without C++ thread local storage
1608         https://bugs.webkit.org/show_bug.cgi?id=135895
1609
1610         Reviewed by Geoffrey Garen.
1611
1612         Implement support for building bmalloc without C++ thread local storage.
1613
1614         * bmalloc/BPlatform.h: Remove macro define BPLATFORM_IOS_SIMULATOR. Added macro function
1615         BCOMPILER_SUPPORTS() and macro define BCOMPILER_SUPPORTS_CXX_THREAD_LOCAL that can be used
1616         to determine whether the compiler supports C++ thread local storage.
1617         * bmalloc/PerThread.h:
1618         (bmalloc::PerThreadStorage::get): Modified to call pthread_getspecific() when building
1619         without C++ thread local storage.
1620         (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Added.
1621         (bmalloc::PerThreadStorage::init): Moved logic to initialize shared Pthread key from here to
1622         PerThreadStorage::initSharedKeyIfNeeded().
1623         (bmalloc::PerThread<T>::getFastCase): Modified to call PerThreadStorage::initSharedKeyIfNeeded()
1624         before querying PerThreadStorage::get() when building without C++ thread local storage so as to
1625         ensure that the shared key has been initialized.
1626         (_pthread_setspecific_direct): Deleted.
1627         (_pthread_getspecific_direct): Deleted.
1628
1629 2014-08-13  Daniel Bates  <dabates@apple.com>
1630
1631         [iOS] Make JavaScriptCore and bmalloc build with the public SDK
1632         https://bugs.webkit.org/show_bug.cgi?id=135848
1633
1634         Reviewed by Geoffrey Garen.
1635
1636         * bmalloc/BPlatform.h: Added macro BPLATFORM_IOS_SIMULATOR, which evaluates to true
1637         when building for the iOS Simulator.
1638         * bmalloc/PerThread.h: Use pthread_machdep.h code path when building for iOS Simulator
1639         using the public SDK.
1640         (_pthread_setspecific_direct): Added; only defined when building for the iOS Simulator
1641         using the public SDK.
1642         (_pthread_getspecific_direct): Added; only defined when building for the iOS Simulator
1643         using the public SDK.
1644
1645 2014-08-12  Daniel Bates  <dabates@apple.com>
1646
1647         BPLATFORM(IOS) always evaluates to false
1648         https://bugs.webkit.org/show_bug.cgi?id=135843
1649
1650         Reviewed by Geoffrey Garen.
1651
1652         Fix typo in definition of BPLATFORM() and include system header TargetConditionals.h
1653         (when building on an Apple platform) so that BPLATFORM(X) evaluates to true when
1654         building for platform X. In particular, so that BPLATFORM(IOS) evaluates to true when
1655         building for iOS.
1656
1657         As a side effect of this change, the change made in <http://trac.webkit.org/changeset/167289>
1658         will be honored and iOS will assume a VM page size of 16kB (again) instead of 4kB.
1659
1660         * bmalloc/BPlatform.h:
1661
1662 2014-08-11  Andy Estes  <aestes@apple.com>
1663
1664         [iOS] Get rid of iOS.xcconfig
1665         https://bugs.webkit.org/show_bug.cgi?id=135809
1666
1667         Reviewed by Joseph Pecoraro.
1668
1669         All iOS.xcconfig did was include AspenFamily.xcconfig, so there's no need for the indirection.
1670
1671         * Configurations/Base.xcconfig:
1672         * Configurations/iOS.xcconfig: Removed.
1673         * bmalloc.xcodeproj/project.pbxproj:
1674
1675 2014-05-01  Dan Bernstein  <mitz@apple.com>
1676
1677         Fixed production builds for the iOS Simulator.
1678         <rdar://problem/16792221>
1679
1680         * Configurations/bmalloc.xcconfig: Include INSTALL_PATH_PREFIX in
1681         PRIVATE_HEADERS_FOLDER_PATH when installing.
1682
1683 2014-04-20  Geoffrey Garen  <ggaren@apple.com>
1684
1685         bmalloc: Segregate pages by objects size
1686         https://bugs.webkit.org/show_bug.cgi?id=131909
1687
1688         Reviewed by Andreas Kling.
1689
1690         2% reduction in memory-at-end on the Membuster memory_warning benchmarks.
1691
1692         * bmalloc/Allocator.cpp:
1693         (bmalloc::Allocator::allocateSlowCase):
1694         * bmalloc/Allocator.h:
1695         (bmalloc::Allocator::allocateFastCase):
1696         (bmalloc::Allocator::smallAllocatorFor): Use the new shared helper
1697         function for size class calculation.
1698
1699         * bmalloc/Deallocator.cpp:
1700         (bmalloc::Deallocator::Deallocator):
1701         (bmalloc::Deallocator::scavenge):
1702         (bmalloc::Deallocator::deallocateSmallLine):
1703         (bmalloc::Deallocator::allocateSmallLine):
1704         * bmalloc/Deallocator.h: Keep a cache for every size class, since the
1705         cache can't be shared anymore.
1706
1707         * bmalloc/Heap.cpp:
1708         (bmalloc::Heap::allocateSmallLineSlowCase):
1709         * bmalloc/Heap.h:
1710         (bmalloc::Heap::deallocateSmallLine): Ditto.
1711
1712         (bmalloc::Heap::allocateSmallLine): Check size class in addition to
1713         page refcount when allocating a line because we might have deallocated
1714         the page and the recycled it for another size class.
1715
1716         (bmalloc::Heap::deallocateMediumLine):
1717         (bmalloc::Heap::allocateMediumLine):
1718         * bmalloc/Line.h:
1719         (bmalloc::Line::refCount):
1720         * bmalloc/Page.h:
1721         (bmalloc::Page::refCount):
1722         (bmalloc::Page::smallSizeClass):
1723         (bmalloc::Page::setSmallSizeClass):
1724         (bmalloc::Page<Traits>::refCount): Deleted.
1725         * bmalloc/Sizes.h:
1726         (bmalloc::Sizes::smallSizeClassFor): New shared API for computing
1727         an index into an array from a size.
1728
1729 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1730
1731         bmalloc: Improved alignment in LargeChunk
1732         https://bugs.webkit.org/show_bug.cgi?id=131895
1733
1734         Reviewed by Andreas Kling.
1735
1736         * bmalloc/Chunk.h:
1737         * bmalloc/LargeChunk.h: Align to vmPageSize just like Chunk does.
1738         Technically, the previous alignment was harmless, but I would prefer,
1739         dear reader, not to have to explain the interlocking set of
1740         circumstances that made it so.
1741
1742 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1743
1744         Rolled out r167502 because it caused a crash on the facebook benchmark.
1745
1746         Unreviewed.
1747
1748             bmalloc: Added an XSmall line size
1749             https://bugs.webkit.org/show_bug.cgi?id=131851
1750
1751             Reviewed by Sam Weinig.
1752
1753 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1754
1755         bmalloc: Mutex should be harder to use wrong
1756         https://bugs.webkit.org/show_bug.cgi?id=131879
1757
1758         Reviewed by Andreas Kling.
1759
1760         Mutex now has a proper constructor, so you can't deadlock by forgetting
1761         to initialize it.
1762
1763         * bmalloc.xcodeproj/project.pbxproj:
1764         * bmalloc/Allocator.cpp:
1765         (bmalloc::Allocator::processXSmallAllocatorLog):
1766         (bmalloc::Allocator::processSmallAllocatorLog):
1767         (bmalloc::Allocator::processMediumAllocatorLog):
1768         (bmalloc::Allocator::allocateLarge):
1769         (bmalloc::Allocator::allocateXLarge): Global replace Mutex => StaticMutex,
1770         since the Heap mutex is a static.
1771
1772         * bmalloc/AsyncTask.h:
1773         (bmalloc::Function>::AsyncTask): Use Mutex, since we're not static. No
1774         need for explicit initialization anymore.
1775
1776         * bmalloc/Deallocator.cpp:
1777         (bmalloc::Deallocator::scavenge):
1778         (bmalloc::Deallocator::deallocateLarge):
1779         (bmalloc::Deallocator::deallocateXLarge):
1780         (bmalloc::Deallocator::processObjectLog):
1781         (bmalloc::Deallocator::deallocateSmallLine):
1782         (bmalloc::Deallocator::deallocateXSmallLine):
1783         (bmalloc::Deallocator::allocateSmallLine):
1784         (bmalloc::Deallocator::allocateXSmallLine):
1785         (bmalloc::Deallocator::deallocateMediumLine):
1786         (bmalloc::Deallocator::allocateMediumLine):
1787         * bmalloc/Deallocator.h:
1788         * bmalloc/Heap.cpp:
1789         (bmalloc::sleep):
1790         (bmalloc::Heap::Heap):
1791         (bmalloc::Heap::concurrentScavenge):
1792         (bmalloc::Heap::scavenge):
1793         (bmalloc::Heap::scavengeSmallPages):
1794         (bmalloc::Heap::scavengeXSmallPages):
1795         (bmalloc::Heap::scavengeMediumPages):
1796         (bmalloc::Heap::scavengeLargeRanges):
1797         (bmalloc::Heap::allocateXSmallLineSlowCase):
1798         (bmalloc::Heap::allocateSmallLineSlowCase):
1799         (bmalloc::Heap::allocateMediumLineSlowCase):
1800         (bmalloc::Heap::allocateXLarge):
1801         (bmalloc::Heap::deallocateXLarge):
1802         (bmalloc::Heap::allocateLarge):
1803         (bmalloc::Heap::deallocateLarge):
1804         * bmalloc/Heap.h:
1805         (bmalloc::Heap::deallocateXSmallLine):
1806         (bmalloc::Heap::allocateXSmallLine):
1807         (bmalloc::Heap::deallocateSmallLine):
1808         (bmalloc::Heap::allocateSmallLine):
1809         (bmalloc::Heap::deallocateMediumLine):
1810         (bmalloc::Heap::allocateMediumLine):
1811         * bmalloc/Line.h:
1812         (bmalloc::Line<Traits>::deref):
1813         * bmalloc/Mutex.cpp: Removed.
1814         * bmalloc/Mutex.h:
1815         (bmalloc::Mutex::Mutex):
1816         (bmalloc::Mutex::init): Deleted.
1817         (bmalloc::Mutex::try_lock): Deleted.
1818         (bmalloc::Mutex::lock): Deleted.
1819         (bmalloc::Mutex::unlock): Deleted.
1820         * bmalloc/Page.h:
1821         (bmalloc::Page<Traits>::ref):
1822         (bmalloc::Page<Traits>::deref):
1823         (bmalloc::Page<Traits>::refCount):
1824         * bmalloc/PerProcess.h:
1825         (bmalloc::PerProcess::mutex):
1826         (bmalloc::PerProcess<T>::getSlowCase):
1827         * bmalloc/StaticMutex.cpp: Added.
1828         (bmalloc::StaticMutex::lockSlowCase):
1829         * bmalloc/StaticMutex.h: Added.
1830         (bmalloc::StaticMutex::init):
1831         (bmalloc::StaticMutex::try_lock):
1832         (bmalloc::StaticMutex::lock):
1833         (bmalloc::StaticMutex::unlock):
1834         * bmalloc/VMHeap.h:
1835         (bmalloc::VMHeap::deallocateXSmallPage):
1836         (bmalloc::VMHeap::deallocateSmallPage):
1837         (bmalloc::VMHeap::deallocateMediumPage):
1838         (bmalloc::VMHeap::deallocateLargeRange):
1839         * bmalloc/bmalloc.h:
1840         (bmalloc::api::scavenge): Global replace Mutex => StaticMutex,
1841         since the Heap mutex is a static.
1842
1843 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
1844
1845         bmalloc: AsyncTask should use Mutex instead of std::mutex
1846         https://bugs.webkit.org/show_bug.cgi?id=131865
1847
1848         Reviewed by Gavin Barraclough.
1849
1850         std::mutex is so slow that it makes parallelizing simple tasks through
1851         AsyncTask a net regression. Mutex fixes this.
1852
1853         * bmalloc/AsyncTask.h:
1854         (bmalloc::Function>::AsyncTask):
1855         (bmalloc::Function>::join):
1856         (bmalloc::Function>::runSlowCase):
1857         (bmalloc::Function>::entryPoint):
1858         * bmalloc/Mutex.h:
1859         (bmalloc::Mutex::init):
1860
1861 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
1862
1863         bmalloc: Added an XSmall line size
1864         https://bugs.webkit.org/show_bug.cgi?id=131851
1865
1866         Reviewed by Sam Weinig.
1867
1868         Reduces malloc footprint on Membuster recordings by 10%.
1869
1870         This is a throughput regression, but we're still way ahead of TCMalloc.
1871         I have some ideas for how to recover the regression -- but I wanted to
1872         get this win in first.
1873
1874         Full set of benchmark results:
1875
1876                 bmalloc> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks --measure-heap nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1877                                                                                                 
1878                                                                nopatch                      patch                                Δ
1879                 Peak Memory:
1880                     reddit_memory_warning                      7,896kB                    7,532kB                  ^ 1.05x smaller
1881                     flickr_memory_warning                     12,968kB                   12,324kB                  ^ 1.05x smaller
1882                     theverge_memory_warning                   16,672kB                   15,200kB                   ^ 1.1x smaller
1883
1884                     <geometric mean>                          11,952kB                   11,216kB                  ^ 1.07x smaller
1885                     <arithmetic mean>                         12,512kB                   11,685kB                  ^ 1.07x smaller
1886                     <harmonic mean>                           11,375kB                   10,726kB                  ^ 1.06x smaller
1887
1888                 Memory at End:
1889                     reddit_memory_warning                      7,320kB                    6,856kB                  ^ 1.07x smaller
1890                     flickr_memory_warning                     10,848kB                    9,692kB                  ^ 1.12x smaller
1891                     theverge_memory_warning                   16,380kB                   14,872kB                   ^ 1.1x smaller
1892
1893                     <geometric mean>                          10,916kB                    9,961kB                   ^ 1.1x smaller
1894                     <arithmetic mean>                         11,516kB                   10,473kB                   ^ 1.1x smaller
1895                     <harmonic mean>                           10,350kB                    9,485kB                  ^ 1.09x smaller
1896
1897                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1898                                                                                                 
1899                                                    nopatch                patch                         Δ
1900                 Execution Time:
1901                     churn                            127ms                151ms            ! 1.19x slower
1902                     list_allocate                    130ms                164ms            ! 1.26x slower
1903                     tree_allocate                    109ms                127ms            ! 1.17x slower
1904                     tree_churn                       115ms                120ms            ! 1.04x slower
1905                     facebook                         240ms                259ms            ! 1.08x slower
1906                     fragment                          91ms                131ms            ! 1.44x slower
1907                     fragment_iterate                 105ms                106ms            ! 1.01x slower
1908                     message_one                      260ms                259ms             ^ 1.0x faster
1909                     message_many                     149ms                154ms            ! 1.03x slower
1910                     medium                           194ms                248ms            ! 1.28x slower
1911                     big                              157ms                160ms            ! 1.02x slower
1912
1913                     <geometric mean>                 144ms                163ms            ! 1.13x slower
1914                     <arithmetic mean>                152ms                171ms            ! 1.12x slower
1915                     <harmonic mean>                  137ms                156ms            ! 1.14x slower
1916
1917                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1918                                                                                                 
1919                                                                        nopatch                          patch                                     Δ
1920                 Execution Time:
1921                     churn                                                126ms                          148ms                        ! 1.17x slower
1922                     churn --parallel                                      62ms                           76ms                        ! 1.23x slower
1923                     list_allocate                                        130ms                          164ms                        ! 1.26x slower
1924                     list_allocate --parallel                             120ms                          175ms                        ! 1.46x slower
1925                     tree_allocate                                        111ms                          127ms                        ! 1.14x slower
1926                     tree_allocate --parallel                              95ms                          135ms                        ! 1.42x slower
1927                     tree_churn                                           115ms                          124ms                        ! 1.08x slower
1928                     tree_churn --parallel                                107ms                          126ms                        ! 1.18x slower
1929                     facebook                                             240ms                          276ms                        ! 1.15x slower
1930                     facebook --parallel                                  802ms                        1,088ms                        ! 1.36x slower
1931                     fragment                                              92ms                          130ms                        ! 1.41x slower
1932                     fragment --parallel                                   66ms                          124ms                        ! 1.88x slower
1933                     fragment_iterate                                     109ms                          127ms                        ! 1.17x slower
1934                     fragment_iterate --parallel                           55ms                           64ms                        ! 1.16x slower
1935                     message_one                                          260ms                          260ms                                      
1936                     message_many                                         170ms                          238ms                         ! 1.4x slower
1937                     medium                                               185ms                          250ms                        ! 1.35x slower
1938                     medium --parallel                                    210ms                          334ms                        ! 1.59x slower
1939                     big                                                  150ms                          169ms                        ! 1.13x slower
1940                     big --parallel                                       138ms                          144ms                        ! 1.04x slower
1941
1942                     <geometric mean>                                     135ms                          170ms                        ! 1.26x slower
1943                     <arithmetic mean>                                    167ms                          214ms                        ! 1.28x slower
1944                     <harmonic mean>                                      117ms                          148ms                        ! 1.26x slower
1945
1946                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
1947
1948                                                                     TC                      patch                                Δ
1949                 Peak Memory:
1950                     reddit_memory_warning                     13,836kB                   13,436kB                  ^ 1.03x smaller
1951                     flickr_memory_warning                     24,868kB                   25,188kB                   ! 1.01x bigger
1952                     theverge_memory_warning                   24,504kB                   26,636kB                   ! 1.09x bigger
1953
1954                     <geometric mean>                          20,353kB                   20,812kB                   ! 1.02x bigger
1955                     <arithmetic mean>                         21,069kB                   21,753kB                   ! 1.03x bigger
1956                     <harmonic mean>                           19,570kB                   19,780kB                   ! 1.01x bigger
1957
1958                 Memory at End:
1959                     reddit_memory_warning                      8,656kB                   10,016kB                   ! 1.16x bigger
1960                     flickr_memory_warning                     11,844kB                   13,784kB                   ! 1.16x bigger
1961                     theverge_memory_warning                   18,516kB                   22,748kB                   ! 1.23x bigger
1962
1963                     <geometric mean>                          12,382kB                   14,644kB                   ! 1.18x bigger
1964                     <arithmetic mean>                         13,005kB                   15,516kB                   ! 1.19x bigger
1965                     <harmonic mean>                           11,813kB                   13,867kB                   ! 1.17x bigger
1966
1967                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
1968                                                                                                 
1969                                                         TC                patch                         Δ
1970                 Execution Time:
1971                     churn                            416ms                148ms            ^ 2.81x faster
1972                     list_allocate                    463ms                164ms            ^ 2.82x faster
1973                     tree_allocate                    292ms                127ms             ^ 2.3x faster
1974                     tree_churn                       157ms                120ms            ^ 1.31x faster
1975                     facebook                         327ms                276ms            ^ 1.18x faster
1976                     fragment                         335ms                129ms             ^ 2.6x faster
1977                     fragment_iterate                 344ms                108ms            ^ 3.19x faster
1978                     message_one                      386ms                258ms             ^ 1.5x faster
1979                     message_many                     410ms                154ms            ^ 2.66x faster
1980                     medium                           391ms                245ms             ^ 1.6x faster
1981                     big                              261ms                167ms            ^ 1.56x faster
1982
1983                     <geometric mean>                 332ms                164ms            ^ 2.02x faster
1984                     <arithmetic mean>                344ms                172ms            ^ 1.99x faster
1985                     <harmonic mean>                  317ms                157ms            ^ 2.02x faster
1986
1987         * bmalloc.xcodeproj/project.pbxproj:
1988         * bmalloc/Allocator.cpp:
1989         (bmalloc::Allocator::Allocator): Don't assume that each allocator's
1990         index corresponds with its size. Instead, use the size selection function
1991         explicitly. Now that we have XSmall, some small allocator entries are
1992         unused.
1993
1994         (bmalloc::Allocator::scavenge):
1995         (bmalloc::Allocator::log):
1996         (bmalloc::Allocator::processXSmallAllocatorLog):
1997         (bmalloc::Allocator::allocateSlowCase):
1998         * bmalloc/Allocator.h:
1999         (bmalloc::Allocator::xSmallAllocatorFor):
2000         (bmalloc::Allocator::allocateFastCase):
2001         * bmalloc/Chunk.h:
2002         * bmalloc/Deallocator.cpp:
2003         (bmalloc::Deallocator::scavenge):
2004         (bmalloc::Deallocator::processObjectLog):
2005         (bmalloc::Deallocator::deallocateSlowCase):
2006         (bmalloc::Deallocator::deallocateXSmallLine):
2007         (bmalloc::Deallocator::allocateXSmallLine):
2008         * bmalloc/Deallocator.h:
2009         (bmalloc::Deallocator::deallocateFastCase):
2010         * bmalloc/Heap.cpp:
2011         (bmalloc::Heap::scavenge):
2012         (bmalloc::Heap::scavengeXSmallPages):
2013         (bmalloc::Heap::allocateXSmallLineSlowCase):
2014         * bmalloc/Heap.h:
2015         (bmalloc::Heap::deallocateXSmallLine):
2016         (bmalloc::Heap::allocateXSmallLine):
2017         * bmalloc/LargeChunk.h:
2018         (bmalloc::LargeChunk::get):
2019         (bmalloc::LargeChunk::endTag):
2020         * bmalloc/Line.h:
2021         * bmalloc/MediumAllocator.h:
2022         (bmalloc::MediumAllocator::allocate):
2023         (bmalloc::MediumAllocator::refill):
2024         * bmalloc/ObjectType.cpp:
2025         (bmalloc::objectType):
2026         * bmalloc/ObjectType.h:
2027         (bmalloc::isXSmall):
2028         (bmalloc::isSmall):
2029         (bmalloc::isMedium):
2030         (bmalloc::isLarge):
2031         (bmalloc::isSmallOrMedium): Deleted.
2032         * bmalloc/SegregatedFreeList.h: I boiler-plate copied existing code for
2033         handling small objects. There's probably a reasonable way to share this
2034         code in the future -- I'll look into that once it's stopped changing.
2035
2036         * bmalloc/Sizes.h: Tweaked size classes to make Membuster happy. This
2037         is the main reason things got slower.
2038
2039         * bmalloc/SmallAllocator.h:
2040         (bmalloc::SmallAllocator::allocate):
2041         * bmalloc/SmallTraits.h:
2042         * bmalloc/VMHeap.cpp:
2043         (bmalloc::VMHeap::allocateXSmallChunk):
2044         * bmalloc/VMHeap.h:
2045         (bmalloc::VMHeap::allocateXSmallPage):
2046         (bmalloc::VMHeap::deallocateXSmallPage):
2047         * bmalloc/XSmallAllocator.h: Added.
2048         (bmalloc::XSmallAllocator::isNull):
2049         (bmalloc::XSmallAllocator::canAllocate):
2050         (bmalloc::XSmallAllocator::XSmallAllocator):
2051         (bmalloc::XSmallAllocator::line):
2052         (bmalloc::XSmallAllocator::allocate):
2053         (bmalloc::XSmallAllocator::objectCount):
2054         (bmalloc::XSmallAllocator::derefCount):
2055         (bmalloc::XSmallAllocator::refill):
2056         * bmalloc/XSmallChunk.h: Added.
2057         * bmalloc/XSmallLine.h: Added.
2058         * bmalloc/XSmallPage.h: Added.
2059         * bmalloc/XSmallTraits.h: Added.
2060         * bmalloc/bmalloc.h:
2061         (bmalloc::api::realloc): Boiler-plate copy, as above.
2062
2063 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
2064
2065         MallocBench should scavenge explicitly instead of waiting
2066         https://bugs.webkit.org/show_bug.cgi?id=131661
2067
2068         Reviewed by Andreas Kling.
2069
2070         Added explicit scavenge support to bmalloc. This isn't a memory win,
2071         since bmalloc's per-thread cache is so small. But it makes testing
2072         simpler.
2073
2074         * bmalloc/Allocator.cpp:
2075         (bmalloc::Allocator::~Allocator):
2076         (bmalloc::Allocator::scavenge):
2077         * bmalloc/Allocator.h:
2078         * bmalloc/Cache.cpp:
2079         (bmalloc::Cache::operator new):
2080         (bmalloc::Cache::operator delete):
2081         (bmalloc::Cache::Cache):
2082         (bmalloc::Cache::scavenge):
2083         * bmalloc/Cache.h:
2084         * bmalloc/Deallocator.cpp:
2085         (bmalloc::Deallocator::~Deallocator):
2086         (bmalloc::Deallocator::scavenge):
2087         * bmalloc/Deallocator.h: Factored existing scavenging code into helper
2088         functions, for reuse.
2089
2090         * bmalloc/Heap.cpp:
2091         (bmalloc::sleep):
2092         (bmalloc::Heap::concurrentScavenge):
2093         (bmalloc::Heap::scavenge):
2094         (bmalloc::Heap::scavengeSmallPages):
2095         (bmalloc::Heap::scavengeMediumPages):
2096         (bmalloc::Heap::scavengeLargeRanges):
2097         * bmalloc/Heap.h: Made scavenge sleep duration a parameter. Forced
2098         scavenging -- in response to a benchmark or a low memory warning --
2099         wants to complete as soon as possible, so its sleep duration is 0.
2100
2101         * bmalloc/bmalloc.h:
2102         (bmalloc::api::scavenge):
2103         * bmalloc/mbmalloc.cpp: Exported the scavenge API for MallocBench's use.
2104
2105 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
2106
2107         Use 4kB pages on Mac
2108         https://bugs.webkit.org/show_bug.cgi?id=131658
2109
2110         Reviewed by Sam Weinig.
2111
2112         This reduces memory use a lot on Membuster:
2113
2114                                                                   base                      patch                                Δ
2115                 Execution Time:
2116                     reddit_memory_warning                         18ms                       17ms                   ^ 1.06x faster
2117                     flickr_memory_warning                         34ms                       36ms                   ! 1.06x slower
2118                     theverge_memory_warning                       39ms                       41ms                   ! 1.05x slower
2119
2120                     <geometric mean>                              29ms                       29ms                   ! 1.02x slower
2121                     <arithmetic mean>                             30ms                       31ms                   ! 1.03x slower
2122                     <harmonic mean>                               27ms                       27ms                    ^ 1.0x faster
2123
2124                 Peak Memory:
2125                     reddit_memory_warning                     16,412kB                   16,436kB                    ! 1.0x bigger
2126                     flickr_memory_warning                     30,120kB                   30,184kB                    ! 1.0x bigger
2127                     theverge_memory_warning                   33,408kB                   33,420kB                    ! 1.0x bigger
2128
2129                     <geometric mean>                          25,466kB                   25,499kB                    ! 1.0x bigger
2130                     <arithmetic mean>                         26,647kB                   26,680kB                    ! 1.0x bigger
2131                     <harmonic mean>                           24,181kB                   24,214kB                    ! 1.0x bigger
2132
2133                 Memory at End:
2134                     reddit_memory_warning                      2,404kB                    1,920kB                  ^ 1.25x smaller
2135                     flickr_memory_warning                      3,764kB                    3,072kB                  ^ 1.23x smaller
2136                     theverge_memory_warning                    3,648kB                    3,132kB                  ^ 1.16x smaller
2137
2138                     <geometric mean>                           3,208kB                    2,644kB                  ^ 1.21x smaller
2139                     <arithmetic mean>                          3,272kB                    2,708kB                  ^ 1.21x smaller
2140                     <harmonic mean>                            3,139kB                    2,574kB                  ^ 1.22x smaller
2141
2142
2143         * bmalloc.xcodeproj/project.pbxproj:
2144         * bmalloc/BPlatform.h: Added.
2145         * bmalloc/VMAllocate.h: Only use 16kB pages on iOS because the page size
2146         is 4kB on Mac.
2147
2148 2014-04-14  Alexey Proskuryakov  <ap@apple.com>
2149
2150         Fixed svn:ignore on bmalloc.xcodeproj, it had erroneous leading spaces.
2151
2152         * bmalloc.xcodeproj: Modified property svn:ignore.
2153
2154 2014-04-13  Geoffrey Garen  <ggaren@apple.com>
2155
2156         Fixed some mbmalloc exports
2157         https://bugs.webkit.org/show_bug.cgi?id=131599
2158
2159         Reviewed by Ryosuke Niwa.
2160
2161         * bmalloc.xcodeproj/project.pbxproj: Made some headers a private part
2162         of the project, so we can call them from API.
2163
2164         * bmalloc/mbmalloc.cpp: Marked the mbmalloc functions with default
2165         visibility, so they show up as exported in the .dylib.
2166
2167 2014-04-09  Geoffrey Garen  <ggaren@apple.com>
2168
2169         Put bmalloc headers in the right place
2170         https://bugs.webkit.org/show_bug.cgi?id=131464
2171
2172         Reviewed by Mark Rowe.
2173
2174         * Configurations/bmalloc.xcconfig: Set PRIVATE_HEADERS_FOLDER_PATH to
2175         specify that we don't just want to dump all of our generically-named
2176         headers into /usr/local/include.
2177
2178 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
2179
2180         Made bmalloc more #include friendly
2181         https://bugs.webkit.org/show_bug.cgi?id=131386
2182
2183         Reviewed by Andreas Kling.
2184
2185         Marked a bunch of headers private so they can be used from client code
2186         that #includes bmalloc.h.
2187
2188         Renamed ASSERT macros to BASSERT. This matches their header, which already
2189         had to be renamed, and fixes conflicts with WTF's ASSERT macros.
2190
2191         * bmalloc.xcodeproj/project.pbxproj:
2192         * bmalloc/Allocator.cpp:
2193         (bmalloc::Allocator::allocateSlowCase):
2194         * bmalloc/AsyncTask.h:
2195         (bmalloc::Function>::runSlowCase):
2196         * bmalloc/BAssert.h:
2197         * bmalloc/BoundaryTag.h:
2198         (bmalloc::BoundaryTag::setSize):
2199         * bmalloc/BoundaryTagInlines.h:
2200         (bmalloc::validate):
2201         (bmalloc::BoundaryTag::init):
2202         (bmalloc::BoundaryTag::deallocate):
2203         (bmalloc::BoundaryTag::splitLarge):
2204         (bmalloc::BoundaryTag::allocate):
2205         * bmalloc/Chunk.h:
2206         * bmalloc/Deallocator.cpp:
2207         (bmalloc::Deallocator::processObjectLog):
2208         (bmalloc::Deallocator::deallocateSlowCase):
2209         * bmalloc/Deallocator.h:
2210         (bmalloc::Deallocator::deallocateFastCase):
2211         * bmalloc/FixedVector.h:
2212         (bmalloc::Capacity>::operator):
2213         (bmalloc::Capacity>::push):
2214         (bmalloc::Capacity>::pop):
2215         (bmalloc::Capacity>::shrink):
2216         * bmalloc/Heap.cpp:
2217         (bmalloc::Heap::allocateLarge):
2218         * bmalloc/LargeChunk.h:
2219         (bmalloc::LargeChunk::get):
2220         (bmalloc::LargeChunk::endTag):
2221         * bmalloc/Line.h:
2222         (bmalloc::Line<Traits>::concurrentRef):
2223         (bmalloc::Line<Traits>::deref):
2224         * bmalloc/MediumAllocator.h:
2225         (bmalloc::MediumAllocator::allocate):
2226         * bmalloc/ObjectType.h:
2227         (bmalloc::isSmall):
2228         * bmalloc/Page.h:
2229         (bmalloc::Page<Traits>::ref):
2230         (bmalloc::Page<Traits>::deref):
2231         * bmalloc/PerThread.h:
2232         (bmalloc::PerThread<T>::getSlowCase):
2233         * bmalloc/SegregatedFreeList.cpp:
2234         (bmalloc::SegregatedFreeList::SegregatedFreeList):
2235         (bmalloc::SegregatedFreeList::insert):
2236         * bmalloc/SmallAllocator.h:
2237         (bmalloc::SmallAllocator::allocate):
2238         (bmalloc::SmallAllocator::refill):
2239         * bmalloc/Syscall.h:
2240         * bmalloc/VMAllocate.h:
2241         (bmalloc::vmValidate):
2242         (bmalloc::vmAllocate):
2243         (bmalloc::vmDeallocatePhysicalPagesSloppy):
2244         * bmalloc/Vector.h:
2245         (bmalloc::Vector<T>::operator):
2246         (bmalloc::Vector<T>::pop):
2247         (bmalloc::Vector<T>::shrink):
2248         * bmalloc/XLargeChunk.h:
2249         (bmalloc::XLargeChunk::range):
2250         (bmalloc::XLargeChunk::size):
2251
2252 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
2253
2254         Removed an unused file.
2255
2256         Unreviewed.
2257
2258         * bmalloc/AsyncTask.cpp: Removed.
2259
2260 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
2261
2262         Build bmalloc on Mac
2263         https://bugs.webkit.org/show_bug.cgi?id=131333
2264
2265         Reviewed by Mark Rowe.
2266
2267         * Makefile: Added. For make clients.
2268
2269         These files are required for building any project in WebKit. I copied
2270         them from WTF:
2271         * Configurations: Added.
2272         * Configurations/Base.xcconfig: Added.
2273         * Configurations/DebugRelease.xcconfig: Added.
2274         * Configurations/bmalloc.xcconfig: Added.
2275         * Configurations/iOS.xcconfig: Added.
2276         * Configurations/mbmalloc.xcconfig: Added.
2277
2278         * bmalloc.xcodeproj/project.pbxproj: I removed per-project-file stuff
2279         from here because everything is in .xcconfig files now.
2280
2281         I had to fix a bunch of minor warnings, since they're enabled in our
2282         .xcconfig files:
2283
2284         * bmalloc/AsyncTask.h:
2285         (bmalloc::Function>::AsyncTask):
2286         * bmalloc/BAssert.h:
2287         * bmalloc/BoundaryTagInlines.h:
2288         (bmalloc::validate):
2289         * bmalloc/Heap.cpp:
2290         (bmalloc::Heap::Heap):
2291         (bmalloc::Heap::allocateLarge):
2292         (bmalloc::Heap::deallocateLarge):
2293         * bmalloc/Mutex.h:
2294         (bmalloc::Mutex::Mutex): Deleted.
2295         * bmalloc/VMAllocate.h:
2296         (bmalloc::vmValidate):
2297         * bmalloc/mbmalloc.cpp:
2298
2299 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
2300
2301         bmalloc: Fixed a leak in the per-thread cache
2302         https://bugs.webkit.org/show_bug.cgi?id=131330
2303
2304         Reviewed by Andreas Kling.
2305
2306         Remember to deallocate our line caches upon thread exit.
2307
2308         * bmalloc/Deallocator.cpp:
2309         (bmalloc::Deallocator::~Deallocator):
2310
2311 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
2312
2313         bmalloc: rolled out the tryLock experiment
2314         https://bugs.webkit.org/show_bug.cgi?id=131328
2315
2316         Reviewed by Andreas Kling.
2317
2318         It wasn't a speedup.
2319
2320         * bmalloc.xcodeproj/project.pbxproj:
2321         * bmalloc/Allocator.cpp:
2322         (bmalloc::Allocator::processSmallAllocatorLog):
2323         (bmalloc::Allocator::processMediumAllocatorLog):
2324         * bmalloc/Deallocator.cpp:
2325         (bmalloc::Deallocator::processObjectLog):
2326         (bmalloc::Deallocator::deallocateSlowCase):
2327         (bmalloc::Deallocator::deallocateSmallLine):
2328         (bmalloc::Deallocator::deallocateMediumLine):
2329         * bmalloc/Deallocator.h:
2330         (bmalloc::Deallocator::deallocateFastCase):
2331         * bmalloc/Heap.h:
2332         (bmalloc::Heap::deallocateSmallLine):
2333         (bmalloc::Heap::deallocateMediumLine):
2334         * bmalloc/Line.h:
2335         (bmalloc::Line<Traits>::deref):
2336         * bmalloc/Page.h:
2337         (bmalloc::Page<Traits>::deref):
2338
2339 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
2340
2341         bmalloc
2342         https://bugs.webkit.org/show_bug.cgi?id=131170
2343
2344         Reviewed by Andreas Kling.
2345
2346         Initial commit.
2347
2348         * bmalloc: Added.
2349         * bmalloc.xcodeproj: Added.
2350         * bmalloc.xcodeproj/project.pbxproj: Added.
2351         * bmalloc/Algorithm.h: Added.
2352         (bmalloc::max):
2353         (bmalloc::min):
2354         (bmalloc::mask):
2355         (bmalloc::test):
2356         (bmalloc::roundUpToMultipleOf):
2357         (bmalloc::roundDownToMultipleOf):
2358         (bmalloc::sizeOf):
2359         (bmalloc::bitCount):
2360         (bmalloc::isPowerOfTwo):
2361         * bmalloc/Allocator.cpp: Added.
2362         (bmalloc::Allocator::Allocator):
2363         (bmalloc::Allocator::~Allocator):
2364         (bmalloc::Allocator::log):
2365         (bmalloc::Allocator::processSmallAllocatorLog):
2366         (bmalloc::Allocator::processMediumAllocatorLog):
2367         (bmalloc::Allocator::allocateLarge):
2368         (bmalloc::Allocator::allocateXLarge):
2369         (bmalloc::Allocator::allocateMedium):
2370         (bmalloc::Allocator::allocateSlowCase):
2371         * bmalloc/Allocator.h: Added.
2372         (bmalloc::Allocator::smallAllocatorFor):
2373         (bmalloc::Allocator::allocateFastCase):
2374         (bmalloc::Allocator::allocate):
2375         * bmalloc/AsyncTask.cpp: Added.
2376         (bmalloc::AsyncTask<Function>::runSlowCase):
2377         (bmalloc::AsyncTask<Function>::pthreadEntryPoint):
2378         (bmalloc::AsyncTask<Function>::entryPoint):
2379         * bmalloc/AsyncTask.h: Added.
2380         (bmalloc::Function>::AsyncTask):
2381         (bmalloc::Function>::join):
2382         (bmalloc::Function>::run):
2383         (bmalloc::Function>::runSlowCase):
2384         (bmalloc::Function>::pthreadEntryPoint):
2385         (bmalloc::Function>::entryPoint):
2386         * bmalloc/BAssert.h: Added.
2387         * bmalloc/BeginTag.h: Added.
2388         (bmalloc::BeginTag::isInFreeList):
2389         * bmalloc/BoundaryTag.h: Added.
2390         (bmalloc::BoundaryTag::isXLarge):
2391         (bmalloc::BoundaryTag::setXLarge):
2392         (bmalloc::BoundaryTag::isFree):
2393         (bmalloc::BoundaryTag::setFree):
2394         (bmalloc::BoundaryTag::isEnd):
2395         (bmalloc::BoundaryTag::setEnd):
2396         (bmalloc::BoundaryTag::hasPhysicalPages):
2397         (bmalloc::BoundaryTag::setHasPhysicalPages):
2398         (bmalloc::BoundaryTag::isNull):
2399         (bmalloc::BoundaryTag::clear):
2400         (bmalloc::BoundaryTag::size):
2401         (bmalloc::BoundaryTag::setSize):
2402         (bmalloc::BoundaryTag::prev):
2403         (bmalloc::BoundaryTag::next):
2404         * bmalloc/BoundaryTagInlines.h: Added.
2405         (bmalloc::validate):
2406         (bmalloc::validatePrev):
2407         (bmalloc::validateNext):
2408         (bmalloc::BoundaryTag::init):
2409         (bmalloc::BoundaryTag::mergeLargeLeft):
2410         (bmalloc::BoundaryTag::mergeLargeRight):
2411         (bmalloc::BoundaryTag::mergeLarge):
2412         (bmalloc::BoundaryTag::deallocate):
2413         (bmalloc::BoundaryTag::splitLarge):
2414         (bmalloc::BoundaryTag::allocate):
2415         * bmalloc/Cache.cpp: Added.
2416         (bmalloc::Cache::operator new):
2417         (bmalloc::Cache::operator delete):
2418         (bmalloc::Cache::Cache):
2419         (bmalloc::Cache::allocateSlowCase):
2420         (bmalloc::Cache::allocateSlowCaseNullCache):
2421         (bmalloc::Cache::deallocateSlowCase):
2422         (bmalloc::Cache::deallocateSlowCaseNullCache):
2423         * bmalloc/Cache.h: Added.
2424         (bmalloc::Cache::allocator):
2425         (bmalloc::Cache::deallocator):
2426         (bmalloc::Cache::allocateFastCase):
2427         (bmalloc::Cache::deallocateFastCase):
2428         (bmalloc::Cache::allocate):
2429         (bmalloc::Cache::deallocate):
2430         * bmalloc/Chunk.h: Added.
2431         (bmalloc::Chunk::begin):
2432         (bmalloc::Chunk::end):
2433         (bmalloc::Chunk::lines):
2434         (bmalloc::Chunk::pages):
2435         * bmalloc/Deallocator.cpp: Added.
2436         (bmalloc::Deallocator::Deallocator):
2437         (bmalloc::Deallocator::~Deallocator):
2438         (bmalloc::Deallocator::deallocateLarge):
2439         (bmalloc::Deallocator::deallocateXLarge):
2440         (bmalloc::Deallocator::processObjectLog):
2441         (bmalloc::Deallocator::deallocateSlowCase):
2442         (bmalloc::Deallocator::deallocateSmallLine):
2443         (bmalloc::Deallocator::allocateSmallLine):
2444         (bmalloc::Deallocator::deallocateMediumLine):
2445         (bmalloc::Deallocator::allocateMediumLine):
2446         * bmalloc/Deallocator.h: Added.
2447         (bmalloc::Deallocator::deallocateFastCase):
2448         (bmalloc::Deallocator::deallocate):
2449         * bmalloc/EndTag.h: Added.
2450         (bmalloc::EndTag::operator=):
2451         * bmalloc/FixedVector.h: Added.
2452         (bmalloc::FixedVector::begin):
2453         (bmalloc::FixedVector::end):
2454         (bmalloc::FixedVector::size):
2455         (bmalloc::FixedVector::capacity):
2456         (bmalloc::FixedVector::clear):
2457         (bmalloc::FixedVector::isEmpty):
2458         (bmalloc::Capacity>::FixedVector):
2459         (bmalloc::Capacity>::operator):
2460         (bmalloc::Capacity>::push):
2461         (bmalloc::Capacity>::pop):
2462         (bmalloc::Capacity>::shrink):
2463         * bmalloc/Heap.cpp: Added.
2464         (bmalloc::sleep):
2465         (bmalloc::Heap::Heap):
2466         (bmalloc::Heap::concurrentScavenge):
2467         (bmalloc::Heap::scavengeSmallPages):
2468         (bmalloc::Heap::scavengeMediumPages):
2469         (bmalloc::Heap::scavengeLargeRanges):
2470         (bmalloc::Heap::allocateSmallLineSlowCase):
2471         (bmalloc::Heap::allocateMediumLineSlowCase):
2472         (bmalloc::Heap::allocateXLarge):
2473         (bmalloc::Heap::deallocateXLarge):
2474         (bmalloc::Heap::allocateLarge):
2475         (bmalloc::Heap::deallocateLarge):
2476         * bmalloc/Heap.h: Added.
2477         (bmalloc::Heap::deallocateSmallLine):
2478         (bmalloc::Heap::allocateSmallLine):
2479         (bmalloc::Heap::deallocateMediumLine):
2480         (bmalloc::Heap::allocateMediumLine):
2481         * bmalloc/Inline.h: Added.
2482         * bmalloc/LargeChunk.h: Added.
2483         (bmalloc::LargeChunk::begin):
2484         (bmalloc::LargeChunk::end):
2485         (bmalloc::LargeChunk::create):
2486         (bmalloc::LargeChunk::get):
2487         (bmalloc::LargeChunk::beginTag):
2488         (bmalloc::LargeChunk::endTag):
2489         * bmalloc/Line.h: Added.
2490         (bmalloc::Line<Traits>::begin):
2491         (bmalloc::Line<Traits>::end):
2492         (bmalloc::Line<Traits>::concurrentRef):
2493         (bmalloc::Line<Traits>::deref):
2494         * bmalloc/MediumAllocator.h: Added.
2495         (bmalloc::MediumAllocator::isNull):
2496         (bmalloc::MediumAllocator::MediumAllocator):
2497         (bmalloc::MediumAllocator::line):
2498         (bmalloc::MediumAllocator::allocate):
2499         (bmalloc::MediumAllocator::derefCount):
2500         (bmalloc::MediumAllocator::refill):
2501         * bmalloc/MediumChunk.h: Added.
2502         * bmalloc/MediumLine.h: Added.
2503         * bmalloc/MediumPage.h: Added.
2504         * bmalloc/MediumTraits.h: Added.
2505         * bmalloc/Mutex.cpp: Added.
2506         (bmalloc::Mutex::lockSlowCase):
2507         * bmalloc/Mutex.h: Added.
2508         (bmalloc::Mutex::Mutex):
2509         (bmalloc::Mutex::try_lock):
2510         (bmalloc::Mutex::lock):
2511         (bmalloc::Mutex::unlock):
2512         * bmalloc/ObjectType.cpp: Added.
2513         (bmalloc::objectType):
2514         * bmalloc/ObjectType.h: Added.
2515         (bmalloc::isSmallOrMedium):
2516         (bmalloc::isSmall):
2517         * bmalloc/Page.h: Added.
2518         (bmalloc::Page<Traits>::ref):
2519         (bmalloc::Page<Traits>::deref):
2520         (bmalloc::Page<Traits>::refCount):
2521         * bmalloc/PerProcess.h: Added.
2522         (bmalloc::PerProcess::mutex):
2523         (bmalloc::PerProcess<T>::getFastCase):
2524         (bmalloc::PerProcess<T>::get):
2525         (bmalloc::PerProcess<T>::getSlowCase):
2526         * bmalloc/PerThread.h: Added.
2527         (bmalloc::PerThreadStorage<Cache>::get):
2528         (bmalloc::PerThreadStorage<Cache>::init):
2529         (bmalloc::PerThreadStorage::get):
2530         (bmalloc::PerThreadStorage::init):
2531         (bmalloc::PerThread<T>::getFastCase):
2532         (bmalloc::PerThread<T>::get):
2533         (bmalloc::PerThread<T>::destructor):
2534         (bmalloc::PerThread<T>::getSlowCase):
2535         * bmalloc/Range.h: Added.
2536         (bmalloc::Range::Range):
2537         (bmalloc::Range::begin):
2538         (bmalloc::Range::end):
2539         (bmalloc::Range::size):
2540         (bmalloc::Range::operator!):
2541         (bmalloc::Range::operator<):
2542         * bmalloc/SegregatedFreeList.cpp: Added.
2543         (bmalloc::SegregatedFreeList::SegregatedFreeList):
2544         (bmalloc::SegregatedFreeList::insert):
2545         (bmalloc::SegregatedFreeList::takeGreedy):
2546         (bmalloc::SegregatedFreeList::take):
2547         * bmalloc/SegregatedFreeList.h: Added.
2548         * bmalloc/Sizes.h: Added.
2549         * bmalloc/SmallAllocator.h: Added.
2550         (bmalloc::SmallAllocator::isNull):
2551         (bmalloc::SmallAllocator::canAllocate):
2552         (bmalloc::SmallAllocator::SmallAllocator):
2553         (bmalloc::SmallAllocator::line):
2554         (bmalloc::SmallAllocator::allocate):
2555         (bmalloc::SmallAllocator::objectCount):
2556         (bmalloc::SmallAllocator::derefCount):
2557         (bmalloc::SmallAllocator::refill):
2558         * bmalloc/SmallChunk.h: Added.
2559         * bmalloc/SmallLine.h: Added.
2560         * bmalloc/SmallPage.h: Added.
2561         * bmalloc/SmallTraits.h: Added.
2562         * bmalloc/Syscall.h: Added.
2563         * bmalloc/VMAllocate.h: Added.
2564         (bmalloc::vmSize):
2565         (bmalloc::vmValidate):
2566         (bmalloc::vmAllocate):
2567         (bmalloc::vmDeallocate):
2568         (bmalloc::vmDeallocatePhysicalPages):
2569         (bmalloc::vmAllocatePhysicalPages):
2570         (bmalloc::vmDeallocatePhysicalPagesSloppy):
2571         (bmalloc::vmAllocatePhysicalPagesSloppy):
2572         * bmalloc/VMHeap.cpp: Added.
2573         (bmalloc::VMHeap::VMHeap):
2574         (bmalloc::VMHeap::allocateSmallChunk):
2575         (bmalloc::VMHeap::allocateMediumChunk):
2576         (bmalloc::VMHeap::allocateLargeChunk):
2577         * bmalloc/VMHeap.h: Added.
2578         (bmalloc::VMHeap::allocateSmallPage):
2579         (bmalloc::VMHeap::allocateMediumPage):
2580         (bmalloc::VMHeap::allocateLargeRange):
2581         (bmalloc::VMHeap::deallocateSmallPage):
2582         (bmalloc::VMHeap::deallocateMediumPage):
2583         (bmalloc::VMHeap::deallocateLargeRange):
2584         * bmalloc/Vector.h: Added.
2585         (bmalloc::Vector::begin):
2586         (bmalloc::Vector::end):
2587         (bmalloc::Vector::size):
2588         (bmalloc::Vector::capacity):
2589         (bmalloc::Vector::last):
2590         (bmalloc::Vector::pop):
2591         (bmalloc::Vector<T>::Vector):
2592         (bmalloc::Vector<T>::~Vector):
2593         (bmalloc::Vector<T>::operator):
2594         (bmalloc::Vector<T>::push):
2595         (bmalloc::Vector<T>::pop):
2596         (bmalloc::Vector<T>::shrink):
2597         (bmalloc::Vector<T>::reallocateBuffer):
2598         (bmalloc::Vector<T>::shrinkCapacity):
2599         (bmalloc::Vector<T>::growCapacity):
2600         * bmalloc/XLargeChunk.h: Added.
2601         (bmalloc::XLargeChunk::get):
2602         (bmalloc::XLargeChunk::begin):
2603         (bmalloc::XLargeChunk::XLargeChunk):
2604         (bmalloc::XLargeChunk::create):
2605         (bmalloc::XLargeChunk::destroy):
2606         (bmalloc::XLargeChunk::range):
2607         (bmalloc::XLargeChunk::size):
2608         * bmalloc/bmalloc.h: Added.
2609         (bmalloc::api::malloc):
2610         (bmalloc::api::free):
2611         (bmalloc::api::realloc):
2612         * bmalloc/mbmalloc.cpp: Added.
2613