EFL build fix after r180797.
[WebKit-https.git] / Source / bmalloc / ChangeLog
1 2015-02-27  Ryosuke Niwa  <rniwa@webkit.org>
2
3         EFL build fix after r180797.
4
5         * bmalloc/BoundaryTag.h:
6         (bmalloc::BoundaryTag::owner):
7         (bmalloc::BoundaryTag::setOwner):
8
9 2015-02-27  Geoffrey Garen  <ggaren@apple.com>
10
11         bmalloc: Pathological madvise churn on the free(malloc(x)) benchmark
12         https://bugs.webkit.org/show_bug.cgi?id=142058
13
14         Reviewed by Andreas Kling.
15
16         The churn was caused by repeatedly splitting an object with physical
17         pages from an object without, and then merging them back together again.
18         The merge would conservatively forget that we had physical pages, forcing
19         a new call to madvise on the next allocation.
20
21         This patch more strictly segregates objects in the heap from objects in
22         the VM heap, with these changes:
23
24         (1) Objects in the heap are not allowed to merge with objects in the VM
25         heap, and vice versa -- since that would erase our precise knowledge of
26         which physical pages had been allocated.
27
28         (2) The VM heap is exclusively responsible for allocating and deallocating
29         physical pages.
30
31         (3) The heap free list must consider entries for objects that are in the
32         VM heap to be invalid, and vice versa. (This condition can arise
33         because the free list does not eagerly remove items.)
34
35         With these changes, we can know that any valid object in the heap's free
36         list already has physical pages, and does not need to call madvise.
37
38         Note that the VM heap -- as before -- might sometimes contain ranges
39         or pieces of ranges that have physical pages, since we allow splitting
40         of ranges at granularities smaller than the VM page size. These ranges
41         can eventually merge with ranges in the heap during scavenging.
42
43         * bmalloc.xcodeproj/project.pbxproj:
44
45         * bmalloc/BoundaryTag.h:
46         (bmalloc::BoundaryTag::owner):
47         (bmalloc::BoundaryTag::setOwner):
48         (bmalloc::BoundaryTag::initSentinel):
49         (bmalloc::BoundaryTag::hasPhysicalPages): Deleted.
50         (bmalloc::BoundaryTag::setHasPhysicalPages): Deleted. Replaced the concept
51         of "has physical pages" with a bit indicating which heap owns the large
52         object. This is a more precise concept, since the old bit was really a
53         Yes / Maybe bit.
54
55         * bmalloc/Deallocator.cpp:
56
57         * bmalloc/FreeList.cpp: Adopt
58         (bmalloc::FreeList::takeGreedy):
59         (bmalloc::FreeList::take):
60         (bmalloc::FreeList::removeInvalidAndDuplicateEntries):
61         * bmalloc/FreeList.h:
62         (bmalloc::FreeList::push): Added API for considering the owner when
63         deciding if a free list entry is valid.
64
65         * bmalloc/Heap.cpp:
66         (bmalloc::Heap::Heap): Adopt new API.
67
68         (bmalloc::Heap::scavengeLargeRanges): Scavenge all ranges with no minimum,
69         since some ranges might be able to merge with ranges in the VM heap, and
70         they won't be allowed to until we scavenge them.
71
72         (bmalloc::Heap::allocateSmallPage):
73         (bmalloc::Heap::allocateMediumPage):
74         (bmalloc::Heap::allocateLarge): New VM heap API makes this function
75         simpler, since we always get back physical pages now.
76
77         * bmalloc/Heap.h:
78         * bmalloc/LargeObject.h:
79         (bmalloc::LargeObject::end):
80         (bmalloc::LargeObject::owner):
81         (bmalloc::LargeObject::setOwner):
82         (bmalloc::LargeObject::isValidAndFree):
83         (bmalloc::LargeObject::merge): Do not merge objects across heaps since
84         that causes madvise churn.
85         (bmalloc::LargeObject::validateSelf):
86         (bmalloc::LargeObject::init):
87         (bmalloc::LargeObject::hasPhysicalPages): Deleted.
88         (bmalloc::LargeObject::setHasPhysicalPages): Deleted. Propogate the Owner API.
89
90         * bmalloc/Owner.h: Added.
91
92         * bmalloc/SegregatedFreeList.cpp:
93         (bmalloc::SegregatedFreeList::SegregatedFreeList):
94         (bmalloc::SegregatedFreeList::insert):
95         (bmalloc::SegregatedFreeList::takeGreedy):
96         (bmalloc::SegregatedFreeList::take):
97         * bmalloc/SegregatedFreeList.h: Propogate the owner API.
98
99         * bmalloc/VMAllocate.h:
100         (bmalloc::vmDeallocatePhysicalPagesSloppy):
101         (bmalloc::vmAllocatePhysicalPagesSloppy): Clarified these functions and
102         removed an edge case.
103
104         * bmalloc/VMHeap.cpp:
105         (bmalloc::VMHeap::VMHeap):
106         * bmalloc/VMHeap.h:
107         (bmalloc::VMHeap::allocateSmallPage):
108         (bmalloc::VMHeap::allocateMediumPage):
109         (bmalloc::VMHeap::allocateLargeObject):
110         (bmalloc::VMHeap::deallocateLargeObject): Be sure to give each object
111         a new chance to merge, since it might have been prohibited from merging
112         before by virtue of not being in the VM heap.
113
114         (bmalloc::VMHeap::allocateLargeRange): Deleted.
115         (bmalloc::VMHeap::deallocateLargeRange): Deleted.
116
117 2015-02-26  Geoffrey Garen  <ggaren@apple.com>
118
119         bmalloc: Large object free list can grow infinitely
120         https://bugs.webkit.org/show_bug.cgi?id=142055
121
122         Reviewed by Andreas Kling.
123
124         By design, we don't eagerly remove large objects from the free list.
125         This creates two simple pathologies:
126
127             (1) If you free and then allocate the same object repeatedly, it will
128             duplicate itself in the free list repeatedly. Since it is never
129             invalid at the time of allocation, it will never be removed.
130
131             (2) If you split and then merge the same object repeatedly, it will
132             duplicate its split sibling in the free list repeatedly. If its
133             sibling is in a separate free list size class, it will never be
134             consulted at the time of allocation, so it will never be removed.
135
136         So, a simple "while (1) { free(malloc(x)); }" causes infinite memory
137         use in the free list.
138
139         The solution in this patch is a simple helper to remove garbage from the
140         free list if it grows too large. This pathology is not common, so the
141         cost is OK.
142
143         Long-term, perhaps we should rethink the laziness of these free lists.
144
145         * bmalloc/BoundaryTag.h:
146         (bmalloc::BoundaryTag::isMarked):
147         (bmalloc::BoundaryTag::setMarked): New bit, used by free list GC.
148
149         * bmalloc/FreeList.cpp:
150         (bmalloc::FreeList::removeInvalidAndDuplicateEntries): The GC algorithm.
151
152         * bmalloc/FreeList.h:
153         (bmalloc::FreeList::FreeList):
154         (bmalloc::FreeList::push): Invoke the GC if we're getting huge.
155
156         * bmalloc/LargeObject.h:
157         (bmalloc::LargeObject::isMarked):
158         (bmalloc::LargeObject::setMarked):
159         (bmalloc::LargeObject::validateSelf): Expose the new bit.
160
161         * bmalloc/Sizes.h: New constant to control GC frequency.
162
163 2015-02-26  Csaba Osztrogonác  <ossy@webkit.org>
164
165         URTBF after r180693.
166
167         * CMakeLists.txt:
168
169 2015-02-26  Geoffrey Garen  <ggaren@apple.com>
170
171         Try to fix the Mac build.
172
173         Unreviewed.
174
175         * bmalloc.xcodeproj/project.pbxproj: Make FreeList.h available.
176
177 2015-02-26  Geoffrey Garen  <ggaren@apple.com>
178
179         bmalloc: Refactored SegregatedFreeList and BoundaryTag::init
180         https://bugs.webkit.org/show_bug.cgi?id=142049
181
182         Reviewed by Anders Carlsson.
183
184         Split out a FreeList class from SegregatedFreeList. This will make it
185         easier to add behaviors on free list insertion and removal -- and it's
186         probably how I should have designed things at the start.
187
188         Moved BoundaryTag::init into LargeObject, since all the related logic
189         lives in LargeObject now too, and this allows us to remove BoundaryTagInlines.h.
190
191         * bmalloc.xcodeproj/project.pbxproj:
192         * bmalloc/BoundaryTagInlines.h: Removed.
193         * bmalloc/FreeList.cpp: Copied from Source/bmalloc/bmalloc/SegregatedFreeList.cpp.
194         (bmalloc::FreeList::takeGreedy):
195         (bmalloc::FreeList::take):
196         (bmalloc::SegregatedFreeList::SegregatedFreeList): Deleted.
197         (bmalloc::SegregatedFreeList::insert): Deleted.
198         (bmalloc::SegregatedFreeList::takeGreedy): Deleted.
199         (bmalloc::SegregatedFreeList::take): Deleted.
200         * bmalloc/FreeList.h: Copied from Source/bmalloc/bmalloc/SegregatedFreeList.h.
201         (bmalloc::FreeList::push):
202         * bmalloc/LargeObject.h:
203         (bmalloc::LargeObject::init):
204         * bmalloc/SegregatedFreeList.cpp:
205         (bmalloc::SegregatedFreeList::SegregatedFreeList):
206         (bmalloc::SegregatedFreeList::insert):
207         (bmalloc::SegregatedFreeList::takeGreedy):
208         (bmalloc::SegregatedFreeList::take):
209         * bmalloc/SegregatedFreeList.h:
210         * bmalloc/Sizes.h:
211         * bmalloc/VMHeap.cpp:
212         (bmalloc::VMHeap::grow):
213
214 2015-02-26  Geoffrey Garen  <ggaren@apple.com>
215
216         bmalloc: free up a bit in BoundaryTag
217         https://bugs.webkit.org/show_bug.cgi?id=142048
218
219         Reviewed by Brady Eidson.
220
221         We were wasting a bit by accident, and I need one now.
222
223         * bmalloc/Algorithm.h:
224         (bmalloc::rightShift): Deleted. Not needed, now that I've simplified
225         the math.
226
227         * bmalloc/BoundaryTag.h: Since each boundary tag bucket is 1024 bytes
228         long, the maximum offset into a bucket is 1023.
229
230         You need 5 bits to count up to 1024, but only 4 to count up to 1023.
231         
232         Math is hard.
233
234         (bmalloc::BoundaryTag::compactBegin): Switched to division because it
235         is simpler, and easier to match up with our ASSERT. The compiler will
236         turn division by constant power of two into a shift for us.
237
238         (bmalloc::BoundaryTag::setRange): Added an ASSERT for compactBegin
239         because we do encode it, so we should ASSERT that encoding did not
240         lose information.
241
242         * bmalloc/Sizes.h: Shifting is no longer used since we use division
243         instead.
244
245 2015-02-24  Stephanie Lewis  <slewis@apple.com>
246
247         Rolling out http://trac.webkit.org/changeset/180430 as it causes the PLT to crash.
248         <rdar://problem/19948015>
249
250         Unreviewed.
251
252         * bmalloc/VMHeap.cpp:
253         (bmalloc::VMHeap::grow):
254         * bmalloc/VMHeap.h:
255         * bmalloc/Zone.cpp:
256         (bmalloc::Zone::Zone):
257         (bmalloc::Zone::size): Deleted.
258         * bmalloc/Zone.h:
259
260 2015-02-24  Geoffrey Garen  <ggaren@apple.com>
261
262         bmalloc: Added a little more abstraction for large objects
263         https://bugs.webkit.org/show_bug.cgi?id=141978
264
265         Reviewed by Sam Weinig.
266
267         Previously, each client needed to manage the boundary tags of
268         a large object using free functions. This patch introduces a LargeObject
269         class that does things a little more automatically.
270
271         * bmalloc.xcodeproj/project.pbxproj:
272
273         * bmalloc/Allocator.cpp:
274         (bmalloc::Allocator::reallocate): Use the new LargeObject class.
275
276         * bmalloc/BeginTag.h:
277         (bmalloc::BeginTag::isInFreeList): Deleted. Moved this logic into the
278         LargeObject class.
279
280         * bmalloc/BoundaryTag.h:
281         (bmalloc::BoundaryTag::isSentinel):
282         (bmalloc::BoundaryTag::compactBegin):
283         (bmalloc::BoundaryTag::setRange):
284         (bmalloc::BoundaryTag::initSentinel): Added an explicit API for sentinels,
285         which we used to create and test for implicitly.
286
287         * bmalloc/BoundaryTagInlines.h:
288         (bmalloc::BoundaryTag::init):
289         (bmalloc::validate): Deleted.
290         (bmalloc::validatePrev): Deleted.
291         (bmalloc::validateNext): Deleted.
292         (bmalloc::BoundaryTag::mergeLeft): Deleted.
293         (bmalloc::BoundaryTag::mergeRight): Deleted.
294         (bmalloc::BoundaryTag::merge): Deleted.
295         (bmalloc::BoundaryTag::deallocate): Deleted.
296         (bmalloc::BoundaryTag::split): Deleted.
297         (bmalloc::BoundaryTag::allocate): Deleted. Moved this logic into the
298         LargeObject class.
299
300         * bmalloc/EndTag.h:
301         (bmalloc::EndTag::init):
302         (bmalloc::EndTag::operator=): Deleted. Re-reading this code, I found
303         special behavior in the assignment operator to be a surprising API.
304         So, I replaced the assignment operation with an explicit initializing
305         function.
306
307         * bmalloc/Heap.cpp:
308         (bmalloc::Heap::scavengeLargeRanges):
309         (bmalloc::Heap::allocateXLarge):
310         (bmalloc::Heap::findXLarge):
311         (bmalloc::Heap::deallocateXLarge):
312         (bmalloc::Heap::allocateLarge):
313         (bmalloc::Heap::deallocateLarge):
314         * bmalloc/Heap.h: No behavior changes here -- just adopting the
315         LargeObject interface.
316
317         * bmalloc/LargeObject.h: Added.
318         (bmalloc::LargeObject::operator!):
319         (bmalloc::LargeObject::begin):
320         (bmalloc::LargeObject::size):
321         (bmalloc::LargeObject::range):
322         (bmalloc::LargeObject::LargeObject):
323         (bmalloc::LargeObject::setFree):
324         (bmalloc::LargeObject::isFree):
325         (bmalloc::LargeObject::hasPhysicalPages):
326         (bmalloc::LargeObject::setHasPhysicalPages):
327         (bmalloc::LargeObject::isValidAndFree):
328         (bmalloc::LargeObject::merge):
329         (bmalloc::LargeObject::split):
330         (bmalloc::LargeObject::validateSelf):
331         (bmalloc::LargeObject::validate): Moved this code into a class, out of
332         BoundaryTag free functions.
333
334         New to the class are these features:
335
336             (1) Every reference to an object is validated upon creation and use.
337
338             (2) There's an explicit API for "This is a reference to an object
339             that might be stale (the DoNotValidate API)".
340
341             (3) The begin and end tags are kept in sync automatically.
342
343         * bmalloc/SegregatedFreeList.cpp:
344         (bmalloc::SegregatedFreeList::insert):
345         (bmalloc::SegregatedFreeList::takeGreedy):
346         (bmalloc::SegregatedFreeList::take):
347         * bmalloc/SegregatedFreeList.h: Adopt the LargeObject interface.
348
349         * bmalloc/VMHeap.cpp:
350         (bmalloc::VMHeap::grow):
351         * bmalloc/VMHeap.h:
352         (bmalloc::VMHeap::allocateLargeRange):
353         (bmalloc::VMHeap::deallocateLargeRange): Adopt the LargeObject interface.
354
355 2015-02-20  Geoffrey Garen  <ggaren@apple.com>
356
357         bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
358         https://bugs.webkit.org/show_bug.cgi?id=141802
359
360         Reviewed by Andreas Kling.
361
362         Rolling back in with a fix for a crash seen while using GuardMalloc.
363
364         * bmalloc/VMHeap.cpp:
365         (bmalloc::VMHeap::grow):
366         * bmalloc/VMHeap.h:
367         * bmalloc/Zone.cpp: Re-land the old patch.
368
369         (bmalloc::Zone::size): Be sure to implement the size() function since
370         it's accessible indirectly via the malloc_zone_from_ptr public API --
371         and GuardMalloc calls it all the time.
372
373         (bmalloc::Zone::Zone):
374         * bmalloc/Zone.h: Re-land the old patch.
375
376 2015-02-19  Commit Queue  <commit-queue@webkit.org>
377
378         Unreviewed, rolling out r180363.
379         https://bugs.webkit.org/show_bug.cgi?id=141814
380
381         Caused >50 crashes when running LayoutTests in GuardMalloc or
382         ASAN modes. (Requested by jernoble on #webkit).
383
384         Reverted changeset:
385
386         "bmalloc should implement malloc introspection (to stop false-
387         positive leaks when MallocStackLogging is off)"
388         https://bugs.webkit.org/show_bug.cgi?id=141802
389         http://trac.webkit.org/changeset/180363
390
391 2015-02-19  Geoffrey Garen  <ggaren@apple.com>
392
393         bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
394         https://bugs.webkit.org/show_bug.cgi?id=141802
395
396         Reviewed by Andreas Kling.
397
398         Fixed a last-minute type.
399
400         The macro is OS, not PLATFORM.
401
402         * bmalloc/VMHeap.cpp:
403         (bmalloc::VMHeap::grow):
404         * bmalloc/VMHeap.h:
405         * bmalloc/Zone.h:
406
407 2015-02-19  Geoffrey Garen  <ggaren@apple.com>
408
409         bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
410         https://bugs.webkit.org/show_bug.cgi?id=141802
411
412         Reviewed by Andreas Kling.
413
414         This patch does the bare minimum to stop false positive leaks from
415         being reported by the Darwin leaks tool. We register each super chunk
416         as a single object, and then request that the leaks tool scan it.
417
418         * bmalloc.xcodeproj/project.pbxproj: Added an abstraction for the malloc
419         zone introspection API.
420
421         * bmalloc/Algorithm.h: Missing #include.
422
423         * bmalloc/VMHeap.cpp:
424         (bmalloc::VMHeap::grow):
425         * bmalloc/VMHeap.h: Adopt the new abstraction.
426
427         * bmalloc/Zone.cpp: Added.
428         (bmalloc::remoteRead): Helper for reading an object out of another process.
429         (bmalloc::Zone::enumerator):
430         (bmalloc::Zone::Zone): Register a malloc zone so that we will participate
431         in introspection.
432
433         * bmalloc/Zone.h: Added.
434         (bmalloc::Zone::superChunks):
435         (bmalloc::Zone::addSuperChunk): Use a non-dynamically-allocated vector
436         since our dynamic allocations will not be scanned by leaks since they
437         will have the malloc VM tag.
438
439 2015-02-18  Geoffrey Garen  <ggaren@apple.com>
440
441         bmalloc: VMHeap should keep a record of all of its VM ranges (for malloc introspection)
442         https://bugs.webkit.org/show_bug.cgi?id=141759
443
444         Reviewed by Andreas Kling.
445
446         * bmalloc.xcodeproj/project.pbxproj:
447         * bmalloc/SuperChunk.h: Added.
448         (bmalloc::SuperChunk::create):
449         (bmalloc::SuperChunk::SuperChunk):
450         (bmalloc::SuperChunk::smallChunk):
451         (bmalloc::SuperChunk::mediumChunk):
452         (bmalloc::SuperChunk::largeChunk): Factored out super chunk creation
453         into a separate class, for clarity and type safety.
454
455         * bmalloc/VMHeap.cpp:
456         (bmalloc::VMHeap::grow):
457         (bmalloc::VMHeap::allocateSuperChunk): Renamed "allocateSuperChunk" to
458         "grow" because Andreas found "allocateSuperChunk" to be unclear.
459
460         * bmalloc/VMHeap.h: Track all our VM ranges. We will use this information
461         for malloc introspection.
462
463         (bmalloc::VMHeap::allocateSmallPage):
464         (bmalloc::VMHeap::allocateMediumPage):
465         (bmalloc::VMHeap::allocateLargeRange): Updated for renames.
466
467 2015-02-18  Zan Dobersek  <zdobersek@igalia.com>
468
469         Build bmalloc through CMake as a static library. It's then linked either
470         into the WTF library (if built as a shared library) or into the JSC and
471         WebKit2 libraries. There's no need to build it as a standalone shared library.
472
473         Rubber-stamped by Carlos Garcia Campos.
474
475         * CMakeLists.txt:
476
477 2015-02-13  Gyuyoung Kim  <gyuyoung.kim@samsung.com>
478
479         [BMalloc] Add a FIXME comment for memory alignas
480         https://bugs.webkit.org/show_bug.cgi?id=141556
481
482         Reviewed by Csaba Osztrogonác.
483
484         * bmalloc/Chunk.h: Add a FIXME comment.
485         * bmalloc/LargeChunk.h: ditto.
486
487 2015-02-11  Csaba Osztrogonác  <ossy@webkit.org>
488
489         bmalloc buildfix on 32 bit Linux (x86/ARM)
490         https://bugs.webkit.org/show_bug.cgi?id=141472
491
492         Reviewed by Gyuyoung Kim.
493
494         * bmalloc/Algorithm.h:
495         (bmalloc::roundUpToMultipleOf):
496         * bmalloc/FixedVector.h:
497         (bmalloc::FixedVector::clear):
498         * bmalloc/Sizes.h:
499         (bmalloc::Sizes::sizeClass):
500
501 2015-02-11  Gyuyoung Kim  <gyuyoung.kim@samsung.com>
502
503         [EFL][GTK] Use bmalloc instead of tcmalloc
504         https://bugs.webkit.org/show_bug.cgi?id=140162
505
506         Reviewed by Carlos Garcia Campos.
507
508         Support to use bmalloc on EFL and GTK ports.
509
510         * CMakeLists.txt: Added.
511         * bmalloc/Allocator.cpp:
512         (bmalloc::Allocator::allocate):
513             Fix unused return value caused by posix_memalign().
514         * bmalloc/AsyncTask.h:
515         * bmalloc/BoundaryTag.h:
516         (bmalloc::BoundaryTag::clear):
517         * bmalloc/Chunk.h:
518             Change Traits::Page with Traits::PageType in order to fix
519             -fpermitive build error on EFL and GTK port. 
520         * bmalloc/EndTag.h:
521         (bmalloc::EndTag::operator=):
522         * bmalloc/Line.h: ditto.
523         * bmalloc/MediumTraits.h:
524         * bmalloc/Page.h: ditto.
525         * bmalloc/PerThread.h:
526             EFL port doesn't support __has_include definition yet.
527             Define HAVE_PTHREAD_MACHDEP_H according to check if __has_include is supported.
528         * bmalloc/SmallTraits.h: ditto.
529         * bmalloc/VMAllocate.h:
530         (bmalloc::vmDeallocatePhysicalPages):
531         (bmalloc::vmAllocatePhysicalPages):
532         * bmalloc/Vector.h:
533         (bmalloc::Vector<T>::push):
534         (bmalloc::Vector<T>::reallocateBuffer):
535
536 2015-01-31  Sam Weinig  <sam@webkit.org>
537
538         Remove even more Mountain Lion support
539         https://bugs.webkit.org/show_bug.cgi?id=141124
540
541         Reviewed by Alexey Proskuryakov.
542
543         * Configurations/Base.xcconfig:
544         * Configurations/DebugRelease.xcconfig:
545
546 2015-01-30  Geoffrey Garen  <ggaren@apple.com>
547
548         GC marking threads should clear malloc caches
549         https://bugs.webkit.org/show_bug.cgi?id=141097
550
551         Reviewed by Andreas Kling.
552
553         Split the scavenging API into per-thread vs global, so that you can
554         request to scavenge your own thread without scavenging the whole heap.
555
556         * bmalloc/Cache.cpp:
557         (bmalloc::Cache::scavenge):
558         * bmalloc/bmalloc.h:
559         (bmalloc::api::scavengeThisThread):
560         (bmalloc::api::scavenge):
561
562 2015-01-28  Dana Burkart  <dburkart@apple.com>
563
564         Move ASan flag settings from DebugRelease.xcconfig to Base.xcconfig
565         https://bugs.webkit.org/show_bug.cgi?id=136765
566
567         Reviewed by Alexey Proskuryakov.
568
569         * Configurations/Base.xcconfig:
570         * Configurations/DebugRelease.xcconfig:
571
572 2015-01-21  Geoffrey Garen  <ggaren@apple.com>
573
574         bmalloc: support aligned allocation
575         https://bugs.webkit.org/show_bug.cgi?id=140732
576
577         Reviewed by Andreas Kling.
578
579         * bmalloc/Allocator.cpp:
580         (bmalloc::Allocator::allocate): New function for aligned allocation.
581
582         Small and medium requests just allocate and free until they find an
583         aligned pointer. This is slightly inefficient in the worst case, but
584         still constant-time with little-to-no space overhead.
585
586         Large requests use a new API that requires the client to specify both
587         its ideal size and alignment, and the worst-case size you would have to
588         allocate in order to produce some interior pointer of the requested size
589         and alignment. We put the burden of this calculation on the client
590         because it simplifies things if we guarantee that allocation won't fail.
591
592         XLarge requests are easy: we just forward them to vmAllocate, which
593         already supported aligned requests.
594
595         * bmalloc/BoundaryTag.h:
596         * bmalloc/BoundaryTagInlines.h:
597         (bmalloc::BoundaryTag::mergeLeft):
598         (bmalloc::BoundaryTag::mergeRight):
599         (bmalloc::BoundaryTag::merge):
600         (bmalloc::BoundaryTag::deallocate):
601         (bmalloc::BoundaryTag::split):
602         (bmalloc::BoundaryTag::allocate): No behavior change here. I just
603         refactored the interface to remove some reference out parameters in
604         order to clarify what changes and what doesn't.
605
606         * bmalloc/Heap.cpp:
607         (bmalloc::Heap::allocateXLarge): Added an alignment API.
608
609         (bmalloc::Heap::allocateLarge):
610         * bmalloc/Heap.h: Added an alignment API. I split out allocateLarge into
611         a few variants, so aligned and unaligned allocation could share some code.
612
613         * bmalloc/SegregatedFreeList.cpp:
614         (bmalloc::SegregatedFreeList::take):
615         * bmalloc/SegregatedFreeList.h: Changed to use a separate, explicit API
616         for aligned allocation. It turns out that the aligned path is pretty
617         different, since it ends up searching for two potential ways to satisfy
618         an allocation: either large enough and aligned, or large enough to split
619         into something not aligned and something large enough and aligned.
620
621         * bmalloc/VMAllocate.h:
622         (bmalloc::vmAllocate): Switched alignment to come before size because
623         that's how the memalign API specifies it.
624
625         * bmalloc/VMHeap.h:
626         (bmalloc::VMHeap::allocateLargeRange): Added an alignment API.
627
628 2015-01-20  Geoffrey Garen  <ggaren@apple.com>
629
630         bmalloc: a little bit of cleanup
631         https://bugs.webkit.org/show_bug.cgi?id=140687
632
633         Reviewed by Anders Carlsson.
634
635         * bmalloc/Algorithm.h:
636         (bmalloc::isPowerOfTwo): Added a check for 0, since 0 would break a lot
637         of code.
638
639         * bmalloc/BoundaryTag.h:
640         * bmalloc/BoundaryTagInlines.h:
641         (bmalloc::BoundaryTag::mergeLeft):
642         (bmalloc::BoundaryTag::mergeRight):
643         (bmalloc::BoundaryTag::merge):
644         (bmalloc::BoundaryTag::deallocate):
645         (bmalloc::BoundaryTag::split):
646         (bmalloc::BoundaryTag::allocate):
647         (bmalloc::BoundaryTag::mergeLargeLeft): Deleted.
648         (bmalloc::BoundaryTag::mergeLargeRight): Deleted.
649         (bmalloc::BoundaryTag::mergeLarge): Deleted.
650         (bmalloc::BoundaryTag::splitLarge): Deleted. Removed the word "Large"
651         from all these functions, since boundary tags always pertain to large
652         objects, and putting the word "Large" everywhere wasn't helping to
653         explain that.
654
655         * bmalloc/Heap.cpp:
656         (bmalloc::Heap::allocateXLarge):
657         (bmalloc::Heap::findXLarge):
658         (bmalloc::Heap::deallocateXLarge):
659         * bmalloc/Heap.h:
660         * bmalloc/VMHeap.h:
661         (bmalloc::VMHeap::allocateXLarge): Deleted.
662         (bmalloc::VMHeap::findXLarge): Deleted.
663         (bmalloc::VMHeap::deallocateXLarge): Deleted. Moved XLarge allocation
664         from VMHeap to Heap. Since the purpose of the VMHeap is to cache VM
665         ranges, and the VMHeap never caches any XLarge ranges, it doesn't
666         really make sense for the VMHeap to be involved.
667
668 2015-01-16  Geoffrey Garen  <ggaren@apple.com>
669
670         bmalloc: refactored XLarge allocation for better alignment
671         https://bugs.webkit.org/show_bug.cgi?id=140582
672
673         Reviewed by Andreas Kling.
674
675         XLarge objects used to be Large objects with an extra bit of metadata
676         that said "actually, I'm not large -- I'm extra large".
677
678         The metadata header in an XLarge allocation made it impossible for the
679         XLarge object to honor a very large alignment request.
680
681         The solution is to stop using a metadata header for XLarge objects, and
682         instead to store explicit metadata on the side.
683
684         This is a bit less astonishing, which is also nice.
685
686         Finding XLarge metadata is now a linear search. That's probably OK, since
687         it was always so in TCMalloc, and the usual number of XLarge allocations
688         in a process is 0.
689
690         This design makes it possible for the heap to cache XLarge allocations
691         with and/or without physical pages. I haven't actually done that yet
692         because the tradeoffs are subtle, so I don't want to do anything without
693         a motivating test case.
694
695         * bmalloc.xcodeproj/project.pbxproj:
696         * bmalloc/Allocator.cpp:
697         (bmalloc::Allocator::reallocate): Removed the concept of an XLargeChunk,
698         since an XLarge allocation is now just a naked buffer without a header.
699
700         (bmalloc::Allocator::allocateXLarge): Added an explicit qualifier for
701         XLarge alignment, since XLargeChunk won't give this to us implicitly
702         anymore.
703
704         * bmalloc/BoundaryTag.h:
705         (bmalloc::BoundaryTag::setRange):
706         (bmalloc::BoundaryTag::isXLarge): Deleted.
707         (bmalloc::BoundaryTag::setXLarge): Deleted.
708         * bmalloc/BoundaryTagInlines.h:
709         (bmalloc::validate):
710         (bmalloc::BoundaryTag::deallocate): Removed the XLarge hacks from Large allocations.
711
712         * bmalloc/Deallocator.cpp:
713         (bmalloc::Deallocator::deallocateXLarge):
714         (bmalloc::Deallocator::deallocateSlowCase):
715         * bmalloc/Heap.cpp:
716         (bmalloc::Heap::findXLarge):
717         (bmalloc::Heap::allocateXLarge):
718         (bmalloc::Heap::deallocateXLarge):
719         * bmalloc/Heap.h: Updated for interface changes.
720
721         * bmalloc/ObjectType.cpp:
722         (bmalloc::objectType):
723         * bmalloc/ObjectType.h:
724         (bmalloc::isXLarge): We can now tell if a pointer is XLarge just by
725         examining its bit pattern -- just like we do for other kinds of
726         allocations -- which is nice.
727
728         * bmalloc/Sizes.h:
729         * bmalloc/VMHeap.h:
730         (bmalloc::VMHeap::allocateXLarge):
731         (bmalloc::VMHeap::findXLarge):
732         (bmalloc::VMHeap::deallocateXLarge): Keep an explicit vector of metadata
733         for XLarge allocations.
734
735         * bmalloc/XLargeChunk.h: Removed.
736
737 2015-01-16  Geoffrey Garen  <ggaren@apple.com>
738
739         bmalloc: added some infrastructure for aligned allocation
740         https://bugs.webkit.org/show_bug.cgi?id=140572
741
742         Reviewed by Andreas Kling.
743
744         * bmalloc/Algorithm.h:
745         (bmalloc::isPowerOfTwo):
746         (bmalloc::roundUpToMultipleOf):
747         (bmalloc::roundDownToMultipleOf): Refactored some duplicate code to use our
748         isPowerOfTwo helper function.
749
750         * bmalloc/Allocator.cpp:
751         (bmalloc::Allocator::allocate):
752         * bmalloc/Allocator.h: Stubbed out an implementation of aligned allocation.
753         Doesn't do anything yet, but does correctly forward to system malloc
754         when bmalloc is disabled.
755
756         * bmalloc/Cache.cpp:
757         (bmalloc::Cache::allocateSlowCaseNullCache):
758         * bmalloc/Cache.h:
759         (bmalloc::Cache::allocate):
760         * bmalloc/bmalloc.h:
761         (bmalloc::api::memalign):
762         * bmalloc/mbmalloc.cpp: Stubbed out an API for aligned allocation.
763
764 2015-01-13  Geoffrey Garen  <ggaren@apple.com>
765
766         Consider alignment when allocating from a SegregatedFreeList
767         https://bugs.webkit.org/show_bug.cgi?id=140408
768
769         Reviewed by Sam Weinig.
770
771         In preparation for supporting aligned allocation.
772
773         No performance change.
774
775         Since this is just one extra branch in an already expensive function,
776         I decided not to duplicate the function just to avoid the branch in
777         the un-aligned case.
778
779         * bmalloc/SegregatedFreeList.cpp:
780         (bmalloc::SegregatedFreeList::take):
781         * bmalloc/SegregatedFreeList.h:
782
783 2015-01-13  Geoffrey Garen  <ggaren@apple.com>
784
785         Renamed minimum to size in SegregatedFreeList
786         https://bugs.webkit.org/show_bug.cgi?id=140406
787
788         Reviewed by Sam Weinig.
789
790         In preparation for supporting aligned allocation.
791
792         * bmalloc/SegregatedFreeList.cpp:
793         (bmalloc::SegregatedFreeList::takeGreedy):
794         (bmalloc::SegregatedFreeList::take): Every size passed to malloc is
795         really just a minimum. Let's not imply that this value is special.
796
797 2015-01-11  Dan Bernstein  <mitz@apple.com>
798
799         Geoff is organized, but he is not an organization.
800
801         Rubber-stamped by Anders Carlsson.
802
803         * bmalloc.xcodeproj/project.pbxproj: Removed the ORGANIZATIONNAME project attribute.
804
805 2015-01-07  Geoffrey Garen  <ggaren@apple.com>
806
807         Make bmalloc work with ASan
808         https://bugs.webkit.org/show_bug.cgi?id=140194
809
810         Reviewed by Mark Lam.
811
812         * bmalloc/BPlatform.h: Added a way to detect Darwin OSes, since we need
813         an OS-specific API to test for loaded runtime libraries.
814
815         * bmalloc/Environment.cpp:
816         (bmalloc::isASanEnabled):
817         (bmalloc::Environment::computeIsBmallocEnabled): Disabled bmalloc if
818         ASan is enabled, since system malloc has the Asan hooks we need.
819
820         You could check for the ASan compile-time flag instead, but doing this
821         check at runtime prepares bmalloc for a world where it is a dynamic
822         library that might be loaded into projects it did not compile with.
823
824 2015-01-05  Geoffrey Garen  <ggaren@apple.com>
825
826         Fix up bmalloc's PerThread for use on Linux
827         https://bugs.webkit.org/show_bug.cgi?id=139804
828
829         Reviewed by Anders Carlsson.
830
831         The previous implementation was a bit slow.
832
833         * bmalloc/PerThread.h:
834         (bmalloc::PerThreadStorage<Cache>::get):
835         (bmalloc::PerThreadStorage::get):
836         (bmalloc::PerThreadStorage::init): Added a catch-all cross-platform Unix
837         way to do fast per-thread access without taking a lock every time. This
838         probably works on all the platforms we care about, and it matches other
839         techniques we use elsewhere in WebKit.
840
841         (bmalloc::PerThread<T>::getFastCase): Removed the conditional from
842         this class because PerThreadStorage now encapsulates everything that
843         needs to be conditional.
844
845         (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Deleted.
846
847 2014-12-26  Dan Bernstein  <mitz@apple.com>
848
849         <rdar://problem/19348208> REGRESSION (r177027): iOS builds use the wrong toolchain
850         https://bugs.webkit.org/show_bug.cgi?id=139950
851
852         Reviewed by David Kilzer.
853
854         * Configurations/Base.xcconfig: Only define TOOLCHAINS when building for OS X, doing so
855         in a manner that works with Xcode 5.1.1.
856
857 2014-12-15  Geoffrey Garen  <ggaren@apple.com>
858
859         Safari crashes when you set Malloc environment variables
860         https://bugs.webkit.org/show_bug.cgi?id=139656
861
862         Reviewed by Michael Saboff.
863
864         I forgot to cover the realloc() case. Whoops. (OoPS?)
865
866         This time around, I ran the full MallocBench test suite in Malloc=1
867         mode, and it passed.
868
869         * bmalloc/Allocator.cpp:
870         (bmalloc::Allocator::reallocate):
871         * bmalloc/Allocator.h: Pushed realloc() logic down into the allocator.
872         It needs to be down there so that we can do the short-circuiting check
873         for whether bmalloc is enabled first.
874
875         Also added the check.
876
877         * bmalloc/Cache.cpp:
878         (bmalloc::Cache::scavenge):
879         (bmalloc::Cache::Cache):
880         (bmalloc::Cache::reallocateSlowCaseNullCache):
881         * bmalloc/Cache.h:
882         (bmalloc::Cache::deallocator):
883         (bmalloc::Cache::reallocate): Ditto.
884
885         * bmalloc/bmalloc.h:
886         (bmalloc::api::free):
887         (bmalloc::api::realloc): Ditto.
888
889         (bmalloc::api::scavenge): Pushed this down into Cache to match the
890         surrounding functions.
891
892 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
893
894         bmalloc should support system memory analysis tools (part 2)
895         https://bugs.webkit.org/show_bug.cgi?id=139565
896
897         Reviewed by Mark Lam.
898
899         This patch actually queries the environment to see if memory analysis
900         tools have been enabled.
901
902         * bmalloc/Deallocator.cpp:
903         (bmalloc::Deallocator::scavenge): Don't process the object log if
904         we've disabled bmalloc because it will be full of invalid nullptrs.
905
906         * bmalloc/Environment.cpp:
907         (bmalloc::isMallocEnvironmentVariableSet): Test for the list of known
908         Malloc debugging flags. I also added a plain "Malloc" catch-all for
909         when you want to disable bmalloc without enabling any kind of funny
910         business.
911
912         It would be slightly nicer just to iterate the list of environment
913         variables and strstr them, but getenv is the more portable option,
914         and performance here doesn't really matter.
915
916         (bmalloc::isLibgmallocEnabled): Test for the libgmalloc insertion
917         environment variable.
918
919         (bmalloc::Environment::computeIsBmallocEnabled):
920
921 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
922
923         Try to fix the iOS simulator build.
924
925         #include the declaration of malloc / free.
926
927         * bmalloc/Allocator.cpp:
928         * bmalloc/Deallocator.cpp:
929
930 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
931
932         Try to fix the build.
933
934         * bmalloc.xcodeproj/project.pbxproj: Marked a header exported.
935
936 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
937
938         bmalloc should support system memory analysis tools (part 1)
939         https://bugs.webkit.org/show_bug.cgi?id=139559
940
941         Reviewed by Mark Lam.
942
943         This patch adds the hooks to disable bmalloc at runtime if certain
944         environment variables are set, but doesn't actually read from the
945         environment yet.
946
947         No performance change.
948
949         * bmalloc.xcodeproj/project.pbxproj: Added the Environment class, which
950         we'll use to read environment variables and see if memory analysis tools
951         have been enabled.
952
953         * bmalloc/Allocator.cpp:
954         (bmalloc::Allocator::Allocator):
955         (bmalloc::Allocator::allocateSlowCase): Added a hook to disable bmalloc
956         on the allocation path. We cache the setting to make the check fast.
957
958         * bmalloc/Allocator.h: Interface changes.
959
960         * bmalloc/Cache.cpp:
961         (bmalloc::Cache::Cache): Pass a heap pointer through to our allocator
962         and deallocator. This main purpose is to enable them to query the
963         environment for whether bmalloc is enabled; but this is also a slightly
964         cleaner way to guarantee to them that the Heap has been pre-initialized.
965
966         * bmalloc/Deallocator.cpp:
967         (bmalloc::Deallocator::Deallocator): If bmalloc is disable, artificially
968         fill the object log to force us to take the slow path on all deallocations.
969
970         (bmalloc::Deallocator::deallocateSlowCase): Do the disabled check.
971
972         * bmalloc/Deallocator.h: Interface changes.
973
974         * bmalloc/Environment.cpp: Added.
975         (bmalloc::Environment::Environment):
976         (bmalloc::Environment::computeIsBmallocEnabled):
977         * bmalloc/Environment.h: Added.
978         (bmalloc::Environment::isBmallocEnabled): This is the class that will
979         encapsulate looking for environment variables that turn on heap
980         analysis tools.
981
982         * bmalloc/Heap.h:
983         (bmalloc::Heap::environment):
984
985         * bmalloc/Mutex.h:
986         (bmalloc::Mutex::Mutex):
987         * bmalloc/StaticMutex.h: A little refactoring to clarify these comments,
988         since I got super confused about them while writing this patch.
989
990         * bmalloc/VMHeap.cpp: Fixed an #include.
991
992 2014-12-09  David Kilzer  <ddkilzer@apple.com>
993
994         Switch from using PLATFORM_NAME to SDK selectors in ANGLE, bmalloc, gtest, JavaScriptCore, WTF
995         <http://webkit.org/b/139212>
996
997         Reviewed by Joseph Pecoraro.
998
999         * Configurations/Base.xcconfig:
1000         - Only set GCC_ENABLE_OBJC_GC, GCC_MODEL_TUNING and TOOLCHAINS
1001           on OS X.
1002         * Configurations/DebugRelease.xcconfig:
1003         - Only set MACOSX_DEPLOYMENT_TARGET and SDKROOT on OS X.
1004
1005 2014-11-07  Geoffrey Garen  <ggaren@apple.com>
1006
1007         bmalloc uses 8X more virtual memory than necessary
1008         https://bugs.webkit.org/show_bug.cgi?id=138495
1009
1010         Reviewed by Mark Lam.
1011
1012         iOS has a per-process virtual memory cap around 1GB, so there's some
1013         value to not going totally ham with virtual memory.
1014
1015         We currently use about 8X the necessary amount:
1016             - 2X to align our VM allocation
1017             - 4X to reserve small / medium / (2) large chunk VM ranges per superchunk
1018
1019         We can cut that down:
1020             - Return the unaligned portion of our VM allocation (-2X)
1021             - Use all the chunks in a superchunk, instead of allocating one
1022               chunk per superchunk (-4X)
1023
1024         * bmalloc/Algorithm.h:
1025         (bmalloc::roundUpToMultipleOf): Added a non-constant version of this
1026         function so we can call it with getpagesize() at runtime.
1027
1028         * bmalloc/Chunk.h:
1029         * bmalloc/LargeChunk.h:
1030         (bmalloc::LargeChunk::create): Deleted. Instead of each chunk allocating
1031         its own VM, VMHeap allocates the superchunk and all the chunks in it at a time.
1032
1033         * bmalloc/VMAllocate.h:
1034         (bmalloc::vmValidate):
1035         (bmalloc::vmAllocate): ASSERT that mmap succeeds to make crashes clearer
1036         if it does not succeed. Allocate precisely, and give back the extra.
1037
1038         * bmalloc/VMHeap.cpp:
1039         (bmalloc::VMHeap::allocateSuperChunk):
1040         (bmalloc::VMHeap::allocateSmallChunk): Deleted.
1041         (bmalloc::VMHeap::allocateMediumChunk): Deleted.
1042         (bmalloc::VMHeap::allocateLargeChunk): Deleted. Use all the chunks
1043         in a superchunk, instead of just one.
1044
1045         * bmalloc/VMHeap.h:
1046         (bmalloc::VMHeap::allocateSmallPage):
1047         (bmalloc::VMHeap::allocateMediumPage):
1048         (bmalloc::VMHeap::allocateLargeRange):
1049         * bmalloc/XLargeChunk.h:
1050         (bmalloc::XLargeChunk::create): Updated to match changes above.
1051
1052 2014-11-01  David Kilzer  <ddkilzer@apple.com>
1053
1054         JavaScriptCore is missing debug info for bmalloc because libbmalloc.a is stripped
1055         <https://webkit.org/b/138286>
1056         <rdar://problem/18847087>
1057
1058         Reviewed by Dan Bernstein.
1059
1060         * Configurations/bmalloc.xcconfig: Set STRIP_INSTALLED_PRODUCT
1061         to NO for the target that produces libbmalloc.a so that the
1062         debug symbols will be linked into JavaScriptCore and end up in
1063         its dSYM file.
1064
1065 2014-10-30  Dana Burkart  <dburkart@apple.com>
1066
1067         <rdar://problem/18821260> Prepare for the mysterious future
1068
1069         Reviewed by Lucas Forschler.
1070
1071         * Configurations/Base.xcconfig:
1072         * Configurations/DebugRelease.xcconfig:
1073
1074 2014-09-24  Geoffrey Garen  <ggaren@apple.com>
1075
1076         bmalloc: cleaned up fast path vs slow path
1077         https://bugs.webkit.org/show_bug.cgi?id=137081
1078
1079         Reviewed by Sam Weinig.
1080
1081         Might be a 1% speedup on MallocBench. Also cleans up the code a bit.
1082
1083         * bmalloc/Allocator.cpp:
1084         (bmalloc::Allocator::Allocator): Merged the small and medium range
1085         caches, just like the small and medium allocators. Ranges are abstract
1086         objects that don't really care whether they hold small or medium objects,
1087         so they don't need to be segregated.
1088
1089         (bmalloc::Allocator::scavenge): Ditto.
1090
1091         (bmalloc::Allocator::allocateBumpRangeSlowCase):
1092         (bmalloc::Allocator::allocateBumpRange): Same thing here, except that
1093         we do care a tiny bit, because we need to specify small vs medium when
1094         allocating new ranges from the heap, to ensure that the heap allocates
1095         from the right segment of VM.
1096
1097         (bmalloc::Allocator::allocateLarge):
1098         (bmalloc::Allocator::allocateXLarge): NO_INLINE because this was clouding
1099         up the fast path. Large allocation performance is dominated by allocation
1100         logic and initialization, so inlining it doesn't help.
1101
1102         (bmalloc::Allocator::allocateSlowCase): Slow path got a bit cleaner since
1103         it doesn't need to distinguish small vs medium objects.
1104
1105         (bmalloc::Allocator::allocateSmallBumpRange): Deleted.
1106         (bmalloc::Allocator::allocateMediumBumpRange): Deleted.
1107
1108         * bmalloc/Allocator.h:
1109         * bmalloc/BumpRange.h:
1110
1111         * bmalloc/Cache.cpp:
1112         (bmalloc::Cache::allocateSlowCase): Deleted.
1113         (bmalloc::Cache::deallocateSlowCase): Deleted.
1114         * bmalloc/Cache.h:
1115         (bmalloc::Cache::allocate):
1116         (bmalloc::Cache::deallocate):
1117         (bmalloc::Cache::allocateFastCase): Deleted.
1118         (bmalloc::Cache::deallocateFastCase): Deleted. Removed the Cache slow
1119         paths. The downside to this change is that the fast path branches to two
1120         distinct failure cases instead of one. The upside is that the slow path
1121         doesn't need to re-read the segment register, which is not as cheap as a
1122         normal register, and it doesn't need to do an extra level of function 
1123         call. Seems to be worth it.
1124
1125         * bmalloc/Deallocator.h:
1126         * bmalloc/Heap.cpp:
1127         (bmalloc::Heap::refillSmallBumpRangeCache):
1128         (bmalloc::Heap::refillMediumBumpRangeCache):
1129         * bmalloc/Heap.h: Updated for interface changes.
1130
1131         * bmalloc/Sizes.h: The most ranges a cache will hold is the number of
1132         small lines in a page / 2, since any other free lines will coalesce
1133         with their neighbors.
1134
1135 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
1136
1137         Rolled out r173346.
1138
1139             bmalloc should honor the FastMalloc statistics API
1140             https://bugs.webkit.org/show_bug.cgi?id=136592
1141
1142         This didn't really work. Because we allow ranges with and without
1143         physical pages to merge, and we allow double-committing and
1144         double-decommitting, we can't rely on commit actions to track memory
1145         footprint.
1146
1147         * bmalloc/Heap.cpp:
1148         (bmalloc::Heap::size): Deleted.
1149         (bmalloc::Heap::capacity): Deleted.
1150         * bmalloc/Heap.h:
1151         * bmalloc/VMHeap.cpp:
1152         (bmalloc::VMHeap::VMHeap):
1153         (bmalloc::VMHeap::allocateSmallChunk):
1154         (bmalloc::VMHeap::allocateMediumChunk):
1155         (bmalloc::VMHeap::allocateLargeChunk):
1156         * bmalloc/VMHeap.h:
1157         (bmalloc::VMHeap::allocateSmallPage):
1158         (bmalloc::VMHeap::allocateMediumPage):
1159         (bmalloc::VMHeap::allocateLargeRange):
1160         (bmalloc::VMHeap::deallocateSmallPage):
1161         (bmalloc::VMHeap::deallocateMediumPage):
1162         (bmalloc::VMHeap::deallocateLargeRange):
1163         (bmalloc::VMHeap::size): Deleted.
1164         (bmalloc::VMHeap::capacity): Deleted.
1165         * bmalloc/bmalloc.h:
1166         (bmalloc::api::heapSize): Deleted.
1167         (bmalloc::api::heapCapacity): Deleted.
1168
1169 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
1170
1171         bmalloc: Allocation should be more precise
1172         https://bugs.webkit.org/show_bug.cgi?id=136993
1173
1174         Reviewed by Gavin Barraclough.
1175
1176         13% reduction in heap size on the MallocBench *_memory_warning benchmarks.
1177
1178         This patch teaches the allocator to merge adjacent free lines into a
1179         single allocatable range. This allows us to shrink the size of an
1180         individual line without increasing fragmentation or the rate of allocator
1181         slow paths.
1182
1183         We'll only take more slow paths when available memory is sparse, which
1184         is exactly when it's worth it. When available memory is dense, we'll
1185         take fewer slow paths.
1186
1187         * bmalloc.xcodeproj/project.pbxproj:
1188         * bmalloc/Algorithm.h:
1189         (bmalloc::divideRoundingUp):
1190
1191         * bmalloc/Allocator.cpp:
1192         (bmalloc::Allocator::Allocator): Updated for interface changes.
1193
1194         (bmalloc::Allocator::scavenge): Scavenge by object instead of by line.
1195         Now that we merge lines, it's not convenient to scavenge by line.
1196
1197         (bmalloc::Allocator::allocateSmallBumpRange):
1198         (bmalloc::Allocator::allocateMediumBumpRange): Allocate whole ranges
1199         instead of individual lines.
1200
1201         (bmalloc::Allocator::allocateSlowCase):
1202         (bmalloc::Allocator::allocateSmallLine): Deleted.
1203         (bmalloc::Allocator::allocateMediumLine): Deleted.
1204         (bmalloc::Allocator::allocateMedium): Deleted.
1205         * bmalloc/Allocator.h:
1206         (bmalloc::Allocator::allocateFastCase): Folded medium allocations
1207         into the standard fast path with small allocations. Since a BumpAllocator
1208         just allocates out of an arbitrary range, it doesn't need to distinguish
1209         between small and medium lines.
1210
1211         * bmalloc/BumpAllocator.h:
1212         (bmalloc::BumpAllocator::size):
1213         (bmalloc::BumpAllocator::BumpAllocator):
1214         (bmalloc::BumpAllocator::init):
1215         (bmalloc::BumpAllocator::refill):
1216         (bmalloc::BumpAllocator::line): Deleted. No need to track line information
1217         anymore: the heap just gives us a pointer and a pre-computed number of
1218         objects, and we allocate them.
1219
1220         * bmalloc/Deallocator.cpp:
1221         (bmalloc::Deallocator::processObjectLog): Updated for interface changes.
1222
1223         * bmalloc/Heap.cpp:
1224         (bmalloc::Heap::Heap):
1225         (bmalloc::Heap::initializeLineMetadata): Pre-compute precise metadata
1226         detailing where all objects will lie in memory. After we merge two lines,
1227         we might allocate an object that spans from one line to the next. This
1228         metadata details which bits of memory overlap in that way, and how they
1229         overlap.
1230
1231         (bmalloc::Heap::refillSmallBumpRangeCache):
1232         (bmalloc::Heap::refillMediumBumpRangeCache): Scan a whole page at a time,
1233         and merge adjacent free lines into BumpRanges.
1234
1235         (bmalloc::Heap::allocateSmallPage):
1236         (bmalloc::Heap::allocateMediumPage):
1237         (bmalloc::Heap::deallocateSmallLine):
1238         (bmalloc::Heap::deallocateMediumLine): Track pages rather than lines,
1239         since we scan for free memory a page at a time.
1240
1241         (bmalloc::Heap::allocateSmallLineSlowCase): Deleted.
1242         (bmalloc::Heap::allocateMediumLineSlowCase): Deleted. Folded into the
1243         fast path.
1244
1245         * bmalloc/Heap.h:
1246         (bmalloc::Heap::derefSmallLine):
1247         (bmalloc::Heap::derefMediumLine):
1248         (bmalloc::Heap::deallocateSmallLine): Deleted.
1249         (bmalloc::Heap::allocateSmallLine): Deleted.
1250         (bmalloc::Heap::deallocateMediumLine): Deleted.
1251         (bmalloc::Heap::allocateMediumLine): Deleted. Updated for interface changes.
1252
1253         * bmalloc/Line.h:
1254         (bmalloc::Line<Traits>::ref):
1255         (bmalloc::Line<Traits>::deref):
1256         (bmalloc::Line<Traits>::concurrentRef): Deleted. We don't pass a derefCount
1257         anymore, since we only ever deref by 1 now.
1258
1259         * bmalloc/MediumAllocator.h:
1260         (bmalloc::MediumAllocator::isNull): Deleted.
1261         (bmalloc::MediumAllocator::MediumAllocator): Deleted.
1262         (bmalloc::MediumAllocator::line): Deleted.
1263         (bmalloc::MediumAllocator::allocate): Deleted.
1264         (bmalloc::MediumAllocator::derefCount): Deleted.
1265         (bmalloc::MediumAllocator::refill): Deleted.
1266         (bmalloc::MediumAllocator::clear): Deleted. Deleted some code that's
1267         been dead for a while, since it doesn't build anymore with this patch.
1268
1269         * bmalloc/Page.h:
1270         (bmalloc::Page::sizeClass):
1271         (bmalloc::Page::setSizeClass):
1272         (bmalloc::Page::smallSizeClass): Deleted.
1273         (bmalloc::Page::setSmallSizeClass): Deleted. Renamed setSmallSizeClass
1274         to sizeClass, since we use it for medium sizes too.
1275
1276         * bmalloc/Sizes.h:
1277         (bmalloc::Sizes::sizeClass):
1278         (bmalloc::Sizes::objectSize): Shrank line sizes to save memory.
1279
1280         (bmalloc::Sizes::smallSizeClassFor): Deleted.
1281         (bmalloc::Sizes::mediumSizeClassFor): Deleted.
1282
1283         * bmalloc/bmalloc.h:
1284         (bmalloc::api::realloc): Now that we have precise objects sizes, realloc
1285         can be a bit more precise. It also has to be, since we can't guarantee
1286         that an object ends at the end of a line anymore.
1287
1288 2014-09-19  Daniel Bates  <dabates@apple.com>
1289
1290         Always assume internal SDK when building configuration Production
1291         https://bugs.webkit.org/show_bug.cgi?id=136925
1292         <rdar://problem/18362399>
1293
1294         Reviewed by Dan Bernstein.
1295
1296         * Configurations/Base.xcconfig:
1297
1298 2014-09-16  Geoffrey Garen  <ggaren@apple.com>
1299
1300         bmalloc: moved line caches from the deallocator to the allocator
1301         https://bugs.webkit.org/show_bug.cgi?id=136868
1302
1303         Reviewed by Gavin Barraclough.
1304
1305         I did this mostly as a simplification, to make it easier to change the
1306         allocation strategy.
1307
1308         No throughput change on MallocBench. Saves about 50kB.
1309
1310         Since the deallocator needs to lock the heap when freeing lines anyway,
1311         there isn't much benefit to giving the deallocator a local cache of
1312         deallocated lines.
1313
1314         We still give the allocator a local cache of lines because that does
1315         reduce the frequency at which it needs to lock the heap in order to
1316         acquire more lines.
1317
1318         * bmalloc/Allocator.cpp:
1319         (bmalloc::Allocator::scavenge):
1320         (bmalloc::Allocator::allocateSmallLine):
1321         (bmalloc::Allocator::allocateMediumLine):
1322         (bmalloc::Allocator::allocateMedium):
1323         (bmalloc::Allocator::allocateSlowCase):
1324         * bmalloc/Allocator.h:
1325         * bmalloc/Deallocator.cpp:
1326         (bmalloc::Deallocator::Deallocator):
1327         (bmalloc::Deallocator::scavenge):
1328         (bmalloc::Deallocator::processObjectLog):
1329         (bmalloc::Deallocator::deallocateSmallLine): Deleted.
1330         (bmalloc::Deallocator::allocateSmallLine): Deleted.
1331         (bmalloc::Deallocator::deallocateMediumLine): Deleted.
1332         (bmalloc::Deallocator::allocateMediumLine): Deleted.
1333         * bmalloc/Deallocator.h:
1334
1335         * bmalloc/Sizes.h:
1336         * bmalloc/VMAllocate.h: Took the opportunity to make the line cache size
1337         exactly one page in size. That's about what we were shooting for anyway,
1338         and it may make it easier to switch to per-page allocation in future.
1339
1340 2014-09-15  Geoffrey Garen  <ggaren@apple.com>
1341
1342         bmalloc: allocate small and medium objects using the same bump pointer class
1343         https://bugs.webkit.org/show_bug.cgi?id=136843
1344
1345         Reviewed by Gavin Barraclough.
1346
1347         4% speedup on MallocBench.
1348
1349         Now that medium-sized objects have dedicated per-size allocators, they
1350         don't need to use an arbitrary bump pointer allocator. This means that
1351         every allocator knows how many objects it will allocate from the start,
1352         and we don't need a post-processing step to adjust refcounts based on
1353         real allocation count.
1354
1355         * bmalloc.xcodeproj/project.pbxproj: Renamed SmallAllocator to BumpAllocator
1356         since it's used for small and medium objects now.
1357
1358         * bmalloc/Allocator.cpp:
1359         (bmalloc::Allocator::Allocator): Updated to use new interface.
1360         (bmalloc::Allocator::scavenge): To "retire" an allocator, we just need
1361         to make sure that we finish allocating all the objects in it.
1362
1363         (bmalloc::Allocator::allocateMedium):
1364         (bmalloc::Allocator::allocateSlowCase):
1365         (bmalloc::Allocator::retire): Deleted.
1366         (bmalloc::Allocator::processSmallAllocatorLog): Deleted.
1367         (bmalloc::Allocator::processMediumAllocatorLog): Deleted.
1368         * bmalloc/Allocator.h:
1369         (bmalloc::Allocator::allocateFastCase): Removed abstractions and data
1370         used to post-process an allocator based on how many objects it allocated.
1371
1372         * bmalloc/BumpAllocator.h: Copied from Source/bmalloc/bmalloc/SmallAllocator.h.
1373         (bmalloc::BumpAllocator::BumpAllocator):
1374         (bmalloc::BumpAllocator::init):
1375         (bmalloc::BumpAllocator::line):
1376         (bmalloc::BumpAllocator::validate):
1377         (bmalloc::BumpAllocator::allocate):
1378         (bmalloc::BumpAllocator::refill):
1379         (bmalloc::BumpAllocator::clear): Updated these functions to be agnostic
1380         about the kinds of lines they allocate into. In some cases, the line
1381         type must be provided as a template parameter by the caller.
1382
1383         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
1384         (bmalloc::SmallAllocator::line): Deleted.
1385         (bmalloc::SmallAllocator::allocate): Deleted.
1386         (bmalloc::SmallAllocator::objectCount): Deleted.
1387         (bmalloc::SmallAllocator::derefCount): Deleted.
1388         (bmalloc::SmallAllocator::refill): Deleted.
1389         (bmalloc::SmallAllocator::clear): Deleted.
1390
1391         * bmalloc/ObjectType.h:
1392         (bmalloc::isMedium):
1393
1394         * bmalloc/SmallAllocator.h:
1395         (bmalloc::SmallAllocator::isNull): Deleted.
1396         (bmalloc::SmallAllocator::canAllocate): Deleted.
1397         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
1398         (bmalloc::SmallAllocator::line): Deleted.
1399         (bmalloc::SmallAllocator::allocate): Deleted.
1400         (bmalloc::SmallAllocator::objectCount): Deleted.
1401         (bmalloc::SmallAllocator::derefCount): Deleted.
1402         (bmalloc::SmallAllocator::refill): Deleted.
1403         (bmalloc::SmallAllocator::clear): Deleted.
1404
1405 2014-09-12  Geoffrey Garen  <ggaren@apple.com>
1406
1407         Fixed a goof in bmalloc Vector sizing
1408         https://bugs.webkit.org/show_bug.cgi?id=136795
1409
1410         Reviewed by Gavin Barraclough and Sam Weinig.
1411
1412         We want our minimum vector to be page-sized since the OS will give us
1413         a page no matter what -- but we want that many bytes, and not enough
1414         bytes to store that many elements.
1415
1416         * bmalloc/Vector.h: Math is hard.
1417
1418 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
1419
1420         bmalloc should segregate medium-sized objects by line like it does for small-sized objects
1421         https://bugs.webkit.org/show_bug.cgi?id=136693
1422
1423         Reviewed by Gavin Barraclough.
1424
1425         4% reduction in heap size on the MallocBench *_memory_warning benchmarks.
1426
1427         No throughput change.
1428
1429         We keep an array of medium allocators, just like our array of small
1430         allocators.
1431
1432         In future, we can simplify the allocation fast path by merging the small
1433         and medium allocator arrays. For now, this is the simplest change that
1434         gets the win.
1435
1436         * bmalloc/Allocator.cpp:
1437         (bmalloc::Allocator::Allocator):
1438         (bmalloc::Allocator::scavenge):
1439         (bmalloc::Allocator::allocateMedium):
1440         * bmalloc/Allocator.h:
1441         * bmalloc/Sizes.h:
1442         (bmalloc::Sizes::mediumSizeClassFor):
1443
1444 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
1445
1446         Reviewed by Sam Weinig.
1447
1448         Renamed log => retire for clarity.
1449
1450         * bmalloc/Allocator.cpp:
1451         (bmalloc::Allocator::scavenge):
1452         (bmalloc::Allocator::retire):
1453         (bmalloc::Allocator::allocateMedium):
1454         (bmalloc::Allocator::allocateSlowCase):
1455         (bmalloc::Allocator::log): Deleted.
1456         * bmalloc/Allocator.h:
1457
1458 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
1459
1460         bmalloc: eager scavenge leaves behind a bogus allocator
1461         https://bugs.webkit.org/show_bug.cgi?id=136743
1462
1463         Reviewed by Sam Weinig.
1464
1465         Be sure to clear the allocator after logging it in the eager scavenge
1466         case, so that we don't later try to allocate out of the lines that we
1467         have thrown away.
1468
1469         We didn't need to do this previously because scavenge would only happen
1470         at thread exit time, after which no further allocation from the per-thread
1471         cache would take place.
1472
1473         * bmalloc/Allocator.cpp:
1474         (bmalloc::Allocator::scavenge):
1475         * bmalloc/MediumAllocator.h:
1476         (bmalloc::MediumAllocator::clear):
1477         * bmalloc/SmallAllocator.h:
1478         (bmalloc::SmallAllocator::clear):
1479
1480 2014-09-05  Geoffrey Garen  <ggaren@apple.com>
1481
1482         bmalloc should honor the FastMalloc statistics API
1483         https://bugs.webkit.org/show_bug.cgi?id=136592
1484
1485         Reviewed by Gavin Barraclough.
1486
1487         We do this by tracking "size" and "capacity" in the VM heap.
1488
1489         The VM heap's "capacity" is all the VM we ever allocated.
1490
1491         The VM heap's "size" the subset of VM currently held onto by the
1492         VM heap (and therefore not in use by the regular heap).
1493
1494         Somewhat ironically, reducing the process's memory footprint, increases
1495         the size of the VM heap, since the VM heap holds the pages that are
1496         purely virtual and not physical.
1497
1498         * bmalloc/Heap.cpp:
1499         (bmalloc::Heap::size):
1500         (bmalloc::Heap::capacity):
1501         * bmalloc/Heap.h:
1502         * bmalloc/VMHeap.cpp:
1503         (bmalloc::VMHeap::VMHeap):
1504         (bmalloc::VMHeap::allocateSmallChunk):
1505         (bmalloc::VMHeap::allocateMediumChunk):
1506         (bmalloc::VMHeap::allocateLargeChunk):
1507         * bmalloc/VMHeap.h:
1508         (bmalloc::VMHeap::size):
1509         (bmalloc::VMHeap::capacity):
1510         (bmalloc::VMHeap::allocateSmallPage):
1511         (bmalloc::VMHeap::allocateMediumPage):
1512         (bmalloc::VMHeap::allocateLargeRange):
1513         (bmalloc::VMHeap::deallocateSmallPage):
1514         (bmalloc::VMHeap::deallocateMediumPage):
1515         (bmalloc::VMHeap::deallocateLargeRange):
1516         * bmalloc/bmalloc.h:
1517         (bmalloc::api::heapSize):
1518         (bmalloc::api::heapCapacity):
1519
1520 2014-09-02  Geoffrey Garen  <ggaren@apple.com>
1521
1522         bmalloc crashes on the EWS bots (due to bad large object allocation)
1523         https://bugs.webkit.org/show_bug.cgi?id=136469
1524
1525         Reviewed by Andreas Kling.
1526
1527         It's possible to convince bmalloc to perform a bad large object allocation,
1528         through these steps:
1529
1530         (1) Insert object A into freelist F0.
1531
1532         (2) Split, merge and split again A's neighbors such that object B is
1533         inserted into freelist F0, with boundary tag and size equal to object A,
1534         but pointer not completely equal to object A. Put object B at the head of F0.
1535
1536         (3) Allocate some other object from F0, swapping its position in the
1537         freelist with object B, such that object A is now ahead of object B.
1538
1539         --> Now, the next allocation for size A/B will allocate object A, which
1540         has a slightly wrong idea about where the object actually begins.
1541         Immediately, you'll corrupt a little memory, and over time, you'll also
1542         corrupt boundary tag metadata.
1543
1544         The solution is to store the begin pointer in the boundary tag. Luckily,
1545         this doesn't make the tag any bigger, and it's not a noticeable slowdown
1546         on MallocBench.
1547
1548         * bmalloc/Algorithm.h:
1549         (bmalloc::rightShift):
1550         * bmalloc/BeginTag.h:
1551         (bmalloc::BeginTag::isInFreeList): This is the bug fix. Make sure to
1552         validate the start pointer when popping off the free list. Through a
1553         very uncommon set of steps, it is possible to have an item in the free
1554         list that is valid by all accounts except for its start pointer.
1555
1556         * bmalloc/BoundaryTag.h:
1557         (bmalloc::BoundaryTag::compactBegin):
1558         (bmalloc::BoundaryTag::setRange):
1559         (bmalloc::BoundaryTag::setSize): Deleted. Record a compact version of the
1560         start pointer. We don't need the whole pointer -- just the offset, in
1561         largeAlignment increments, into the relevant boundary tag bucket.
1562
1563         * bmalloc/BoundaryTagInlines.h:
1564         (bmalloc::validateNext):
1565         (bmalloc::BoundaryTag::init):
1566         (bmalloc::BoundaryTag::mergeLarge):
1567         (bmalloc::BoundaryTag::splitLarge):
1568         * bmalloc/SegregatedFreeList.cpp:
1569         (bmalloc::SegregatedFreeList::insert):
1570         (bmalloc::SegregatedFreeList::takeGreedy):
1571         (bmalloc::SegregatedFreeList::take): Provide the whole range instead of
1572         the size when establishing a boundary tag, as required by the new
1573         interface.
1574
1575         * bmalloc/Sizes.h:
1576
1577 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
1578
1579         Fixed a bmalloc crash seen on the EWS bot
1580         https://bugs.webkit.org/show_bug.cgi?id=135955
1581
1582         Reviewed by Andreas Kling.
1583
1584         * bmalloc/Syscall.h: Some CG APIs vm_copy their input buffers. If the
1585         input buffer is a malloc region, that region will get marked Copy-On-Write
1586         by the kernel. Calls to madvise() for COW regions fail and return EINVAL
1587         on older OS X's. In 10.10, they still fail, but they do not return
1588         EINVAL.
1589
1590         So, we can only ASSERT that our syscalls succeed starting with 10.10.
1591
1592 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
1593
1594         Fixed the bmalloc build
1595         https://bugs.webkit.org/show_bug.cgi?id=135953
1596
1597         Reviewed by Andreas Kling.
1598
1599         * bmalloc.xcodeproj/project.pbxproj: Marked a few headers as private.
1600         These headers are used, so they must be available outside the project.
1601
1602 2014-08-13  Daniel Bates  <dabates@apple.com>
1603
1604         Attempt to fix the build following <http://trac.webkit.org/changeset/172576>
1605         (https://bugs.webkit.org/show_bug.cgi?id=135895)
1606
1607         Substitute PerThreadStorage<T>::initSharedKeyIfNeeded() for initSharedKeyIfNeeded() in
1608         implementation of PerThread<T>::getFastCase().
1609
1610         * bmalloc/PerThread.h:
1611         (bmalloc::PerThread<T>::getFastCase):
1612
1613 2014-08-13  Daniel Bates  <dabates@apple.com>
1614
1615         Make bmalloc::PerThread work without C++ thread local storage
1616         https://bugs.webkit.org/show_bug.cgi?id=135895
1617
1618         Reviewed by Geoffrey Garen.
1619
1620         Implement support for building bmalloc without C++ thread local storage.
1621
1622         * bmalloc/BPlatform.h: Remove macro define BPLATFORM_IOS_SIMULATOR. Added macro function
1623         BCOMPILER_SUPPORTS() and macro define BCOMPILER_SUPPORTS_CXX_THREAD_LOCAL that can be used
1624         to determine whether the compiler supports C++ thread local storage.
1625         * bmalloc/PerThread.h:
1626         (bmalloc::PerThreadStorage::get): Modified to call pthread_getspecific() when building
1627         without C++ thread local storage.
1628         (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Added.
1629         (bmalloc::PerThreadStorage::init): Moved logic to initialize shared Pthread key from here to
1630         PerThreadStorage::initSharedKeyIfNeeded().
1631         (bmalloc::PerThread<T>::getFastCase): Modified to call PerThreadStorage::initSharedKeyIfNeeded()
1632         before querying PerThreadStorage::get() when building without C++ thread local storage so as to
1633         ensure that the shared key has been initialized.
1634         (_pthread_setspecific_direct): Deleted.
1635         (_pthread_getspecific_direct): Deleted.
1636
1637 2014-08-13  Daniel Bates  <dabates@apple.com>
1638
1639         [iOS] Make JavaScriptCore and bmalloc build with the public SDK
1640         https://bugs.webkit.org/show_bug.cgi?id=135848
1641
1642         Reviewed by Geoffrey Garen.
1643
1644         * bmalloc/BPlatform.h: Added macro BPLATFORM_IOS_SIMULATOR, which evaluates to true
1645         when building for the iOS Simulator.
1646         * bmalloc/PerThread.h: Use pthread_machdep.h code path when building for iOS Simulator
1647         using the public SDK.
1648         (_pthread_setspecific_direct): Added; only defined when building for the iOS Simulator
1649         using the public SDK.
1650         (_pthread_getspecific_direct): Added; only defined when building for the iOS Simulator
1651         using the public SDK.
1652
1653 2014-08-12  Daniel Bates  <dabates@apple.com>
1654
1655         BPLATFORM(IOS) always evaluates to false
1656         https://bugs.webkit.org/show_bug.cgi?id=135843
1657
1658         Reviewed by Geoffrey Garen.
1659
1660         Fix typo in definition of BPLATFORM() and include system header TargetConditionals.h
1661         (when building on an Apple platform) so that BPLATFORM(X) evaluates to true when
1662         building for platform X. In particular, so that BPLATFORM(IOS) evaluates to true when
1663         building for iOS.
1664
1665         As a side effect of this change, the change made in <http://trac.webkit.org/changeset/167289>
1666         will be honored and iOS will assume a VM page size of 16kB (again) instead of 4kB.
1667
1668         * bmalloc/BPlatform.h:
1669
1670 2014-08-11  Andy Estes  <aestes@apple.com>
1671
1672         [iOS] Get rid of iOS.xcconfig
1673         https://bugs.webkit.org/show_bug.cgi?id=135809
1674
1675         Reviewed by Joseph Pecoraro.
1676
1677         All iOS.xcconfig did was include AspenFamily.xcconfig, so there's no need for the indirection.
1678
1679         * Configurations/Base.xcconfig:
1680         * Configurations/iOS.xcconfig: Removed.
1681         * bmalloc.xcodeproj/project.pbxproj:
1682
1683 2014-05-01  Dan Bernstein  <mitz@apple.com>
1684
1685         Fixed production builds for the iOS Simulator.
1686         <rdar://problem/16792221>
1687
1688         * Configurations/bmalloc.xcconfig: Include INSTALL_PATH_PREFIX in
1689         PRIVATE_HEADERS_FOLDER_PATH when installing.
1690
1691 2014-04-20  Geoffrey Garen  <ggaren@apple.com>
1692
1693         bmalloc: Segregate pages by objects size
1694         https://bugs.webkit.org/show_bug.cgi?id=131909
1695
1696         Reviewed by Andreas Kling.
1697
1698         2% reduction in memory-at-end on the Membuster memory_warning benchmarks.
1699
1700         * bmalloc/Allocator.cpp:
1701         (bmalloc::Allocator::allocateSlowCase):
1702         * bmalloc/Allocator.h:
1703         (bmalloc::Allocator::allocateFastCase):
1704         (bmalloc::Allocator::smallAllocatorFor): Use the new shared helper
1705         function for size class calculation.
1706
1707         * bmalloc/Deallocator.cpp:
1708         (bmalloc::Deallocator::Deallocator):
1709         (bmalloc::Deallocator::scavenge):
1710         (bmalloc::Deallocator::deallocateSmallLine):
1711         (bmalloc::Deallocator::allocateSmallLine):
1712         * bmalloc/Deallocator.h: Keep a cache for every size class, since the
1713         cache can't be shared anymore.
1714
1715         * bmalloc/Heap.cpp:
1716         (bmalloc::Heap::allocateSmallLineSlowCase):
1717         * bmalloc/Heap.h:
1718         (bmalloc::Heap::deallocateSmallLine): Ditto.
1719
1720         (bmalloc::Heap::allocateSmallLine): Check size class in addition to
1721         page refcount when allocating a line because we might have deallocated
1722         the page and the recycled it for another size class.
1723
1724         (bmalloc::Heap::deallocateMediumLine):
1725         (bmalloc::Heap::allocateMediumLine):
1726         * bmalloc/Line.h:
1727         (bmalloc::Line::refCount):
1728         * bmalloc/Page.h:
1729         (bmalloc::Page::refCount):
1730         (bmalloc::Page::smallSizeClass):
1731         (bmalloc::Page::setSmallSizeClass):
1732         (bmalloc::Page<Traits>::refCount): Deleted.
1733         * bmalloc/Sizes.h:
1734         (bmalloc::Sizes::smallSizeClassFor): New shared API for computing
1735         an index into an array from a size.
1736
1737 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1738
1739         bmalloc: Improved alignment in LargeChunk
1740         https://bugs.webkit.org/show_bug.cgi?id=131895
1741
1742         Reviewed by Andreas Kling.
1743
1744         * bmalloc/Chunk.h:
1745         * bmalloc/LargeChunk.h: Align to vmPageSize just like Chunk does.
1746         Technically, the previous alignment was harmless, but I would prefer,
1747         dear reader, not to have to explain the interlocking set of
1748         circumstances that made it so.
1749
1750 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1751
1752         Rolled out r167502 because it caused a crash on the facebook benchmark.
1753
1754         Unreviewed.
1755
1756             bmalloc: Added an XSmall line size
1757             https://bugs.webkit.org/show_bug.cgi?id=131851
1758
1759             Reviewed by Sam Weinig.
1760
1761 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1762
1763         bmalloc: Mutex should be harder to use wrong
1764         https://bugs.webkit.org/show_bug.cgi?id=131879
1765
1766         Reviewed by Andreas Kling.
1767
1768         Mutex now has a proper constructor, so you can't deadlock by forgetting
1769         to initialize it.
1770
1771         * bmalloc.xcodeproj/project.pbxproj:
1772         * bmalloc/Allocator.cpp:
1773         (bmalloc::Allocator::processXSmallAllocatorLog):
1774         (bmalloc::Allocator::processSmallAllocatorLog):
1775         (bmalloc::Allocator::processMediumAllocatorLog):
1776         (bmalloc::Allocator::allocateLarge):
1777         (bmalloc::Allocator::allocateXLarge): Global replace Mutex => StaticMutex,
1778         since the Heap mutex is a static.
1779
1780         * bmalloc/AsyncTask.h:
1781         (bmalloc::Function>::AsyncTask): Use Mutex, since we're not static. No
1782         need for explicit initialization anymore.
1783
1784         * bmalloc/Deallocator.cpp:
1785         (bmalloc::Deallocator::scavenge):
1786         (bmalloc::Deallocator::deallocateLarge):
1787         (bmalloc::Deallocator::deallocateXLarge):
1788         (bmalloc::Deallocator::processObjectLog):
1789         (bmalloc::Deallocator::deallocateSmallLine):
1790         (bmalloc::Deallocator::deallocateXSmallLine):
1791         (bmalloc::Deallocator::allocateSmallLine):
1792         (bmalloc::Deallocator::allocateXSmallLine):
1793         (bmalloc::Deallocator::deallocateMediumLine):
1794         (bmalloc::Deallocator::allocateMediumLine):
1795         * bmalloc/Deallocator.h:
1796         * bmalloc/Heap.cpp:
1797         (bmalloc::sleep):
1798         (bmalloc::Heap::Heap):
1799         (bmalloc::Heap::concurrentScavenge):
1800         (bmalloc::Heap::scavenge):
1801         (bmalloc::Heap::scavengeSmallPages):
1802         (bmalloc::Heap::scavengeXSmallPages):
1803         (bmalloc::Heap::scavengeMediumPages):
1804         (bmalloc::Heap::scavengeLargeRanges):
1805         (bmalloc::Heap::allocateXSmallLineSlowCase):
1806         (bmalloc::Heap::allocateSmallLineSlowCase):
1807         (bmalloc::Heap::allocateMediumLineSlowCase):
1808         (bmalloc::Heap::allocateXLarge):
1809         (bmalloc::Heap::deallocateXLarge):
1810         (bmalloc::Heap::allocateLarge):
1811         (bmalloc::Heap::deallocateLarge):
1812         * bmalloc/Heap.h:
1813         (bmalloc::Heap::deallocateXSmallLine):
1814         (bmalloc::Heap::allocateXSmallLine):
1815         (bmalloc::Heap::deallocateSmallLine):
1816         (bmalloc::Heap::allocateSmallLine):
1817         (bmalloc::Heap::deallocateMediumLine):
1818         (bmalloc::Heap::allocateMediumLine):
1819         * bmalloc/Line.h:
1820         (bmalloc::Line<Traits>::deref):
1821         * bmalloc/Mutex.cpp: Removed.
1822         * bmalloc/Mutex.h:
1823         (bmalloc::Mutex::Mutex):
1824         (bmalloc::Mutex::init): Deleted.
1825         (bmalloc::Mutex::try_lock): Deleted.
1826         (bmalloc::Mutex::lock): Deleted.
1827         (bmalloc::Mutex::unlock): Deleted.
1828         * bmalloc/Page.h:
1829         (bmalloc::Page<Traits>::ref):
1830         (bmalloc::Page<Traits>::deref):
1831         (bmalloc::Page<Traits>::refCount):
1832         * bmalloc/PerProcess.h:
1833         (bmalloc::PerProcess::mutex):
1834         (bmalloc::PerProcess<T>::getSlowCase):
1835         * bmalloc/StaticMutex.cpp: Added.
1836         (bmalloc::StaticMutex::lockSlowCase):
1837         * bmalloc/StaticMutex.h: Added.
1838         (bmalloc::StaticMutex::init):
1839         (bmalloc::StaticMutex::try_lock):
1840         (bmalloc::StaticMutex::lock):
1841         (bmalloc::StaticMutex::unlock):
1842         * bmalloc/VMHeap.h:
1843         (bmalloc::VMHeap::deallocateXSmallPage):
1844         (bmalloc::VMHeap::deallocateSmallPage):
1845         (bmalloc::VMHeap::deallocateMediumPage):
1846         (bmalloc::VMHeap::deallocateLargeRange):
1847         * bmalloc/bmalloc.h:
1848         (bmalloc::api::scavenge): Global replace Mutex => StaticMutex,
1849         since the Heap mutex is a static.
1850
1851 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
1852
1853         bmalloc: AsyncTask should use Mutex instead of std::mutex
1854         https://bugs.webkit.org/show_bug.cgi?id=131865
1855
1856         Reviewed by Gavin Barraclough.
1857
1858         std::mutex is so slow that it makes parallelizing simple tasks through
1859         AsyncTask a net regression. Mutex fixes this.
1860
1861         * bmalloc/AsyncTask.h:
1862         (bmalloc::Function>::AsyncTask):
1863         (bmalloc::Function>::join):
1864         (bmalloc::Function>::runSlowCase):
1865         (bmalloc::Function>::entryPoint):
1866         * bmalloc/Mutex.h:
1867         (bmalloc::Mutex::init):
1868
1869 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
1870
1871         bmalloc: Added an XSmall line size
1872         https://bugs.webkit.org/show_bug.cgi?id=131851
1873
1874         Reviewed by Sam Weinig.
1875
1876         Reduces malloc footprint on Membuster recordings by 10%.
1877
1878         This is a throughput regression, but we're still way ahead of TCMalloc.
1879         I have some ideas for how to recover the regression -- but I wanted to
1880         get this win in first.
1881
1882         Full set of benchmark results:
1883
1884                 bmalloc> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks --measure-heap nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1885                                                                                                 
1886                                                                nopatch                      patch                                Δ
1887                 Peak Memory:
1888                     reddit_memory_warning                      7,896kB                    7,532kB                  ^ 1.05x smaller
1889                     flickr_memory_warning                     12,968kB                   12,324kB                  ^ 1.05x smaller
1890                     theverge_memory_warning                   16,672kB                   15,200kB                   ^ 1.1x smaller
1891
1892                     <geometric mean>                          11,952kB                   11,216kB                  ^ 1.07x smaller
1893                     <arithmetic mean>                         12,512kB                   11,685kB                  ^ 1.07x smaller
1894                     <harmonic mean>                           11,375kB                   10,726kB                  ^ 1.06x smaller
1895
1896                 Memory at End:
1897                     reddit_memory_warning                      7,320kB                    6,856kB                  ^ 1.07x smaller
1898                     flickr_memory_warning                     10,848kB                    9,692kB                  ^ 1.12x smaller
1899                     theverge_memory_warning                   16,380kB                   14,872kB                   ^ 1.1x smaller
1900
1901                     <geometric mean>                          10,916kB                    9,961kB                   ^ 1.1x smaller
1902                     <arithmetic mean>                         11,516kB                   10,473kB                   ^ 1.1x smaller
1903                     <harmonic mean>                           10,350kB                    9,485kB                  ^ 1.09x smaller
1904
1905                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1906                                                                                                 
1907                                                    nopatch                patch                         Δ
1908                 Execution Time:
1909                     churn                            127ms                151ms            ! 1.19x slower
1910                     list_allocate                    130ms                164ms            ! 1.26x slower
1911                     tree_allocate                    109ms                127ms            ! 1.17x slower
1912                     tree_churn                       115ms                120ms            ! 1.04x slower
1913                     facebook                         240ms                259ms            ! 1.08x slower
1914                     fragment                          91ms                131ms            ! 1.44x slower
1915                     fragment_iterate                 105ms                106ms            ! 1.01x slower
1916                     message_one                      260ms                259ms             ^ 1.0x faster
1917                     message_many                     149ms                154ms            ! 1.03x slower
1918                     medium                           194ms                248ms            ! 1.28x slower
1919                     big                              157ms                160ms            ! 1.02x slower
1920
1921                     <geometric mean>                 144ms                163ms            ! 1.13x slower
1922                     <arithmetic mean>                152ms                171ms            ! 1.12x slower
1923                     <harmonic mean>                  137ms                156ms            ! 1.14x slower
1924
1925                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1926                                                                                                 
1927                                                                        nopatch                          patch                                     Δ
1928                 Execution Time:
1929                     churn                                                126ms                          148ms                        ! 1.17x slower
1930                     churn --parallel                                      62ms                           76ms                        ! 1.23x slower
1931                     list_allocate                                        130ms                          164ms                        ! 1.26x slower
1932                     list_allocate --parallel                             120ms                          175ms                        ! 1.46x slower
1933                     tree_allocate                                        111ms                          127ms                        ! 1.14x slower
1934                     tree_allocate --parallel                              95ms                          135ms                        ! 1.42x slower
1935                     tree_churn                                           115ms                          124ms                        ! 1.08x slower
1936                     tree_churn --parallel                                107ms                          126ms                        ! 1.18x slower
1937                     facebook                                             240ms                          276ms                        ! 1.15x slower
1938                     facebook --parallel                                  802ms                        1,088ms                        ! 1.36x slower
1939                     fragment                                              92ms                          130ms                        ! 1.41x slower
1940                     fragment --parallel                                   66ms                          124ms                        ! 1.88x slower
1941                     fragment_iterate                                     109ms                          127ms                        ! 1.17x slower
1942                     fragment_iterate --parallel                           55ms                           64ms                        ! 1.16x slower
1943                     message_one                                          260ms                          260ms                                      
1944                     message_many                                         170ms                          238ms                         ! 1.4x slower
1945                     medium                                               185ms                          250ms                        ! 1.35x slower
1946                     medium --parallel                                    210ms                          334ms                        ! 1.59x slower
1947                     big                                                  150ms                          169ms                        ! 1.13x slower
1948                     big --parallel                                       138ms                          144ms                        ! 1.04x slower
1949
1950                     <geometric mean>                                     135ms                          170ms                        ! 1.26x slower
1951                     <arithmetic mean>                                    167ms                          214ms                        ! 1.28x slower
1952                     <harmonic mean>                                      117ms                          148ms                        ! 1.26x slower
1953
1954                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
1955
1956                                                                     TC                      patch                                Δ
1957                 Peak Memory:
1958                     reddit_memory_warning                     13,836kB                   13,436kB                  ^ 1.03x smaller
1959                     flickr_memory_warning                     24,868kB                   25,188kB                   ! 1.01x bigger
1960                     theverge_memory_warning                   24,504kB                   26,636kB                   ! 1.09x bigger
1961
1962                     <geometric mean>                          20,353kB                   20,812kB                   ! 1.02x bigger
1963                     <arithmetic mean>                         21,069kB                   21,753kB                   ! 1.03x bigger
1964                     <harmonic mean>                           19,570kB                   19,780kB                   ! 1.01x bigger
1965
1966                 Memory at End:
1967                     reddit_memory_warning                      8,656kB                   10,016kB                   ! 1.16x bigger
1968                     flickr_memory_warning                     11,844kB                   13,784kB                   ! 1.16x bigger
1969                     theverge_memory_warning                   18,516kB                   22,748kB                   ! 1.23x bigger
1970
1971                     <geometric mean>                          12,382kB                   14,644kB                   ! 1.18x bigger
1972                     <arithmetic mean>                         13,005kB                   15,516kB                   ! 1.19x bigger
1973                     <harmonic mean>                           11,813kB                   13,867kB                   ! 1.17x bigger
1974
1975                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
1976                                                                                                 
1977                                                         TC                patch                         Δ
1978                 Execution Time:
1979                     churn                            416ms                148ms            ^ 2.81x faster
1980                     list_allocate                    463ms                164ms            ^ 2.82x faster
1981                     tree_allocate                    292ms                127ms             ^ 2.3x faster
1982                     tree_churn                       157ms                120ms            ^ 1.31x faster
1983                     facebook                         327ms                276ms            ^ 1.18x faster
1984                     fragment                         335ms                129ms             ^ 2.6x faster
1985                     fragment_iterate                 344ms                108ms            ^ 3.19x faster
1986                     message_one                      386ms                258ms             ^ 1.5x faster
1987                     message_many                     410ms                154ms            ^ 2.66x faster
1988                     medium                           391ms                245ms             ^ 1.6x faster
1989                     big                              261ms                167ms            ^ 1.56x faster
1990
1991                     <geometric mean>                 332ms                164ms            ^ 2.02x faster
1992                     <arithmetic mean>                344ms                172ms            ^ 1.99x faster
1993                     <harmonic mean>                  317ms                157ms            ^ 2.02x faster
1994
1995         * bmalloc.xcodeproj/project.pbxproj:
1996         * bmalloc/Allocator.cpp:
1997         (bmalloc::Allocator::Allocator): Don't assume that each allocator's
1998         index corresponds with its size. Instead, use the size selection function
1999         explicitly. Now that we have XSmall, some small allocator entries are
2000         unused.
2001
2002         (bmalloc::Allocator::scavenge):
2003         (bmalloc::Allocator::log):
2004         (bmalloc::Allocator::processXSmallAllocatorLog):
2005         (bmalloc::Allocator::allocateSlowCase):
2006         * bmalloc/Allocator.h:
2007         (bmalloc::Allocator::xSmallAllocatorFor):
2008         (bmalloc::Allocator::allocateFastCase):
2009         * bmalloc/Chunk.h:
2010         * bmalloc/Deallocator.cpp:
2011         (bmalloc::Deallocator::scavenge):
2012         (bmalloc::Deallocator::processObjectLog):
2013         (bmalloc::Deallocator::deallocateSlowCase):
2014         (bmalloc::Deallocator::deallocateXSmallLine):
2015         (bmalloc::Deallocator::allocateXSmallLine):
2016         * bmalloc/Deallocator.h:
2017         (bmalloc::Deallocator::deallocateFastCase):
2018         * bmalloc/Heap.cpp:
2019         (bmalloc::Heap::scavenge):
2020         (bmalloc::Heap::scavengeXSmallPages):
2021         (bmalloc::Heap::allocateXSmallLineSlowCase):
2022         * bmalloc/Heap.h:
2023         (bmalloc::Heap::deallocateXSmallLine):
2024         (bmalloc::Heap::allocateXSmallLine):
2025         * bmalloc/LargeChunk.h:
2026         (bmalloc::LargeChunk::get):
2027         (bmalloc::LargeChunk::endTag):
2028         * bmalloc/Line.h:
2029         * bmalloc/MediumAllocator.h:
2030         (bmalloc::MediumAllocator::allocate):
2031         (bmalloc::MediumAllocator::refill):
2032         * bmalloc/ObjectType.cpp:
2033         (bmalloc::objectType):
2034         * bmalloc/ObjectType.h:
2035         (bmalloc::isXSmall):
2036         (bmalloc::isSmall):
2037         (bmalloc::isMedium):
2038         (bmalloc::isLarge):
2039         (bmalloc::isSmallOrMedium): Deleted.
2040         * bmalloc/SegregatedFreeList.h: I boiler-plate copied existing code for
2041         handling small objects. There's probably a reasonable way to share this
2042         code in the future -- I'll look into that once it's stopped changing.
2043
2044         * bmalloc/Sizes.h: Tweaked size classes to make Membuster happy. This
2045         is the main reason things got slower.
2046
2047         * bmalloc/SmallAllocator.h:
2048         (bmalloc::SmallAllocator::allocate):
2049         * bmalloc/SmallTraits.h:
2050         * bmalloc/VMHeap.cpp:
2051         (bmalloc::VMHeap::allocateXSmallChunk):
2052         * bmalloc/VMHeap.h:
2053         (bmalloc::VMHeap::allocateXSmallPage):
2054         (bmalloc::VMHeap::deallocateXSmallPage):
2055         * bmalloc/XSmallAllocator.h: Added.
2056         (bmalloc::XSmallAllocator::isNull):
2057         (bmalloc::XSmallAllocator::canAllocate):
2058         (bmalloc::XSmallAllocator::XSmallAllocator):
2059         (bmalloc::XSmallAllocator::line):
2060         (bmalloc::XSmallAllocator::allocate):
2061         (bmalloc::XSmallAllocator::objectCount):
2062         (bmalloc::XSmallAllocator::derefCount):
2063         (bmalloc::XSmallAllocator::refill):
2064         * bmalloc/XSmallChunk.h: Added.
2065         * bmalloc/XSmallLine.h: Added.
2066         * bmalloc/XSmallPage.h: Added.
2067         * bmalloc/XSmallTraits.h: Added.
2068         * bmalloc/bmalloc.h:
2069         (bmalloc::api::realloc): Boiler-plate copy, as above.
2070
2071 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
2072
2073         MallocBench should scavenge explicitly instead of waiting
2074         https://bugs.webkit.org/show_bug.cgi?id=131661
2075
2076         Reviewed by Andreas Kling.
2077
2078         Added explicit scavenge support to bmalloc. This isn't a memory win,
2079         since bmalloc's per-thread cache is so small. But it makes testing
2080         simpler.
2081
2082         * bmalloc/Allocator.cpp:
2083         (bmalloc::Allocator::~Allocator):
2084         (bmalloc::Allocator::scavenge):
2085         * bmalloc/Allocator.h:
2086         * bmalloc/Cache.cpp:
2087         (bmalloc::Cache::operator new):
2088         (bmalloc::Cache::operator delete):
2089         (bmalloc::Cache::Cache):
2090         (bmalloc::Cache::scavenge):
2091         * bmalloc/Cache.h:
2092         * bmalloc/Deallocator.cpp:
2093         (bmalloc::Deallocator::~Deallocator):
2094         (bmalloc::Deallocator::scavenge):
2095         * bmalloc/Deallocator.h: Factored existing scavenging code into helper
2096         functions, for reuse.
2097
2098         * bmalloc/Heap.cpp:
2099         (bmalloc::sleep):
2100         (bmalloc::Heap::concurrentScavenge):
2101         (bmalloc::Heap::scavenge):
2102         (bmalloc::Heap::scavengeSmallPages):
2103         (bmalloc::Heap::scavengeMediumPages):
2104         (bmalloc::Heap::scavengeLargeRanges):
2105         * bmalloc/Heap.h: Made scavenge sleep duration a parameter. Forced
2106         scavenging -- in response to a benchmark or a low memory warning --
2107         wants to complete as soon as possible, so its sleep duration is 0.
2108
2109         * bmalloc/bmalloc.h:
2110         (bmalloc::api::scavenge):
2111         * bmalloc/mbmalloc.cpp: Exported the scavenge API for MallocBench's use.
2112
2113 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
2114
2115         Use 4kB pages on Mac
2116         https://bugs.webkit.org/show_bug.cgi?id=131658
2117
2118         Reviewed by Sam Weinig.
2119
2120         This reduces memory use a lot on Membuster:
2121
2122                                                                   base                      patch                                Δ
2123                 Execution Time:
2124                     reddit_memory_warning                         18ms                       17ms                   ^ 1.06x faster
2125                     flickr_memory_warning                         34ms                       36ms                   ! 1.06x slower
2126                     theverge_memory_warning                       39ms                       41ms                   ! 1.05x slower
2127
2128                     <geometric mean>                              29ms                       29ms                   ! 1.02x slower
2129                     <arithmetic mean>                             30ms                       31ms                   ! 1.03x slower
2130                     <harmonic mean>                               27ms                       27ms                    ^ 1.0x faster
2131
2132                 Peak Memory:
2133                     reddit_memory_warning                     16,412kB                   16,436kB                    ! 1.0x bigger
2134                     flickr_memory_warning                     30,120kB                   30,184kB                    ! 1.0x bigger
2135                     theverge_memory_warning                   33,408kB                   33,420kB                    ! 1.0x bigger
2136
2137                     <geometric mean>                          25,466kB                   25,499kB                    ! 1.0x bigger
2138                     <arithmetic mean>                         26,647kB                   26,680kB                    ! 1.0x bigger
2139                     <harmonic mean>                           24,181kB                   24,214kB                    ! 1.0x bigger
2140
2141                 Memory at End:
2142                     reddit_memory_warning                      2,404kB                    1,920kB                  ^ 1.25x smaller
2143                     flickr_memory_warning                      3,764kB                    3,072kB                  ^ 1.23x smaller
2144                     theverge_memory_warning                    3,648kB                    3,132kB                  ^ 1.16x smaller
2145
2146                     <geometric mean>                           3,208kB                    2,644kB                  ^ 1.21x smaller
2147                     <arithmetic mean>                          3,272kB                    2,708kB                  ^ 1.21x smaller
2148                     <harmonic mean>                            3,139kB                    2,574kB                  ^ 1.22x smaller
2149
2150
2151         * bmalloc.xcodeproj/project.pbxproj:
2152         * bmalloc/BPlatform.h: Added.
2153         * bmalloc/VMAllocate.h: Only use 16kB pages on iOS because the page size
2154         is 4kB on Mac.
2155
2156 2014-04-14  Alexey Proskuryakov  <ap@apple.com>
2157
2158         Fixed svn:ignore on bmalloc.xcodeproj, it had erroneous leading spaces.
2159
2160         * bmalloc.xcodeproj: Modified property svn:ignore.
2161
2162 2014-04-13  Geoffrey Garen  <ggaren@apple.com>
2163
2164         Fixed some mbmalloc exports
2165         https://bugs.webkit.org/show_bug.cgi?id=131599
2166
2167         Reviewed by Ryosuke Niwa.
2168
2169         * bmalloc.xcodeproj/project.pbxproj: Made some headers a private part
2170         of the project, so we can call them from API.
2171
2172         * bmalloc/mbmalloc.cpp: Marked the mbmalloc functions with default
2173         visibility, so they show up as exported in the .dylib.
2174
2175 2014-04-09  Geoffrey Garen  <ggaren@apple.com>
2176
2177         Put bmalloc headers in the right place
2178         https://bugs.webkit.org/show_bug.cgi?id=131464
2179
2180         Reviewed by Mark Rowe.
2181
2182         * Configurations/bmalloc.xcconfig: Set PRIVATE_HEADERS_FOLDER_PATH to
2183         specify that we don't just want to dump all of our generically-named
2184         headers into /usr/local/include.
2185
2186 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
2187
2188         Made bmalloc more #include friendly
2189         https://bugs.webkit.org/show_bug.cgi?id=131386
2190
2191         Reviewed by Andreas Kling.
2192
2193         Marked a bunch of headers private so they can be used from client code
2194         that #includes bmalloc.h.
2195
2196         Renamed ASSERT macros to BASSERT. This matches their header, which already
2197         had to be renamed, and fixes conflicts with WTF's ASSERT macros.
2198
2199         * bmalloc.xcodeproj/project.pbxproj:
2200         * bmalloc/Allocator.cpp:
2201         (bmalloc::Allocator::allocateSlowCase):
2202         * bmalloc/AsyncTask.h:
2203         (bmalloc::Function>::runSlowCase):
2204         * bmalloc/BAssert.h:
2205         * bmalloc/BoundaryTag.h:
2206         (bmalloc::BoundaryTag::setSize):
2207         * bmalloc/BoundaryTagInlines.h:
2208         (bmalloc::validate):
2209         (bmalloc::BoundaryTag::init):
2210         (bmalloc::BoundaryTag::deallocate):
2211         (bmalloc::BoundaryTag::splitLarge):
2212         (bmalloc::BoundaryTag::allocate):
2213         * bmalloc/Chunk.h:
2214         * bmalloc/Deallocator.cpp:
2215         (bmalloc::Deallocator::processObjectLog):
2216         (bmalloc::Deallocator::deallocateSlowCase):
2217         * bmalloc/Deallocator.h:
2218         (bmalloc::Deallocator::deallocateFastCase):
2219         * bmalloc/FixedVector.h:
2220         (bmalloc::Capacity>::operator):
2221         (bmalloc::Capacity>::push):
2222         (bmalloc::Capacity>::pop):
2223         (bmalloc::Capacity>::shrink):
2224         * bmalloc/Heap.cpp:
2225         (bmalloc::Heap::allocateLarge):
2226         * bmalloc/LargeChunk.h:
2227         (bmalloc::LargeChunk::get):
2228         (bmalloc::LargeChunk::endTag):
2229         * bmalloc/Line.h:
2230         (bmalloc::Line<Traits>::concurrentRef):
2231         (bmalloc::Line<Traits>::deref):
2232         * bmalloc/MediumAllocator.h:
2233         (bmalloc::MediumAllocator::allocate):
2234         * bmalloc/ObjectType.h:
2235         (bmalloc::isSmall):
2236         * bmalloc/Page.h:
2237         (bmalloc::Page<Traits>::ref):
2238         (bmalloc::Page<Traits>::deref):
2239         * bmalloc/PerThread.h:
2240         (bmalloc::PerThread<T>::getSlowCase):
2241         * bmalloc/SegregatedFreeList.cpp:
2242         (bmalloc::SegregatedFreeList::SegregatedFreeList):
2243         (bmalloc::SegregatedFreeList::insert):
2244         * bmalloc/SmallAllocator.h:
2245         (bmalloc::SmallAllocator::allocate):
2246         (bmalloc::SmallAllocator::refill):
2247         * bmalloc/Syscall.h:
2248         * bmalloc/VMAllocate.h:
2249         (bmalloc::vmValidate):
2250         (bmalloc::vmAllocate):
2251         (bmalloc::vmDeallocatePhysicalPagesSloppy):
2252         * bmalloc/Vector.h:
2253         (bmalloc::Vector<T>::operator):
2254         (bmalloc::Vector<T>::pop):
2255         (bmalloc::Vector<T>::shrink):
2256         * bmalloc/XLargeChunk.h:
2257         (bmalloc::XLargeChunk::range):
2258         (bmalloc::XLargeChunk::size):
2259
2260 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
2261
2262         Removed an unused file.
2263
2264         Unreviewed.
2265
2266         * bmalloc/AsyncTask.cpp: Removed.
2267
2268 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
2269
2270         Build bmalloc on Mac
2271         https://bugs.webkit.org/show_bug.cgi?id=131333
2272
2273         Reviewed by Mark Rowe.
2274
2275         * Makefile: Added. For make clients.
2276
2277         These files are required for building any project in WebKit. I copied
2278         them from WTF:
2279         * Configurations: Added.
2280         * Configurations/Base.xcconfig: Added.
2281         * Configurations/DebugRelease.xcconfig: Added.
2282         * Configurations/bmalloc.xcconfig: Added.
2283         * Configurations/iOS.xcconfig: Added.
2284         * Configurations/mbmalloc.xcconfig: Added.
2285
2286         * bmalloc.xcodeproj/project.pbxproj: I removed per-project-file stuff
2287         from here because everything is in .xcconfig files now.
2288
2289         I had to fix a bunch of minor warnings, since they're enabled in our
2290         .xcconfig files:
2291
2292         * bmalloc/AsyncTask.h:
2293         (bmalloc::Function>::AsyncTask):
2294         * bmalloc/BAssert.h:
2295         * bmalloc/BoundaryTagInlines.h:
2296         (bmalloc::validate):
2297         * bmalloc/Heap.cpp:
2298         (bmalloc::Heap::Heap):
2299         (bmalloc::Heap::allocateLarge):
2300         (bmalloc::Heap::deallocateLarge):
2301         * bmalloc/Mutex.h:
2302         (bmalloc::Mutex::Mutex): Deleted.
2303         * bmalloc/VMAllocate.h:
2304         (bmalloc::vmValidate):
2305         * bmalloc/mbmalloc.cpp:
2306
2307 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
2308
2309         bmalloc: Fixed a leak in the per-thread cache
2310         https://bugs.webkit.org/show_bug.cgi?id=131330
2311
2312         Reviewed by Andreas Kling.
2313
2314         Remember to deallocate our line caches upon thread exit.
2315
2316         * bmalloc/Deallocator.cpp:
2317         (bmalloc::Deallocator::~Deallocator):
2318
2319 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
2320
2321         bmalloc: rolled out the tryLock experiment
2322         https://bugs.webkit.org/show_bug.cgi?id=131328
2323
2324         Reviewed by Andreas Kling.
2325
2326         It wasn't a speedup.
2327
2328         * bmalloc.xcodeproj/project.pbxproj:
2329         * bmalloc/Allocator.cpp:
2330         (bmalloc::Allocator::processSmallAllocatorLog):
2331         (bmalloc::Allocator::processMediumAllocatorLog):
2332         * bmalloc/Deallocator.cpp:
2333         (bmalloc::Deallocator::processObjectLog):
2334         (bmalloc::Deallocator::deallocateSlowCase):
2335         (bmalloc::Deallocator::deallocateSmallLine):
2336         (bmalloc::Deallocator::deallocateMediumLine):
2337         * bmalloc/Deallocator.h:
2338         (bmalloc::Deallocator::deallocateFastCase):
2339         * bmalloc/Heap.h:
2340         (bmalloc::Heap::deallocateSmallLine):
2341         (bmalloc::Heap::deallocateMediumLine):
2342         * bmalloc/Line.h:
2343         (bmalloc::Line<Traits>::deref):
2344         * bmalloc/Page.h:
2345         (bmalloc::Page<Traits>::deref):
2346
2347 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
2348
2349         bmalloc
2350         https://bugs.webkit.org/show_bug.cgi?id=131170
2351
2352         Reviewed by Andreas Kling.
2353
2354         Initial commit.
2355
2356         * bmalloc: Added.
2357         * bmalloc.xcodeproj: Added.
2358         * bmalloc.xcodeproj/project.pbxproj: Added.
2359         * bmalloc/Algorithm.h: Added.
2360         (bmalloc::max):
2361         (bmalloc::min):
2362         (bmalloc::mask):
2363         (bmalloc::test):
2364         (bmalloc::roundUpToMultipleOf):
2365         (bmalloc::roundDownToMultipleOf):
2366         (bmalloc::sizeOf):
2367         (bmalloc::bitCount):
2368         (bmalloc::isPowerOfTwo):
2369         * bmalloc/Allocator.cpp: Added.
2370         (bmalloc::Allocator::Allocator):
2371         (bmalloc::Allocator::~Allocator):
2372         (bmalloc::Allocator::log):
2373         (bmalloc::Allocator::processSmallAllocatorLog):
2374         (bmalloc::Allocator::processMediumAllocatorLog):
2375         (bmalloc::Allocator::allocateLarge):
2376         (bmalloc::Allocator::allocateXLarge):
2377         (bmalloc::Allocator::allocateMedium):
2378         (bmalloc::Allocator::allocateSlowCase):
2379         * bmalloc/Allocator.h: Added.
2380         (bmalloc::Allocator::smallAllocatorFor):
2381         (bmalloc::Allocator::allocateFastCase):
2382         (bmalloc::Allocator::allocate):
2383         * bmalloc/AsyncTask.cpp: Added.
2384         (bmalloc::AsyncTask<Function>::runSlowCase):
2385         (bmalloc::AsyncTask<Function>::pthreadEntryPoint):
2386         (bmalloc::AsyncTask<Function>::entryPoint):
2387         * bmalloc/AsyncTask.h: Added.
2388         (bmalloc::Function>::AsyncTask):
2389         (bmalloc::Function>::join):
2390         (bmalloc::Function>::run):
2391         (bmalloc::Function>::runSlowCase):
2392         (bmalloc::Function>::pthreadEntryPoint):
2393         (bmalloc::Function>::entryPoint):
2394         * bmalloc/BAssert.h: Added.
2395         * bmalloc/BeginTag.h: Added.
2396         (bmalloc::BeginTag::isInFreeList):
2397         * bmalloc/BoundaryTag.h: Added.
2398         (bmalloc::BoundaryTag::isXLarge):
2399         (bmalloc::BoundaryTag::setXLarge):
2400         (bmalloc::BoundaryTag::isFree):
2401         (bmalloc::BoundaryTag::setFree):
2402         (bmalloc::BoundaryTag::isEnd):
2403         (bmalloc::BoundaryTag::setEnd):
2404         (bmalloc::BoundaryTag::hasPhysicalPages):
2405         (bmalloc::BoundaryTag::setHasPhysicalPages):
2406         (bmalloc::BoundaryTag::isNull):
2407         (bmalloc::BoundaryTag::clear):
2408         (bmalloc::BoundaryTag::size):
2409         (bmalloc::BoundaryTag::setSize):
2410         (bmalloc::BoundaryTag::prev):
2411         (bmalloc::BoundaryTag::next):
2412         * bmalloc/BoundaryTagInlines.h: Added.
2413         (bmalloc::validate):
2414         (bmalloc::validatePrev):
2415         (bmalloc::validateNext):
2416         (bmalloc::BoundaryTag::init):
2417         (bmalloc::BoundaryTag::mergeLargeLeft):
2418         (bmalloc::BoundaryTag::mergeLargeRight):
2419         (bmalloc::BoundaryTag::mergeLarge):
2420         (bmalloc::BoundaryTag::deallocate):
2421         (bmalloc::BoundaryTag::splitLarge):
2422         (bmalloc::BoundaryTag::allocate):
2423         * bmalloc/Cache.cpp: Added.
2424         (bmalloc::Cache::operator new):
2425         (bmalloc::Cache::operator delete):
2426         (bmalloc::Cache::Cache):
2427         (bmalloc::Cache::allocateSlowCase):
2428         (bmalloc::Cache::allocateSlowCaseNullCache):
2429         (bmalloc::Cache::deallocateSlowCase):
2430         (bmalloc::Cache::deallocateSlowCaseNullCache):
2431         * bmalloc/Cache.h: Added.
2432         (bmalloc::Cache::allocator):
2433         (bmalloc::Cache::deallocator):
2434         (bmalloc::Cache::allocateFastCase):
2435         (bmalloc::Cache::deallocateFastCase):
2436         (bmalloc::Cache::allocate):
2437         (bmalloc::Cache::deallocate):
2438         * bmalloc/Chunk.h: Added.
2439         (bmalloc::Chunk::begin):
2440         (bmalloc::Chunk::end):
2441         (bmalloc::Chunk::lines):
2442         (bmalloc::Chunk::pages):
2443         * bmalloc/Deallocator.cpp: Added.
2444         (bmalloc::Deallocator::Deallocator):
2445         (bmalloc::Deallocator::~Deallocator):
2446         (bmalloc::Deallocator::deallocateLarge):
2447         (bmalloc::Deallocator::deallocateXLarge):
2448         (bmalloc::Deallocator::processObjectLog):
2449         (bmalloc::Deallocator::deallocateSlowCase):
2450         (bmalloc::Deallocator::deallocateSmallLine):
2451         (bmalloc::Deallocator::allocateSmallLine):
2452         (bmalloc::Deallocator::deallocateMediumLine):
2453         (bmalloc::Deallocator::allocateMediumLine):
2454         * bmalloc/Deallocator.h: Added.
2455         (bmalloc::Deallocator::deallocateFastCase):
2456         (bmalloc::Deallocator::deallocate):
2457         * bmalloc/EndTag.h: Added.
2458         (bmalloc::EndTag::operator=):
2459         * bmalloc/FixedVector.h: Added.
2460         (bmalloc::FixedVector::begin):
2461         (bmalloc::FixedVector::end):
2462         (bmalloc::FixedVector::size):
2463         (bmalloc::FixedVector::capacity):
2464         (bmalloc::FixedVector::clear):
2465         (bmalloc::FixedVector::isEmpty):
2466         (bmalloc::Capacity>::FixedVector):
2467         (bmalloc::Capacity>::operator):
2468         (bmalloc::Capacity>::push):
2469         (bmalloc::Capacity>::pop):
2470         (bmalloc::Capacity>::shrink):
2471         * bmalloc/Heap.cpp: Added.
2472         (bmalloc::sleep):
2473         (bmalloc::Heap::Heap):
2474         (bmalloc::Heap::concurrentScavenge):
2475         (bmalloc::Heap::scavengeSmallPages):
2476         (bmalloc::Heap::scavengeMediumPages):
2477         (bmalloc::Heap::scavengeLargeRanges):
2478         (bmalloc::Heap::allocateSmallLineSlowCase):
2479         (bmalloc::Heap::allocateMediumLineSlowCase):
2480         (bmalloc::Heap::allocateXLarge):
2481         (bmalloc::Heap::deallocateXLarge):
2482         (bmalloc::Heap::allocateLarge):
2483         (bmalloc::Heap::deallocateLarge):
2484         * bmalloc/Heap.h: Added.
2485         (bmalloc::Heap::deallocateSmallLine):
2486         (bmalloc::Heap::allocateSmallLine):
2487         (bmalloc::Heap::deallocateMediumLine):
2488         (bmalloc::Heap::allocateMediumLine):
2489         * bmalloc/Inline.h: Added.
2490         * bmalloc/LargeChunk.h: Added.
2491         (bmalloc::LargeChunk::begin):
2492         (bmalloc::LargeChunk::end):
2493         (bmalloc::LargeChunk::create):
2494         (bmalloc::LargeChunk::get):
2495         (bmalloc::LargeChunk::beginTag):
2496         (bmalloc::LargeChunk::endTag):
2497         * bmalloc/Line.h: Added.
2498         (bmalloc::Line<Traits>::begin):
2499         (bmalloc::Line<Traits>::end):
2500         (bmalloc::Line<Traits>::concurrentRef):
2501         (bmalloc::Line<Traits>::deref):
2502         * bmalloc/MediumAllocator.h: Added.
2503         (bmalloc::MediumAllocator::isNull):
2504         (bmalloc::MediumAllocator::MediumAllocator):
2505         (bmalloc::MediumAllocator::line):
2506         (bmalloc::MediumAllocator::allocate):
2507         (bmalloc::MediumAllocator::derefCount):
2508         (bmalloc::MediumAllocator::refill):
2509         * bmalloc/MediumChunk.h: Added.
2510         * bmalloc/MediumLine.h: Added.
2511         * bmalloc/MediumPage.h: Added.
2512         * bmalloc/MediumTraits.h: Added.
2513         * bmalloc/Mutex.cpp: Added.
2514         (bmalloc::Mutex::lockSlowCase):
2515         * bmalloc/Mutex.h: Added.
2516         (bmalloc::Mutex::Mutex):
2517         (bmalloc::Mutex::try_lock):
2518         (bmalloc::Mutex::lock):
2519         (bmalloc::Mutex::unlock):
2520         * bmalloc/ObjectType.cpp: Added.
2521         (bmalloc::objectType):
2522         * bmalloc/ObjectType.h: Added.
2523         (bmalloc::isSmallOrMedium):
2524         (bmalloc::isSmall):
2525         * bmalloc/Page.h: Added.
2526         (bmalloc::Page<Traits>::ref):
2527         (bmalloc::Page<Traits>::deref):
2528         (bmalloc::Page<Traits>::refCount):
2529         * bmalloc/PerProcess.h: Added.
2530         (bmalloc::PerProcess::mutex):
2531         (bmalloc::PerProcess<T>::getFastCase):
2532         (bmalloc::PerProcess<T>::get):
2533         (bmalloc::PerProcess<T>::getSlowCase):
2534         * bmalloc/PerThread.h: Added.
2535         (bmalloc::PerThreadStorage<Cache>::get):
2536         (bmalloc::PerThreadStorage<Cache>::init):
2537         (bmalloc::PerThreadStorage::get):
2538         (bmalloc::PerThreadStorage::init):
2539         (bmalloc::PerThread<T>::getFastCase):
2540         (bmalloc::PerThread<T>::get):
2541         (bmalloc::PerThread<T>::destructor):
2542         (bmalloc::PerThread<T>::getSlowCase):
2543         * bmalloc/Range.h: Added.
2544         (bmalloc::Range::Range):
2545         (bmalloc::Range::begin):
2546         (bmalloc::Range::end):
2547         (bmalloc::Range::size):
2548         (bmalloc::Range::operator!):
2549         (bmalloc::Range::operator<):
2550         * bmalloc/SegregatedFreeList.cpp: Added.
2551         (bmalloc::SegregatedFreeList::SegregatedFreeList):
2552         (bmalloc::SegregatedFreeList::insert):
2553         (bmalloc::SegregatedFreeList::takeGreedy):
2554         (bmalloc::SegregatedFreeList::take):
2555         * bmalloc/SegregatedFreeList.h: Added.
2556         * bmalloc/Sizes.h: Added.
2557         * bmalloc/SmallAllocator.h: Added.
2558         (bmalloc::SmallAllocator::isNull):
2559         (bmalloc::SmallAllocator::canAllocate):
2560         (bmalloc::SmallAllocator::SmallAllocator):
2561         (bmalloc::SmallAllocator::line):
2562         (bmalloc::SmallAllocator::allocate):
2563         (bmalloc::SmallAllocator::objectCount):
2564         (bmalloc::SmallAllocator::derefCount):
2565         (bmalloc::SmallAllocator::refill):
2566         * bmalloc/SmallChunk.h: Added.
2567         * bmalloc/SmallLine.h: Added.
2568         * bmalloc/SmallPage.h: Added.
2569         * bmalloc/SmallTraits.h: Added.
2570         * bmalloc/Syscall.h: Added.
2571         * bmalloc/VMAllocate.h: Added.
2572         (bmalloc::vmSize):
2573         (bmalloc::vmValidate):
2574         (bmalloc::vmAllocate):
2575         (bmalloc::vmDeallocate):
2576         (bmalloc::vmDeallocatePhysicalPages):
2577         (bmalloc::vmAllocatePhysicalPages):
2578         (bmalloc::vmDeallocatePhysicalPagesSloppy):
2579         (bmalloc::vmAllocatePhysicalPagesSloppy):
2580         * bmalloc/VMHeap.cpp: Added.
2581         (bmalloc::VMHeap::VMHeap):
2582         (bmalloc::VMHeap::allocateSmallChunk):
2583         (bmalloc::VMHeap::allocateMediumChunk):
2584         (bmalloc::VMHeap::allocateLargeChunk):
2585         * bmalloc/VMHeap.h: Added.
2586         (bmalloc::VMHeap::allocateSmallPage):
2587         (bmalloc::VMHeap::allocateMediumPage):
2588         (bmalloc::VMHeap::allocateLargeRange):
2589         (bmalloc::VMHeap::deallocateSmallPage):
2590         (bmalloc::VMHeap::deallocateMediumPage):
2591         (bmalloc::VMHeap::deallocateLargeRange):
2592         * bmalloc/Vector.h: Added.
2593         (bmalloc::Vector::begin):
2594         (bmalloc::Vector::end):
2595         (bmalloc::Vector::size):
2596         (bmalloc::Vector::capacity):
2597         (bmalloc::Vector::last):
2598         (bmalloc::Vector::pop):
2599         (bmalloc::Vector<T>::Vector):
2600         (bmalloc::Vector<T>::~Vector):
2601         (bmalloc::Vector<T>::operator):
2602         (bmalloc::Vector<T>::push):
2603         (bmalloc::Vector<T>::pop):
2604         (bmalloc::Vector<T>::shrink):
2605         (bmalloc::Vector<T>::reallocateBuffer):
2606         (bmalloc::Vector<T>::shrinkCapacity):
2607         (bmalloc::Vector<T>::growCapacity):
2608         * bmalloc/XLargeChunk.h: Added.
2609         (bmalloc::XLargeChunk::get):
2610         (bmalloc::XLargeChunk::begin):
2611         (bmalloc::XLargeChunk::XLargeChunk):
2612         (bmalloc::XLargeChunk::create):
2613         (bmalloc::XLargeChunk::destroy):
2614         (bmalloc::XLargeChunk::range):
2615         (bmalloc::XLargeChunk::size):
2616         * bmalloc/bmalloc.h: Added.
2617         (bmalloc::api::malloc):
2618         (bmalloc::api::free):
2619         (bmalloc::api::realloc):
2620         * bmalloc/mbmalloc.cpp: Added.
2621