Remove unused definitions of WEBKIT_VERSION_MIN_REQUIRED
[WebKit-https.git] / Source / bmalloc / ChangeLog
1 2015-05-23  Dan Bernstein  <mitz@apple.com>
2
3         Remove unused definitions of WEBKIT_VERSION_MIN_REQUIRED
4         https://bugs.webkit.org/show_bug.cgi?id=145345
5
6         Reviewed by Sam Weinig.
7
8         * Configurations/Base.xcconfig: Also changed to use $(inherited).
9
10 2015-05-07  Geoffrey Garen  <ggaren@apple.com>
11
12         Release assert in com.apple.WebKit.WebContent under JavaScriptCore: JSC::JSONProtoFuncStringify
13         https://bugs.webkit.org/show_bug.cgi?id=144758
14
15         Reviewed by Andreas Kling.
16
17         This was an out-of-memory error when trying to shrink a string builder.
18         bmalloc was missing the optimization that allowed realloc() to shrink
19         without copying. So, let's add it.
20
21         * bmalloc/Allocator.cpp:
22         (bmalloc::Allocator::reallocate): Added Large and XLarge cases for
23         shrinking without copying. This isn't possible for small and medium
24         objects, and probably not very profitable, either.
25
26         * bmalloc/Heap.cpp:
27         (bmalloc::Heap::findXLarge):
28         (bmalloc::Heap::deallocateXLarge):
29         * bmalloc/Heap.h: Refactored this code to return a reference to an
30         XLarge range. This makes the code reusable, and also makes it easier
31         for realloc() to update metadata.
32
33         * bmalloc/LargeObject.h:
34         (bmalloc::LargeObject::split): Allow allocated objects to split because
35         that's what realloc() wants to do, and there's nothing intrinsically
36         wrong with it.
37
38 2015-05-07  Dan Bernstein  <mitz@apple.com>
39
40         <rdar://problem/19317140> [Xcode] Remove usage of AspenFamily.xcconfig in Source/
41         https://bugs.webkit.org/show_bug.cgi?id=144727
42
43         Reviewed by Darin Adler.
44
45         * Configurations/Base.xcconfig: Dont’s include AspenFamily.xcconfig, and define
46         INSTALL_PATH_PREFIX for the iOS 8.x Simulator.
47
48 2015-04-01  Alex Christensen  <achristensen@webkit.org>
49
50         Progress towards CMake on Windows and Mac.
51         https://bugs.webkit.org/show_bug.cgi?id=143293
52
53         Reviewed by Filip Pizlo.
54
55         * bmalloc/BAssert.h:
56         Removed ellipses from macros to appease Visual Studio.
57
58 2015-03-13  Alex Christensen  <achristensen@webkit.org>
59
60         Progress towards CMake on Mac.
61         https://bugs.webkit.org/show_bug.cgi?id=142680
62
63         Reviewed by Gyuyoung Kim.
64
65         * CMakeLists.txt:
66         * PlatformMac.cmake:
67         Added Zone.cpp to Mac CMake builds.
68
69 2015-03-12  Geoffrey Garen  <ggaren@apple.com>
70
71         Assertion failure in bmalloc::LargeObject::validateSelf on Mavericks Debug layout test bot
72         https://bugs.webkit.org/show_bug.cgi?id=142642
73
74         Reviewed by Michael Saboff.
75
76         The typical backtrace to this crash shows the main thread trying to
77         realloc a large string while a DFG compiler thread tries to
78         free a large vector buffer.
79
80         I believe that this is a race condition -- at least in debug builds --
81         since the main thread will try to validate its object's neighbors
82         without holding a lock, even though those neighbors might be in the
83         midst of changing.
84
85         In general, there may be sneaky times when it is valid to look at an
86         object's metadata without holding the heap lock, but it is best not to
87         do so unless we have a really really good reason to.
88
89         * bmalloc/Allocator.cpp:
90         (bmalloc::Allocator::reallocate): Take a lock before reading the metadata
91         for this object, since we generally require any access to shared heap
92         metadata to take a lock.
93
94 2015-03-10  Geoffrey Garen  <ggaren@apple.com>
95
96         bmalloc: tryFastMalloc shouldn't crash
97         https://bugs.webkit.org/show_bug.cgi?id=142443
98
99         Reviewed by Sam Weinig.
100
101         Rolling back in r181307 with a check for whether bmalloc is enabled, to
102         avoid crashes when running with ASan and GuardMalloc.
103
104         * bmalloc/Allocator.cpp:
105         (bmalloc::Allocator::tryAllocate):
106         * bmalloc/Allocator.h:
107         * bmalloc/Cache.cpp:
108         (bmalloc::Cache::tryAllocateSlowCaseNullCache):
109         * bmalloc/Cache.h:
110         (bmalloc::Cache::tryAllocate):
111         * bmalloc/Heap.cpp:
112         (bmalloc::Heap::allocateXLarge):
113         (bmalloc::Heap::tryAllocateXLarge):
114         * bmalloc/Heap.h:
115         * bmalloc/VMAllocate.h:
116         (bmalloc::tryVMAllocate):
117         (bmalloc::vmAllocate):
118         * bmalloc/bmalloc.h:
119         (bmalloc::api::tryMalloc):
120         (bmalloc::api::realloc):
121         (bmalloc::api::free):
122
123 2015-03-09  Commit Queue  <commit-queue@webkit.org>
124
125         Unreviewed, rolling out r181307.
126         https://bugs.webkit.org/show_bug.cgi?id=142525
127
128         Broke ASan tests (Requested by ap on #webkit).
129
130         Reverted changeset:
131
132         "bmalloc: tryFastMalloc shouldn't crash"
133         https://bugs.webkit.org/show_bug.cgi?id=142443
134         http://trac.webkit.org/changeset/181307
135
136 2015-03-09  Geoffrey Garen  <ggaren@apple.com>
137
138         bmalloc: tryFastMalloc shouldn't crash
139         https://bugs.webkit.org/show_bug.cgi?id=142443
140
141         Reviewed by Darin Adler.
142
143         Added support for tryMalloc.
144
145         We assume that non-x-large allocations always succeed, and we crash
146         otherwise, since normal allocation failure will just cause the next
147         non-try allocation or internal metadata allocation to fail, and it's
148         hard and not really useful to keep limping along after that. But
149         extra-large allocations can meaningfully fail, and we can recover.
150
151         * bmalloc/Heap.cpp:
152         (bmalloc::Heap::allocateXLarge):
153         (bmalloc::Heap::tryAllocateXLarge):
154         * bmalloc/Heap.h: Added support for non-crashy x-large allocation.
155
156         * bmalloc/VMAllocate.h:
157         (bmalloc::tryVMAllocate):
158         (bmalloc::vmAllocate): Added support for non-crashy VM allocation.
159
160         * bmalloc/bmalloc.h:
161         (bmalloc::api::tryMalloc):
162         (bmalloc::api::realloc):
163         (bmalloc::api::free): Tried to clarify our behavior with some comments.
164         Unfortunately, calling what we do "malloc" is still not quite right, since
165         malloc returns null on failure and we don't.
166
167 2015-03-03  Geoffrey Garen  <ggaren@apple.com>
168
169         bmalloc: Don't branch when setting the owner of a large object
170         https://bugs.webkit.org/show_bug.cgi?id=142241
171
172         Reviewed by Andreas Kling.
173
174         * bmalloc/BoundaryTag.h:
175         (bmalloc::BoundaryTag::owner):
176         (bmalloc::BoundaryTag::setOwner):
177
178 2015-03-03  Geoffrey Garen  <ggaren@apple.com>
179
180         bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
181         https://bugs.webkit.org/show_bug.cgi?id=141802
182
183         Reviewed by Andreas Kling.
184
185         Re-enabled this feature on iOS, now that the iOS crash should be fixed.
186
187         * bmalloc/VMHeap.cpp:
188         (bmalloc::VMHeap::grow):
189         * bmalloc/VMHeap.h:
190
191 2015-03-03  Geoffrey Garen  <ggaren@apple.com>
192
193         bmalloc: Added missing features to the malloc zone introspection API
194         https://bugs.webkit.org/show_bug.cgi?id=142235
195
196         Reviewed by Andreas Kling.
197
198         This should fix the crash we saw on the iOS PLT bot
199         (c.f. http://trac.webkit.org/changeset/180604).
200
201         * bmalloc/Zone.cpp:
202         (bmalloc::good_size):
203         (bmalloc::check):
204         (bmalloc::print):
205         (bmalloc::log):
206         (bmalloc::force_lock):
207         (bmalloc::force_unlock):
208         (bmalloc::statistics):
209         (bmalloc::size):
210         (bmalloc::enumerator): Provide all of these functions since they are called
211         indiscriminately on all zones.
212
213         (bmalloc::Zone::Zone):
214         (bmalloc::Zone::size): Deleted.
215         (bmalloc::Zone::enumerator): Deleted. Moved these functions out of the
216         Zone class since they can stand alone.
217
218         * bmalloc/Zone.h:
219
220 2015-03-03  Geoffrey Garen  <ggaren@apple.com>
221
222         bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
223         https://bugs.webkit.org/show_bug.cgi?id=141802
224
225         Reviewed by Andreas Kling.
226
227         Rolling back in but disabled on iOS until I can debug why the iOS PLT crashes.
228
229         * bmalloc/VMHeap.cpp:
230         (bmalloc::VMHeap::grow):
231         * bmalloc/VMHeap.h:
232         * bmalloc/Zone.cpp:
233         (bmalloc::Zone::size):
234         (bmalloc::Zone::Zone):
235         * bmalloc/Zone.h:
236
237 2015-03-03  Geoffrey Garen  <ggaren@apple.com>
238
239         bmalloc: Miscellaneous cleanup
240         https://bugs.webkit.org/show_bug.cgi?id=142231
241
242         Reviewed by Andreas Kling.
243
244         No performance change -- maybe a tiny reduction in memory use.
245
246         * bmalloc/Heap.cpp: Moved the sleep function into StaticMutex, since
247         it's a helper for working with mutexes.
248
249         (bmalloc::Heap::scavenge): Make sure to wait before we start any
250         scavenging, since individual scavenging functions now always scavenge
251         at least one page before waiting themselves.
252
253         (bmalloc::Heap::scavengeSmallPages):
254         (bmalloc::Heap::scavengeMediumPages):
255         (bmalloc::Heap::scavengeLargeObjects): Use the new wait helper to
256         simplify this code. Also, we now require our caller to wait until at
257         least one deallocation is desirable. This simplifies our loop.
258
259         (bmalloc::Heap::allocateSmallPage):
260         (bmalloc::Heap::allocateMediumPage):
261         (bmalloc::Heap::allocateXLarge):
262         (bmalloc::Heap::allocateLarge): Don't freak out any time the heap does
263         an allocation. Only consider the heap to be growing if it actually needs
264         to allocate new VM. This allows us to shrink the heap back down from a
265         high water mark more reliably even if heap activity continues.
266
267         (bmalloc::sleep): Deleted.
268         (bmalloc::Heap::scavengeLargeRanges): Renamed to match our use of
269         "LargeObject".
270
271         * bmalloc/Heap.h:
272
273         * bmalloc/LargeObject.h:
274         (bmalloc::LargeObject::operator bool): Added to simplify a while loop.
275
276         * bmalloc/StaticMutex.h:
277         (bmalloc::sleep):
278         (bmalloc::waitUntilFalse): New helper for waiting until a condition
279         becomes reliably false.
280
281         * bmalloc/Vector.h:
282         (bmalloc::Vector<T>::~Vector): Oops! Don't deallocate the null pointer.
283         We don't actually run any Vector destructors, but an iteration of this
284         patch did, and then crashed. So, let's fix that.
285
286 2015-03-02  Geoffrey Garen  <ggaren@apple.com>
287
288         bmalloc: Eagerly remove allocated objects from the free list
289         https://bugs.webkit.org/show_bug.cgi?id=142194
290
291         Reviewed by Andreas Kling.
292
293         This reduces the pressure to garbage collect the free list.
294
295         Might be a 1% speedup on MallocBench.
296
297         * bmalloc/FreeList.cpp: Put this comment at the top of the file instead
298         of repeating it inside of each function. Tried to clarify the details.
299
300         (bmalloc::FreeList::takeGreedy): Matched the other iteration code in this
301         file for consistency -- even though either direction works fine in this
302         function.
303
304         (bmalloc::FreeList::take): Change to iterate from low to high so that we
305         can maintain an index into the vector that is not disturbed even if we
306         pop from the middle (which invalidates the last index in the vector).
307
308         Decrement i when popping from the middle to make sure that we don't
309         skip the next item after popping.
310
311         (bmalloc::FreeList::removeInvalidAndDuplicateEntries): Ditto.
312
313 2015-02-27  Ryosuke Niwa  <rniwa@webkit.org>
314
315         Fixed a typo in the previous commit.
316
317         * bmalloc/BoundaryTag.h:
318         (bmalloc::BoundaryTag::setOwner):
319
320 2015-02-27  Ryosuke Niwa  <rniwa@webkit.org>
321
322         EFL build fix after r180797.
323
324         * bmalloc/BoundaryTag.h:
325         (bmalloc::BoundaryTag::owner):
326         (bmalloc::BoundaryTag::setOwner):
327
328 2015-02-27  Geoffrey Garen  <ggaren@apple.com>
329
330         bmalloc: Pathological madvise churn on the free(malloc(x)) benchmark
331         https://bugs.webkit.org/show_bug.cgi?id=142058
332
333         Reviewed by Andreas Kling.
334
335         The churn was caused by repeatedly splitting an object with physical
336         pages from an object without, and then merging them back together again.
337         The merge would conservatively forget that we had physical pages, forcing
338         a new call to madvise on the next allocation.
339
340         This patch more strictly segregates objects in the heap from objects in
341         the VM heap, with these changes:
342
343         (1) Objects in the heap are not allowed to merge with objects in the VM
344         heap, and vice versa -- since that would erase our precise knowledge of
345         which physical pages had been allocated.
346
347         (2) The VM heap is exclusively responsible for allocating and deallocating
348         physical pages.
349
350         (3) The heap free list must consider entries for objects that are in the
351         VM heap to be invalid, and vice versa. (This condition can arise
352         because the free list does not eagerly remove items.)
353
354         With these changes, we can know that any valid object in the heap's free
355         list already has physical pages, and does not need to call madvise.
356
357         Note that the VM heap -- as before -- might sometimes contain ranges
358         or pieces of ranges that have physical pages, since we allow splitting
359         of ranges at granularities smaller than the VM page size. These ranges
360         can eventually merge with ranges in the heap during scavenging.
361
362         * bmalloc.xcodeproj/project.pbxproj:
363
364         * bmalloc/BoundaryTag.h:
365         (bmalloc::BoundaryTag::owner):
366         (bmalloc::BoundaryTag::setOwner):
367         (bmalloc::BoundaryTag::initSentinel):
368         (bmalloc::BoundaryTag::hasPhysicalPages): Deleted.
369         (bmalloc::BoundaryTag::setHasPhysicalPages): Deleted. Replaced the concept
370         of "has physical pages" with a bit indicating which heap owns the large
371         object. This is a more precise concept, since the old bit was really a
372         Yes / Maybe bit.
373
374         * bmalloc/Deallocator.cpp:
375
376         * bmalloc/FreeList.cpp: Adopt
377         (bmalloc::FreeList::takeGreedy):
378         (bmalloc::FreeList::take):
379         (bmalloc::FreeList::removeInvalidAndDuplicateEntries):
380         * bmalloc/FreeList.h:
381         (bmalloc::FreeList::push): Added API for considering the owner when
382         deciding if a free list entry is valid.
383
384         * bmalloc/Heap.cpp:
385         (bmalloc::Heap::Heap): Adopt new API.
386
387         (bmalloc::Heap::scavengeLargeRanges): Scavenge all ranges with no minimum,
388         since some ranges might be able to merge with ranges in the VM heap, and
389         they won't be allowed to until we scavenge them.
390
391         (bmalloc::Heap::allocateSmallPage):
392         (bmalloc::Heap::allocateMediumPage):
393         (bmalloc::Heap::allocateLarge): New VM heap API makes this function
394         simpler, since we always get back physical pages now.
395
396         * bmalloc/Heap.h:
397         * bmalloc/LargeObject.h:
398         (bmalloc::LargeObject::end):
399         (bmalloc::LargeObject::owner):
400         (bmalloc::LargeObject::setOwner):
401         (bmalloc::LargeObject::isValidAndFree):
402         (bmalloc::LargeObject::merge): Do not merge objects across heaps since
403         that causes madvise churn.
404         (bmalloc::LargeObject::validateSelf):
405         (bmalloc::LargeObject::init):
406         (bmalloc::LargeObject::hasPhysicalPages): Deleted.
407         (bmalloc::LargeObject::setHasPhysicalPages): Deleted. Propogate the Owner API.
408
409         * bmalloc/Owner.h: Added.
410
411         * bmalloc/SegregatedFreeList.cpp:
412         (bmalloc::SegregatedFreeList::SegregatedFreeList):
413         (bmalloc::SegregatedFreeList::insert):
414         (bmalloc::SegregatedFreeList::takeGreedy):
415         (bmalloc::SegregatedFreeList::take):
416         * bmalloc/SegregatedFreeList.h: Propogate the owner API.
417
418         * bmalloc/VMAllocate.h:
419         (bmalloc::vmDeallocatePhysicalPagesSloppy):
420         (bmalloc::vmAllocatePhysicalPagesSloppy): Clarified these functions and
421         removed an edge case.
422
423         * bmalloc/VMHeap.cpp:
424         (bmalloc::VMHeap::VMHeap):
425         * bmalloc/VMHeap.h:
426         (bmalloc::VMHeap::allocateSmallPage):
427         (bmalloc::VMHeap::allocateMediumPage):
428         (bmalloc::VMHeap::allocateLargeObject):
429         (bmalloc::VMHeap::deallocateLargeObject): Be sure to give each object
430         a new chance to merge, since it might have been prohibited from merging
431         before by virtue of not being in the VM heap.
432
433         (bmalloc::VMHeap::allocateLargeRange): Deleted.
434         (bmalloc::VMHeap::deallocateLargeRange): Deleted.
435
436 2015-02-26  Geoffrey Garen  <ggaren@apple.com>
437
438         bmalloc: Large object free list can grow infinitely
439         https://bugs.webkit.org/show_bug.cgi?id=142055
440
441         Reviewed by Andreas Kling.
442
443         By design, we don't eagerly remove large objects from the free list.
444         This creates two simple pathologies:
445
446             (1) If you free and then allocate the same object repeatedly, it will
447             duplicate itself in the free list repeatedly. Since it is never
448             invalid at the time of allocation, it will never be removed.
449
450             (2) If you split and then merge the same object repeatedly, it will
451             duplicate its split sibling in the free list repeatedly. If its
452             sibling is in a separate free list size class, it will never be
453             consulted at the time of allocation, so it will never be removed.
454
455         So, a simple "while (1) { free(malloc(x)); }" causes infinite memory
456         use in the free list.
457
458         The solution in this patch is a simple helper to remove garbage from the
459         free list if it grows too large. This pathology is not common, so the
460         cost is OK.
461
462         Long-term, perhaps we should rethink the laziness of these free lists.
463
464         * bmalloc/BoundaryTag.h:
465         (bmalloc::BoundaryTag::isMarked):
466         (bmalloc::BoundaryTag::setMarked): New bit, used by free list GC.
467
468         * bmalloc/FreeList.cpp:
469         (bmalloc::FreeList::removeInvalidAndDuplicateEntries): The GC algorithm.
470
471         * bmalloc/FreeList.h:
472         (bmalloc::FreeList::FreeList):
473         (bmalloc::FreeList::push): Invoke the GC if we're getting huge.
474
475         * bmalloc/LargeObject.h:
476         (bmalloc::LargeObject::isMarked):
477         (bmalloc::LargeObject::setMarked):
478         (bmalloc::LargeObject::validateSelf): Expose the new bit.
479
480         * bmalloc/Sizes.h: New constant to control GC frequency.
481
482 2015-02-26  Csaba Osztrogonác  <ossy@webkit.org>
483
484         URTBF after r180693.
485
486         * CMakeLists.txt:
487
488 2015-02-26  Geoffrey Garen  <ggaren@apple.com>
489
490         Try to fix the Mac build.
491
492         Unreviewed.
493
494         * bmalloc.xcodeproj/project.pbxproj: Make FreeList.h available.
495
496 2015-02-26  Geoffrey Garen  <ggaren@apple.com>
497
498         bmalloc: Refactored SegregatedFreeList and BoundaryTag::init
499         https://bugs.webkit.org/show_bug.cgi?id=142049
500
501         Reviewed by Anders Carlsson.
502
503         Split out a FreeList class from SegregatedFreeList. This will make it
504         easier to add behaviors on free list insertion and removal -- and it's
505         probably how I should have designed things at the start.
506
507         Moved BoundaryTag::init into LargeObject, since all the related logic
508         lives in LargeObject now too, and this allows us to remove BoundaryTagInlines.h.
509
510         * bmalloc.xcodeproj/project.pbxproj:
511         * bmalloc/BoundaryTagInlines.h: Removed.
512         * bmalloc/FreeList.cpp: Copied from Source/bmalloc/bmalloc/SegregatedFreeList.cpp.
513         (bmalloc::FreeList::takeGreedy):
514         (bmalloc::FreeList::take):
515         (bmalloc::SegregatedFreeList::SegregatedFreeList): Deleted.
516         (bmalloc::SegregatedFreeList::insert): Deleted.
517         (bmalloc::SegregatedFreeList::takeGreedy): Deleted.
518         (bmalloc::SegregatedFreeList::take): Deleted.
519         * bmalloc/FreeList.h: Copied from Source/bmalloc/bmalloc/SegregatedFreeList.h.
520         (bmalloc::FreeList::push):
521         * bmalloc/LargeObject.h:
522         (bmalloc::LargeObject::init):
523         * bmalloc/SegregatedFreeList.cpp:
524         (bmalloc::SegregatedFreeList::SegregatedFreeList):
525         (bmalloc::SegregatedFreeList::insert):
526         (bmalloc::SegregatedFreeList::takeGreedy):
527         (bmalloc::SegregatedFreeList::take):
528         * bmalloc/SegregatedFreeList.h:
529         * bmalloc/Sizes.h:
530         * bmalloc/VMHeap.cpp:
531         (bmalloc::VMHeap::grow):
532
533 2015-02-26  Geoffrey Garen  <ggaren@apple.com>
534
535         bmalloc: free up a bit in BoundaryTag
536         https://bugs.webkit.org/show_bug.cgi?id=142048
537
538         Reviewed by Brady Eidson.
539
540         We were wasting a bit by accident, and I need one now.
541
542         * bmalloc/Algorithm.h:
543         (bmalloc::rightShift): Deleted. Not needed, now that I've simplified
544         the math.
545
546         * bmalloc/BoundaryTag.h: Since each boundary tag bucket is 1024 bytes
547         long, the maximum offset into a bucket is 1023.
548
549         You need 5 bits to count up to 1024, but only 4 to count up to 1023.
550         
551         Math is hard.
552
553         (bmalloc::BoundaryTag::compactBegin): Switched to division because it
554         is simpler, and easier to match up with our ASSERT. The compiler will
555         turn division by constant power of two into a shift for us.
556
557         (bmalloc::BoundaryTag::setRange): Added an ASSERT for compactBegin
558         because we do encode it, so we should ASSERT that encoding did not
559         lose information.
560
561         * bmalloc/Sizes.h: Shifting is no longer used since we use division
562         instead.
563
564 2015-02-24  Stephanie Lewis  <slewis@apple.com>
565
566         Rolling out http://trac.webkit.org/changeset/180430 as it causes the PLT to crash.
567         <rdar://problem/19948015>
568
569         Unreviewed.
570
571         * bmalloc/VMHeap.cpp:
572         (bmalloc::VMHeap::grow):
573         * bmalloc/VMHeap.h:
574         * bmalloc/Zone.cpp:
575         (bmalloc::Zone::Zone):
576         (bmalloc::Zone::size): Deleted.
577         * bmalloc/Zone.h:
578
579 2015-02-24  Geoffrey Garen  <ggaren@apple.com>
580
581         bmalloc: Added a little more abstraction for large objects
582         https://bugs.webkit.org/show_bug.cgi?id=141978
583
584         Reviewed by Sam Weinig.
585
586         Previously, each client needed to manage the boundary tags of
587         a large object using free functions. This patch introduces a LargeObject
588         class that does things a little more automatically.
589
590         * bmalloc.xcodeproj/project.pbxproj:
591
592         * bmalloc/Allocator.cpp:
593         (bmalloc::Allocator::reallocate): Use the new LargeObject class.
594
595         * bmalloc/BeginTag.h:
596         (bmalloc::BeginTag::isInFreeList): Deleted. Moved this logic into the
597         LargeObject class.
598
599         * bmalloc/BoundaryTag.h:
600         (bmalloc::BoundaryTag::isSentinel):
601         (bmalloc::BoundaryTag::compactBegin):
602         (bmalloc::BoundaryTag::setRange):
603         (bmalloc::BoundaryTag::initSentinel): Added an explicit API for sentinels,
604         which we used to create and test for implicitly.
605
606         * bmalloc/BoundaryTagInlines.h:
607         (bmalloc::BoundaryTag::init):
608         (bmalloc::validate): Deleted.
609         (bmalloc::validatePrev): Deleted.
610         (bmalloc::validateNext): Deleted.
611         (bmalloc::BoundaryTag::mergeLeft): Deleted.
612         (bmalloc::BoundaryTag::mergeRight): Deleted.
613         (bmalloc::BoundaryTag::merge): Deleted.
614         (bmalloc::BoundaryTag::deallocate): Deleted.
615         (bmalloc::BoundaryTag::split): Deleted.
616         (bmalloc::BoundaryTag::allocate): Deleted. Moved this logic into the
617         LargeObject class.
618
619         * bmalloc/EndTag.h:
620         (bmalloc::EndTag::init):
621         (bmalloc::EndTag::operator=): Deleted. Re-reading this code, I found
622         special behavior in the assignment operator to be a surprising API.
623         So, I replaced the assignment operation with an explicit initializing
624         function.
625
626         * bmalloc/Heap.cpp:
627         (bmalloc::Heap::scavengeLargeRanges):
628         (bmalloc::Heap::allocateXLarge):
629         (bmalloc::Heap::findXLarge):
630         (bmalloc::Heap::deallocateXLarge):
631         (bmalloc::Heap::allocateLarge):
632         (bmalloc::Heap::deallocateLarge):
633         * bmalloc/Heap.h: No behavior changes here -- just adopting the
634         LargeObject interface.
635
636         * bmalloc/LargeObject.h: Added.
637         (bmalloc::LargeObject::operator!):
638         (bmalloc::LargeObject::begin):
639         (bmalloc::LargeObject::size):
640         (bmalloc::LargeObject::range):
641         (bmalloc::LargeObject::LargeObject):
642         (bmalloc::LargeObject::setFree):
643         (bmalloc::LargeObject::isFree):
644         (bmalloc::LargeObject::hasPhysicalPages):
645         (bmalloc::LargeObject::setHasPhysicalPages):
646         (bmalloc::LargeObject::isValidAndFree):
647         (bmalloc::LargeObject::merge):
648         (bmalloc::LargeObject::split):
649         (bmalloc::LargeObject::validateSelf):
650         (bmalloc::LargeObject::validate): Moved this code into a class, out of
651         BoundaryTag free functions.
652
653         New to the class are these features:
654
655             (1) Every reference to an object is validated upon creation and use.
656
657             (2) There's an explicit API for "This is a reference to an object
658             that might be stale (the DoNotValidate API)".
659
660             (3) The begin and end tags are kept in sync automatically.
661
662         * bmalloc/SegregatedFreeList.cpp:
663         (bmalloc::SegregatedFreeList::insert):
664         (bmalloc::SegregatedFreeList::takeGreedy):
665         (bmalloc::SegregatedFreeList::take):
666         * bmalloc/SegregatedFreeList.h: Adopt the LargeObject interface.
667
668         * bmalloc/VMHeap.cpp:
669         (bmalloc::VMHeap::grow):
670         * bmalloc/VMHeap.h:
671         (bmalloc::VMHeap::allocateLargeRange):
672         (bmalloc::VMHeap::deallocateLargeRange): Adopt the LargeObject interface.
673
674 2015-02-20  Geoffrey Garen  <ggaren@apple.com>
675
676         bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
677         https://bugs.webkit.org/show_bug.cgi?id=141802
678
679         Reviewed by Andreas Kling.
680
681         Rolling back in with a fix for a crash seen while using GuardMalloc.
682
683         * bmalloc/VMHeap.cpp:
684         (bmalloc::VMHeap::grow):
685         * bmalloc/VMHeap.h:
686         * bmalloc/Zone.cpp: Re-land the old patch.
687
688         (bmalloc::Zone::size): Be sure to implement the size() function since
689         it's accessible indirectly via the malloc_zone_from_ptr public API --
690         and GuardMalloc calls it all the time.
691
692         (bmalloc::Zone::Zone):
693         * bmalloc/Zone.h: Re-land the old patch.
694
695 2015-02-19  Commit Queue  <commit-queue@webkit.org>
696
697         Unreviewed, rolling out r180363.
698         https://bugs.webkit.org/show_bug.cgi?id=141814
699
700         Caused >50 crashes when running LayoutTests in GuardMalloc or
701         ASAN modes. (Requested by jernoble on #webkit).
702
703         Reverted changeset:
704
705         "bmalloc should implement malloc introspection (to stop false-
706         positive leaks when MallocStackLogging is off)"
707         https://bugs.webkit.org/show_bug.cgi?id=141802
708         http://trac.webkit.org/changeset/180363
709
710 2015-02-19  Geoffrey Garen  <ggaren@apple.com>
711
712         bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
713         https://bugs.webkit.org/show_bug.cgi?id=141802
714
715         Reviewed by Andreas Kling.
716
717         Fixed a last-minute type.
718
719         The macro is OS, not PLATFORM.
720
721         * bmalloc/VMHeap.cpp:
722         (bmalloc::VMHeap::grow):
723         * bmalloc/VMHeap.h:
724         * bmalloc/Zone.h:
725
726 2015-02-19  Geoffrey Garen  <ggaren@apple.com>
727
728         bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
729         https://bugs.webkit.org/show_bug.cgi?id=141802
730
731         Reviewed by Andreas Kling.
732
733         This patch does the bare minimum to stop false positive leaks from
734         being reported by the Darwin leaks tool. We register each super chunk
735         as a single object, and then request that the leaks tool scan it.
736
737         * bmalloc.xcodeproj/project.pbxproj: Added an abstraction for the malloc
738         zone introspection API.
739
740         * bmalloc/Algorithm.h: Missing #include.
741
742         * bmalloc/VMHeap.cpp:
743         (bmalloc::VMHeap::grow):
744         * bmalloc/VMHeap.h: Adopt the new abstraction.
745
746         * bmalloc/Zone.cpp: Added.
747         (bmalloc::remoteRead): Helper for reading an object out of another process.
748         (bmalloc::Zone::enumerator):
749         (bmalloc::Zone::Zone): Register a malloc zone so that we will participate
750         in introspection.
751
752         * bmalloc/Zone.h: Added.
753         (bmalloc::Zone::superChunks):
754         (bmalloc::Zone::addSuperChunk): Use a non-dynamically-allocated vector
755         since our dynamic allocations will not be scanned by leaks since they
756         will have the malloc VM tag.
757
758 2015-02-18  Geoffrey Garen  <ggaren@apple.com>
759
760         bmalloc: VMHeap should keep a record of all of its VM ranges (for malloc introspection)
761         https://bugs.webkit.org/show_bug.cgi?id=141759
762
763         Reviewed by Andreas Kling.
764
765         * bmalloc.xcodeproj/project.pbxproj:
766         * bmalloc/SuperChunk.h: Added.
767         (bmalloc::SuperChunk::create):
768         (bmalloc::SuperChunk::SuperChunk):
769         (bmalloc::SuperChunk::smallChunk):
770         (bmalloc::SuperChunk::mediumChunk):
771         (bmalloc::SuperChunk::largeChunk): Factored out super chunk creation
772         into a separate class, for clarity and type safety.
773
774         * bmalloc/VMHeap.cpp:
775         (bmalloc::VMHeap::grow):
776         (bmalloc::VMHeap::allocateSuperChunk): Renamed "allocateSuperChunk" to
777         "grow" because Andreas found "allocateSuperChunk" to be unclear.
778
779         * bmalloc/VMHeap.h: Track all our VM ranges. We will use this information
780         for malloc introspection.
781
782         (bmalloc::VMHeap::allocateSmallPage):
783         (bmalloc::VMHeap::allocateMediumPage):
784         (bmalloc::VMHeap::allocateLargeRange): Updated for renames.
785
786 2015-02-18  Zan Dobersek  <zdobersek@igalia.com>
787
788         Build bmalloc through CMake as a static library. It's then linked either
789         into the WTF library (if built as a shared library) or into the JSC and
790         WebKit2 libraries. There's no need to build it as a standalone shared library.
791
792         Rubber-stamped by Carlos Garcia Campos.
793
794         * CMakeLists.txt:
795
796 2015-02-13  Gyuyoung Kim  <gyuyoung.kim@samsung.com>
797
798         [BMalloc] Add a FIXME comment for memory alignas
799         https://bugs.webkit.org/show_bug.cgi?id=141556
800
801         Reviewed by Csaba Osztrogonác.
802
803         * bmalloc/Chunk.h: Add a FIXME comment.
804         * bmalloc/LargeChunk.h: ditto.
805
806 2015-02-11  Csaba Osztrogonác  <ossy@webkit.org>
807
808         bmalloc buildfix on 32 bit Linux (x86/ARM)
809         https://bugs.webkit.org/show_bug.cgi?id=141472
810
811         Reviewed by Gyuyoung Kim.
812
813         * bmalloc/Algorithm.h:
814         (bmalloc::roundUpToMultipleOf):
815         * bmalloc/FixedVector.h:
816         (bmalloc::FixedVector::clear):
817         * bmalloc/Sizes.h:
818         (bmalloc::Sizes::sizeClass):
819
820 2015-02-11  Gyuyoung Kim  <gyuyoung.kim@samsung.com>
821
822         [EFL][GTK] Use bmalloc instead of tcmalloc
823         https://bugs.webkit.org/show_bug.cgi?id=140162
824
825         Reviewed by Carlos Garcia Campos.
826
827         Support to use bmalloc on EFL and GTK ports.
828
829         * CMakeLists.txt: Added.
830         * bmalloc/Allocator.cpp:
831         (bmalloc::Allocator::allocate):
832             Fix unused return value caused by posix_memalign().
833         * bmalloc/AsyncTask.h:
834         * bmalloc/BoundaryTag.h:
835         (bmalloc::BoundaryTag::clear):
836         * bmalloc/Chunk.h:
837             Change Traits::Page with Traits::PageType in order to fix
838             -fpermitive build error on EFL and GTK port. 
839         * bmalloc/EndTag.h:
840         (bmalloc::EndTag::operator=):
841         * bmalloc/Line.h: ditto.
842         * bmalloc/MediumTraits.h:
843         * bmalloc/Page.h: ditto.
844         * bmalloc/PerThread.h:
845             EFL port doesn't support __has_include definition yet.
846             Define HAVE_PTHREAD_MACHDEP_H according to check if __has_include is supported.
847         * bmalloc/SmallTraits.h: ditto.
848         * bmalloc/VMAllocate.h:
849         (bmalloc::vmDeallocatePhysicalPages):
850         (bmalloc::vmAllocatePhysicalPages):
851         * bmalloc/Vector.h:
852         (bmalloc::Vector<T>::push):
853         (bmalloc::Vector<T>::reallocateBuffer):
854
855 2015-01-31  Sam Weinig  <sam@webkit.org>
856
857         Remove even more Mountain Lion support
858         https://bugs.webkit.org/show_bug.cgi?id=141124
859
860         Reviewed by Alexey Proskuryakov.
861
862         * Configurations/Base.xcconfig:
863         * Configurations/DebugRelease.xcconfig:
864
865 2015-01-30  Geoffrey Garen  <ggaren@apple.com>
866
867         GC marking threads should clear malloc caches
868         https://bugs.webkit.org/show_bug.cgi?id=141097
869
870         Reviewed by Andreas Kling.
871
872         Split the scavenging API into per-thread vs global, so that you can
873         request to scavenge your own thread without scavenging the whole heap.
874
875         * bmalloc/Cache.cpp:
876         (bmalloc::Cache::scavenge):
877         * bmalloc/bmalloc.h:
878         (bmalloc::api::scavengeThisThread):
879         (bmalloc::api::scavenge):
880
881 2015-01-28  Dana Burkart  <dburkart@apple.com>
882
883         Move ASan flag settings from DebugRelease.xcconfig to Base.xcconfig
884         https://bugs.webkit.org/show_bug.cgi?id=136765
885
886         Reviewed by Alexey Proskuryakov.
887
888         * Configurations/Base.xcconfig:
889         * Configurations/DebugRelease.xcconfig:
890
891 2015-01-21  Geoffrey Garen  <ggaren@apple.com>
892
893         bmalloc: support aligned allocation
894         https://bugs.webkit.org/show_bug.cgi?id=140732
895
896         Reviewed by Andreas Kling.
897
898         * bmalloc/Allocator.cpp:
899         (bmalloc::Allocator::allocate): New function for aligned allocation.
900
901         Small and medium requests just allocate and free until they find an
902         aligned pointer. This is slightly inefficient in the worst case, but
903         still constant-time with little-to-no space overhead.
904
905         Large requests use a new API that requires the client to specify both
906         its ideal size and alignment, and the worst-case size you would have to
907         allocate in order to produce some interior pointer of the requested size
908         and alignment. We put the burden of this calculation on the client
909         because it simplifies things if we guarantee that allocation won't fail.
910
911         XLarge requests are easy: we just forward them to vmAllocate, which
912         already supported aligned requests.
913
914         * bmalloc/BoundaryTag.h:
915         * bmalloc/BoundaryTagInlines.h:
916         (bmalloc::BoundaryTag::mergeLeft):
917         (bmalloc::BoundaryTag::mergeRight):
918         (bmalloc::BoundaryTag::merge):
919         (bmalloc::BoundaryTag::deallocate):
920         (bmalloc::BoundaryTag::split):
921         (bmalloc::BoundaryTag::allocate): No behavior change here. I just
922         refactored the interface to remove some reference out parameters in
923         order to clarify what changes and what doesn't.
924
925         * bmalloc/Heap.cpp:
926         (bmalloc::Heap::allocateXLarge): Added an alignment API.
927
928         (bmalloc::Heap::allocateLarge):
929         * bmalloc/Heap.h: Added an alignment API. I split out allocateLarge into
930         a few variants, so aligned and unaligned allocation could share some code.
931
932         * bmalloc/SegregatedFreeList.cpp:
933         (bmalloc::SegregatedFreeList::take):
934         * bmalloc/SegregatedFreeList.h: Changed to use a separate, explicit API
935         for aligned allocation. It turns out that the aligned path is pretty
936         different, since it ends up searching for two potential ways to satisfy
937         an allocation: either large enough and aligned, or large enough to split
938         into something not aligned and something large enough and aligned.
939
940         * bmalloc/VMAllocate.h:
941         (bmalloc::vmAllocate): Switched alignment to come before size because
942         that's how the memalign API specifies it.
943
944         * bmalloc/VMHeap.h:
945         (bmalloc::VMHeap::allocateLargeRange): Added an alignment API.
946
947 2015-01-20  Geoffrey Garen  <ggaren@apple.com>
948
949         bmalloc: a little bit of cleanup
950         https://bugs.webkit.org/show_bug.cgi?id=140687
951
952         Reviewed by Anders Carlsson.
953
954         * bmalloc/Algorithm.h:
955         (bmalloc::isPowerOfTwo): Added a check for 0, since 0 would break a lot
956         of code.
957
958         * bmalloc/BoundaryTag.h:
959         * bmalloc/BoundaryTagInlines.h:
960         (bmalloc::BoundaryTag::mergeLeft):
961         (bmalloc::BoundaryTag::mergeRight):
962         (bmalloc::BoundaryTag::merge):
963         (bmalloc::BoundaryTag::deallocate):
964         (bmalloc::BoundaryTag::split):
965         (bmalloc::BoundaryTag::allocate):
966         (bmalloc::BoundaryTag::mergeLargeLeft): Deleted.
967         (bmalloc::BoundaryTag::mergeLargeRight): Deleted.
968         (bmalloc::BoundaryTag::mergeLarge): Deleted.
969         (bmalloc::BoundaryTag::splitLarge): Deleted. Removed the word "Large"
970         from all these functions, since boundary tags always pertain to large
971         objects, and putting the word "Large" everywhere wasn't helping to
972         explain that.
973
974         * bmalloc/Heap.cpp:
975         (bmalloc::Heap::allocateXLarge):
976         (bmalloc::Heap::findXLarge):
977         (bmalloc::Heap::deallocateXLarge):
978         * bmalloc/Heap.h:
979         * bmalloc/VMHeap.h:
980         (bmalloc::VMHeap::allocateXLarge): Deleted.
981         (bmalloc::VMHeap::findXLarge): Deleted.
982         (bmalloc::VMHeap::deallocateXLarge): Deleted. Moved XLarge allocation
983         from VMHeap to Heap. Since the purpose of the VMHeap is to cache VM
984         ranges, and the VMHeap never caches any XLarge ranges, it doesn't
985         really make sense for the VMHeap to be involved.
986
987 2015-01-16  Geoffrey Garen  <ggaren@apple.com>
988
989         bmalloc: refactored XLarge allocation for better alignment
990         https://bugs.webkit.org/show_bug.cgi?id=140582
991
992         Reviewed by Andreas Kling.
993
994         XLarge objects used to be Large objects with an extra bit of metadata
995         that said "actually, I'm not large -- I'm extra large".
996
997         The metadata header in an XLarge allocation made it impossible for the
998         XLarge object to honor a very large alignment request.
999
1000         The solution is to stop using a metadata header for XLarge objects, and
1001         instead to store explicit metadata on the side.
1002
1003         This is a bit less astonishing, which is also nice.
1004
1005         Finding XLarge metadata is now a linear search. That's probably OK, since
1006         it was always so in TCMalloc, and the usual number of XLarge allocations
1007         in a process is 0.
1008
1009         This design makes it possible for the heap to cache XLarge allocations
1010         with and/or without physical pages. I haven't actually done that yet
1011         because the tradeoffs are subtle, so I don't want to do anything without
1012         a motivating test case.
1013
1014         * bmalloc.xcodeproj/project.pbxproj:
1015         * bmalloc/Allocator.cpp:
1016         (bmalloc::Allocator::reallocate): Removed the concept of an XLargeChunk,
1017         since an XLarge allocation is now just a naked buffer without a header.
1018
1019         (bmalloc::Allocator::allocateXLarge): Added an explicit qualifier for
1020         XLarge alignment, since XLargeChunk won't give this to us implicitly
1021         anymore.
1022
1023         * bmalloc/BoundaryTag.h:
1024         (bmalloc::BoundaryTag::setRange):
1025         (bmalloc::BoundaryTag::isXLarge): Deleted.
1026         (bmalloc::BoundaryTag::setXLarge): Deleted.
1027         * bmalloc/BoundaryTagInlines.h:
1028         (bmalloc::validate):
1029         (bmalloc::BoundaryTag::deallocate): Removed the XLarge hacks from Large allocations.
1030
1031         * bmalloc/Deallocator.cpp:
1032         (bmalloc::Deallocator::deallocateXLarge):
1033         (bmalloc::Deallocator::deallocateSlowCase):
1034         * bmalloc/Heap.cpp:
1035         (bmalloc::Heap::findXLarge):
1036         (bmalloc::Heap::allocateXLarge):
1037         (bmalloc::Heap::deallocateXLarge):
1038         * bmalloc/Heap.h: Updated for interface changes.
1039
1040         * bmalloc/ObjectType.cpp:
1041         (bmalloc::objectType):
1042         * bmalloc/ObjectType.h:
1043         (bmalloc::isXLarge): We can now tell if a pointer is XLarge just by
1044         examining its bit pattern -- just like we do for other kinds of
1045         allocations -- which is nice.
1046
1047         * bmalloc/Sizes.h:
1048         * bmalloc/VMHeap.h:
1049         (bmalloc::VMHeap::allocateXLarge):
1050         (bmalloc::VMHeap::findXLarge):
1051         (bmalloc::VMHeap::deallocateXLarge): Keep an explicit vector of metadata
1052         for XLarge allocations.
1053
1054         * bmalloc/XLargeChunk.h: Removed.
1055
1056 2015-01-16  Geoffrey Garen  <ggaren@apple.com>
1057
1058         bmalloc: added some infrastructure for aligned allocation
1059         https://bugs.webkit.org/show_bug.cgi?id=140572
1060
1061         Reviewed by Andreas Kling.
1062
1063         * bmalloc/Algorithm.h:
1064         (bmalloc::isPowerOfTwo):
1065         (bmalloc::roundUpToMultipleOf):
1066         (bmalloc::roundDownToMultipleOf): Refactored some duplicate code to use our
1067         isPowerOfTwo helper function.
1068
1069         * bmalloc/Allocator.cpp:
1070         (bmalloc::Allocator::allocate):
1071         * bmalloc/Allocator.h: Stubbed out an implementation of aligned allocation.
1072         Doesn't do anything yet, but does correctly forward to system malloc
1073         when bmalloc is disabled.
1074
1075         * bmalloc/Cache.cpp:
1076         (bmalloc::Cache::allocateSlowCaseNullCache):
1077         * bmalloc/Cache.h:
1078         (bmalloc::Cache::allocate):
1079         * bmalloc/bmalloc.h:
1080         (bmalloc::api::memalign):
1081         * bmalloc/mbmalloc.cpp: Stubbed out an API for aligned allocation.
1082
1083 2015-01-13  Geoffrey Garen  <ggaren@apple.com>
1084
1085         Consider alignment when allocating from a SegregatedFreeList
1086         https://bugs.webkit.org/show_bug.cgi?id=140408
1087
1088         Reviewed by Sam Weinig.
1089
1090         In preparation for supporting aligned allocation.
1091
1092         No performance change.
1093
1094         Since this is just one extra branch in an already expensive function,
1095         I decided not to duplicate the function just to avoid the branch in
1096         the un-aligned case.
1097
1098         * bmalloc/SegregatedFreeList.cpp:
1099         (bmalloc::SegregatedFreeList::take):
1100         * bmalloc/SegregatedFreeList.h:
1101
1102 2015-01-13  Geoffrey Garen  <ggaren@apple.com>
1103
1104         Renamed minimum to size in SegregatedFreeList
1105         https://bugs.webkit.org/show_bug.cgi?id=140406
1106
1107         Reviewed by Sam Weinig.
1108
1109         In preparation for supporting aligned allocation.
1110
1111         * bmalloc/SegregatedFreeList.cpp:
1112         (bmalloc::SegregatedFreeList::takeGreedy):
1113         (bmalloc::SegregatedFreeList::take): Every size passed to malloc is
1114         really just a minimum. Let's not imply that this value is special.
1115
1116 2015-01-11  Dan Bernstein  <mitz@apple.com>
1117
1118         Geoff is organized, but he is not an organization.
1119
1120         Rubber-stamped by Anders Carlsson.
1121
1122         * bmalloc.xcodeproj/project.pbxproj: Removed the ORGANIZATIONNAME project attribute.
1123
1124 2015-01-07  Geoffrey Garen  <ggaren@apple.com>
1125
1126         Make bmalloc work with ASan
1127         https://bugs.webkit.org/show_bug.cgi?id=140194
1128
1129         Reviewed by Mark Lam.
1130
1131         * bmalloc/BPlatform.h: Added a way to detect Darwin OSes, since we need
1132         an OS-specific API to test for loaded runtime libraries.
1133
1134         * bmalloc/Environment.cpp:
1135         (bmalloc::isASanEnabled):
1136         (bmalloc::Environment::computeIsBmallocEnabled): Disabled bmalloc if
1137         ASan is enabled, since system malloc has the Asan hooks we need.
1138
1139         You could check for the ASan compile-time flag instead, but doing this
1140         check at runtime prepares bmalloc for a world where it is a dynamic
1141         library that might be loaded into projects it did not compile with.
1142
1143 2015-01-05  Geoffrey Garen  <ggaren@apple.com>
1144
1145         Fix up bmalloc's PerThread for use on Linux
1146         https://bugs.webkit.org/show_bug.cgi?id=139804
1147
1148         Reviewed by Anders Carlsson.
1149
1150         The previous implementation was a bit slow.
1151
1152         * bmalloc/PerThread.h:
1153         (bmalloc::PerThreadStorage<Cache>::get):
1154         (bmalloc::PerThreadStorage::get):
1155         (bmalloc::PerThreadStorage::init): Added a catch-all cross-platform Unix
1156         way to do fast per-thread access without taking a lock every time. This
1157         probably works on all the platforms we care about, and it matches other
1158         techniques we use elsewhere in WebKit.
1159
1160         (bmalloc::PerThread<T>::getFastCase): Removed the conditional from
1161         this class because PerThreadStorage now encapsulates everything that
1162         needs to be conditional.
1163
1164         (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Deleted.
1165
1166 2014-12-26  Dan Bernstein  <mitz@apple.com>
1167
1168         <rdar://problem/19348208> REGRESSION (r177027): iOS builds use the wrong toolchain
1169         https://bugs.webkit.org/show_bug.cgi?id=139950
1170
1171         Reviewed by David Kilzer.
1172
1173         * Configurations/Base.xcconfig: Only define TOOLCHAINS when building for OS X, doing so
1174         in a manner that works with Xcode 5.1.1.
1175
1176 2014-12-15  Geoffrey Garen  <ggaren@apple.com>
1177
1178         Safari crashes when you set Malloc environment variables
1179         https://bugs.webkit.org/show_bug.cgi?id=139656
1180
1181         Reviewed by Michael Saboff.
1182
1183         I forgot to cover the realloc() case. Whoops. (OoPS?)
1184
1185         This time around, I ran the full MallocBench test suite in Malloc=1
1186         mode, and it passed.
1187
1188         * bmalloc/Allocator.cpp:
1189         (bmalloc::Allocator::reallocate):
1190         * bmalloc/Allocator.h: Pushed realloc() logic down into the allocator.
1191         It needs to be down there so that we can do the short-circuiting check
1192         for whether bmalloc is enabled first.
1193
1194         Also added the check.
1195
1196         * bmalloc/Cache.cpp:
1197         (bmalloc::Cache::scavenge):
1198         (bmalloc::Cache::Cache):
1199         (bmalloc::Cache::reallocateSlowCaseNullCache):
1200         * bmalloc/Cache.h:
1201         (bmalloc::Cache::deallocator):
1202         (bmalloc::Cache::reallocate): Ditto.
1203
1204         * bmalloc/bmalloc.h:
1205         (bmalloc::api::free):
1206         (bmalloc::api::realloc): Ditto.
1207
1208         (bmalloc::api::scavenge): Pushed this down into Cache to match the
1209         surrounding functions.
1210
1211 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
1212
1213         bmalloc should support system memory analysis tools (part 2)
1214         https://bugs.webkit.org/show_bug.cgi?id=139565
1215
1216         Reviewed by Mark Lam.
1217
1218         This patch actually queries the environment to see if memory analysis
1219         tools have been enabled.
1220
1221         * bmalloc/Deallocator.cpp:
1222         (bmalloc::Deallocator::scavenge): Don't process the object log if
1223         we've disabled bmalloc because it will be full of invalid nullptrs.
1224
1225         * bmalloc/Environment.cpp:
1226         (bmalloc::isMallocEnvironmentVariableSet): Test for the list of known
1227         Malloc debugging flags. I also added a plain "Malloc" catch-all for
1228         when you want to disable bmalloc without enabling any kind of funny
1229         business.
1230
1231         It would be slightly nicer just to iterate the list of environment
1232         variables and strstr them, but getenv is the more portable option,
1233         and performance here doesn't really matter.
1234
1235         (bmalloc::isLibgmallocEnabled): Test for the libgmalloc insertion
1236         environment variable.
1237
1238         (bmalloc::Environment::computeIsBmallocEnabled):
1239
1240 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
1241
1242         Try to fix the iOS simulator build.
1243
1244         #include the declaration of malloc / free.
1245
1246         * bmalloc/Allocator.cpp:
1247         * bmalloc/Deallocator.cpp:
1248
1249 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
1250
1251         Try to fix the build.
1252
1253         * bmalloc.xcodeproj/project.pbxproj: Marked a header exported.
1254
1255 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
1256
1257         bmalloc should support system memory analysis tools (part 1)
1258         https://bugs.webkit.org/show_bug.cgi?id=139559
1259
1260         Reviewed by Mark Lam.
1261
1262         This patch adds the hooks to disable bmalloc at runtime if certain
1263         environment variables are set, but doesn't actually read from the
1264         environment yet.
1265
1266         No performance change.
1267
1268         * bmalloc.xcodeproj/project.pbxproj: Added the Environment class, which
1269         we'll use to read environment variables and see if memory analysis tools
1270         have been enabled.
1271
1272         * bmalloc/Allocator.cpp:
1273         (bmalloc::Allocator::Allocator):
1274         (bmalloc::Allocator::allocateSlowCase): Added a hook to disable bmalloc
1275         on the allocation path. We cache the setting to make the check fast.
1276
1277         * bmalloc/Allocator.h: Interface changes.
1278
1279         * bmalloc/Cache.cpp:
1280         (bmalloc::Cache::Cache): Pass a heap pointer through to our allocator
1281         and deallocator. This main purpose is to enable them to query the
1282         environment for whether bmalloc is enabled; but this is also a slightly
1283         cleaner way to guarantee to them that the Heap has been pre-initialized.
1284
1285         * bmalloc/Deallocator.cpp:
1286         (bmalloc::Deallocator::Deallocator): If bmalloc is disable, artificially
1287         fill the object log to force us to take the slow path on all deallocations.
1288
1289         (bmalloc::Deallocator::deallocateSlowCase): Do the disabled check.
1290
1291         * bmalloc/Deallocator.h: Interface changes.
1292
1293         * bmalloc/Environment.cpp: Added.
1294         (bmalloc::Environment::Environment):
1295         (bmalloc::Environment::computeIsBmallocEnabled):
1296         * bmalloc/Environment.h: Added.
1297         (bmalloc::Environment::isBmallocEnabled): This is the class that will
1298         encapsulate looking for environment variables that turn on heap
1299         analysis tools.
1300
1301         * bmalloc/Heap.h:
1302         (bmalloc::Heap::environment):
1303
1304         * bmalloc/Mutex.h:
1305         (bmalloc::Mutex::Mutex):
1306         * bmalloc/StaticMutex.h: A little refactoring to clarify these comments,
1307         since I got super confused about them while writing this patch.
1308
1309         * bmalloc/VMHeap.cpp: Fixed an #include.
1310
1311 2014-12-09  David Kilzer  <ddkilzer@apple.com>
1312
1313         Switch from using PLATFORM_NAME to SDK selectors in ANGLE, bmalloc, gtest, JavaScriptCore, WTF
1314         <http://webkit.org/b/139212>
1315
1316         Reviewed by Joseph Pecoraro.
1317
1318         * Configurations/Base.xcconfig:
1319         - Only set GCC_ENABLE_OBJC_GC, GCC_MODEL_TUNING and TOOLCHAINS
1320           on OS X.
1321         * Configurations/DebugRelease.xcconfig:
1322         - Only set MACOSX_DEPLOYMENT_TARGET and SDKROOT on OS X.
1323
1324 2014-11-07  Geoffrey Garen  <ggaren@apple.com>
1325
1326         bmalloc uses 8X more virtual memory than necessary
1327         https://bugs.webkit.org/show_bug.cgi?id=138495
1328
1329         Reviewed by Mark Lam.
1330
1331         iOS has a per-process virtual memory cap around 1GB, so there's some
1332         value to not going totally ham with virtual memory.
1333
1334         We currently use about 8X the necessary amount:
1335             - 2X to align our VM allocation
1336             - 4X to reserve small / medium / (2) large chunk VM ranges per superchunk
1337
1338         We can cut that down:
1339             - Return the unaligned portion of our VM allocation (-2X)
1340             - Use all the chunks in a superchunk, instead of allocating one
1341               chunk per superchunk (-4X)
1342
1343         * bmalloc/Algorithm.h:
1344         (bmalloc::roundUpToMultipleOf): Added a non-constant version of this
1345         function so we can call it with getpagesize() at runtime.
1346
1347         * bmalloc/Chunk.h:
1348         * bmalloc/LargeChunk.h:
1349         (bmalloc::LargeChunk::create): Deleted. Instead of each chunk allocating
1350         its own VM, VMHeap allocates the superchunk and all the chunks in it at a time.
1351
1352         * bmalloc/VMAllocate.h:
1353         (bmalloc::vmValidate):
1354         (bmalloc::vmAllocate): ASSERT that mmap succeeds to make crashes clearer
1355         if it does not succeed. Allocate precisely, and give back the extra.
1356
1357         * bmalloc/VMHeap.cpp:
1358         (bmalloc::VMHeap::allocateSuperChunk):
1359         (bmalloc::VMHeap::allocateSmallChunk): Deleted.
1360         (bmalloc::VMHeap::allocateMediumChunk): Deleted.
1361         (bmalloc::VMHeap::allocateLargeChunk): Deleted. Use all the chunks
1362         in a superchunk, instead of just one.
1363
1364         * bmalloc/VMHeap.h:
1365         (bmalloc::VMHeap::allocateSmallPage):
1366         (bmalloc::VMHeap::allocateMediumPage):
1367         (bmalloc::VMHeap::allocateLargeRange):
1368         * bmalloc/XLargeChunk.h:
1369         (bmalloc::XLargeChunk::create): Updated to match changes above.
1370
1371 2014-11-01  David Kilzer  <ddkilzer@apple.com>
1372
1373         JavaScriptCore is missing debug info for bmalloc because libbmalloc.a is stripped
1374         <https://webkit.org/b/138286>
1375         <rdar://problem/18847087>
1376
1377         Reviewed by Dan Bernstein.
1378
1379         * Configurations/bmalloc.xcconfig: Set STRIP_INSTALLED_PRODUCT
1380         to NO for the target that produces libbmalloc.a so that the
1381         debug symbols will be linked into JavaScriptCore and end up in
1382         its dSYM file.
1383
1384 2014-10-30  Dana Burkart  <dburkart@apple.com>
1385
1386         <rdar://problem/18821260> Prepare for the mysterious future
1387
1388         Reviewed by Lucas Forschler.
1389
1390         * Configurations/Base.xcconfig:
1391         * Configurations/DebugRelease.xcconfig:
1392
1393 2014-09-24  Geoffrey Garen  <ggaren@apple.com>
1394
1395         bmalloc: cleaned up fast path vs slow path
1396         https://bugs.webkit.org/show_bug.cgi?id=137081
1397
1398         Reviewed by Sam Weinig.
1399
1400         Might be a 1% speedup on MallocBench. Also cleans up the code a bit.
1401
1402         * bmalloc/Allocator.cpp:
1403         (bmalloc::Allocator::Allocator): Merged the small and medium range
1404         caches, just like the small and medium allocators. Ranges are abstract
1405         objects that don't really care whether they hold small or medium objects,
1406         so they don't need to be segregated.
1407
1408         (bmalloc::Allocator::scavenge): Ditto.
1409
1410         (bmalloc::Allocator::allocateBumpRangeSlowCase):
1411         (bmalloc::Allocator::allocateBumpRange): Same thing here, except that
1412         we do care a tiny bit, because we need to specify small vs medium when
1413         allocating new ranges from the heap, to ensure that the heap allocates
1414         from the right segment of VM.
1415
1416         (bmalloc::Allocator::allocateLarge):
1417         (bmalloc::Allocator::allocateXLarge): NO_INLINE because this was clouding
1418         up the fast path. Large allocation performance is dominated by allocation
1419         logic and initialization, so inlining it doesn't help.
1420
1421         (bmalloc::Allocator::allocateSlowCase): Slow path got a bit cleaner since
1422         it doesn't need to distinguish small vs medium objects.
1423
1424         (bmalloc::Allocator::allocateSmallBumpRange): Deleted.
1425         (bmalloc::Allocator::allocateMediumBumpRange): Deleted.
1426
1427         * bmalloc/Allocator.h:
1428         * bmalloc/BumpRange.h:
1429
1430         * bmalloc/Cache.cpp:
1431         (bmalloc::Cache::allocateSlowCase): Deleted.
1432         (bmalloc::Cache::deallocateSlowCase): Deleted.
1433         * bmalloc/Cache.h:
1434         (bmalloc::Cache::allocate):
1435         (bmalloc::Cache::deallocate):
1436         (bmalloc::Cache::allocateFastCase): Deleted.
1437         (bmalloc::Cache::deallocateFastCase): Deleted. Removed the Cache slow
1438         paths. The downside to this change is that the fast path branches to two
1439         distinct failure cases instead of one. The upside is that the slow path
1440         doesn't need to re-read the segment register, which is not as cheap as a
1441         normal register, and it doesn't need to do an extra level of function 
1442         call. Seems to be worth it.
1443
1444         * bmalloc/Deallocator.h:
1445         * bmalloc/Heap.cpp:
1446         (bmalloc::Heap::refillSmallBumpRangeCache):
1447         (bmalloc::Heap::refillMediumBumpRangeCache):
1448         * bmalloc/Heap.h: Updated for interface changes.
1449
1450         * bmalloc/Sizes.h: The most ranges a cache will hold is the number of
1451         small lines in a page / 2, since any other free lines will coalesce
1452         with their neighbors.
1453
1454 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
1455
1456         Rolled out r173346.
1457
1458             bmalloc should honor the FastMalloc statistics API
1459             https://bugs.webkit.org/show_bug.cgi?id=136592
1460
1461         This didn't really work. Because we allow ranges with and without
1462         physical pages to merge, and we allow double-committing and
1463         double-decommitting, we can't rely on commit actions to track memory
1464         footprint.
1465
1466         * bmalloc/Heap.cpp:
1467         (bmalloc::Heap::size): Deleted.
1468         (bmalloc::Heap::capacity): Deleted.
1469         * bmalloc/Heap.h:
1470         * bmalloc/VMHeap.cpp:
1471         (bmalloc::VMHeap::VMHeap):
1472         (bmalloc::VMHeap::allocateSmallChunk):
1473         (bmalloc::VMHeap::allocateMediumChunk):
1474         (bmalloc::VMHeap::allocateLargeChunk):
1475         * bmalloc/VMHeap.h:
1476         (bmalloc::VMHeap::allocateSmallPage):
1477         (bmalloc::VMHeap::allocateMediumPage):
1478         (bmalloc::VMHeap::allocateLargeRange):
1479         (bmalloc::VMHeap::deallocateSmallPage):
1480         (bmalloc::VMHeap::deallocateMediumPage):
1481         (bmalloc::VMHeap::deallocateLargeRange):
1482         (bmalloc::VMHeap::size): Deleted.
1483         (bmalloc::VMHeap::capacity): Deleted.
1484         * bmalloc/bmalloc.h:
1485         (bmalloc::api::heapSize): Deleted.
1486         (bmalloc::api::heapCapacity): Deleted.
1487
1488 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
1489
1490         bmalloc: Allocation should be more precise
1491         https://bugs.webkit.org/show_bug.cgi?id=136993
1492
1493         Reviewed by Gavin Barraclough.
1494
1495         13% reduction in heap size on the MallocBench *_memory_warning benchmarks.
1496
1497         This patch teaches the allocator to merge adjacent free lines into a
1498         single allocatable range. This allows us to shrink the size of an
1499         individual line without increasing fragmentation or the rate of allocator
1500         slow paths.
1501
1502         We'll only take more slow paths when available memory is sparse, which
1503         is exactly when it's worth it. When available memory is dense, we'll
1504         take fewer slow paths.
1505
1506         * bmalloc.xcodeproj/project.pbxproj:
1507         * bmalloc/Algorithm.h:
1508         (bmalloc::divideRoundingUp):
1509
1510         * bmalloc/Allocator.cpp:
1511         (bmalloc::Allocator::Allocator): Updated for interface changes.
1512
1513         (bmalloc::Allocator::scavenge): Scavenge by object instead of by line.
1514         Now that we merge lines, it's not convenient to scavenge by line.
1515
1516         (bmalloc::Allocator::allocateSmallBumpRange):
1517         (bmalloc::Allocator::allocateMediumBumpRange): Allocate whole ranges
1518         instead of individual lines.
1519
1520         (bmalloc::Allocator::allocateSlowCase):
1521         (bmalloc::Allocator::allocateSmallLine): Deleted.
1522         (bmalloc::Allocator::allocateMediumLine): Deleted.
1523         (bmalloc::Allocator::allocateMedium): Deleted.
1524         * bmalloc/Allocator.h:
1525         (bmalloc::Allocator::allocateFastCase): Folded medium allocations
1526         into the standard fast path with small allocations. Since a BumpAllocator
1527         just allocates out of an arbitrary range, it doesn't need to distinguish
1528         between small and medium lines.
1529
1530         * bmalloc/BumpAllocator.h:
1531         (bmalloc::BumpAllocator::size):
1532         (bmalloc::BumpAllocator::BumpAllocator):
1533         (bmalloc::BumpAllocator::init):
1534         (bmalloc::BumpAllocator::refill):
1535         (bmalloc::BumpAllocator::line): Deleted. No need to track line information
1536         anymore: the heap just gives us a pointer and a pre-computed number of
1537         objects, and we allocate them.
1538
1539         * bmalloc/Deallocator.cpp:
1540         (bmalloc::Deallocator::processObjectLog): Updated for interface changes.
1541
1542         * bmalloc/Heap.cpp:
1543         (bmalloc::Heap::Heap):
1544         (bmalloc::Heap::initializeLineMetadata): Pre-compute precise metadata
1545         detailing where all objects will lie in memory. After we merge two lines,
1546         we might allocate an object that spans from one line to the next. This
1547         metadata details which bits of memory overlap in that way, and how they
1548         overlap.
1549
1550         (bmalloc::Heap::refillSmallBumpRangeCache):
1551         (bmalloc::Heap::refillMediumBumpRangeCache): Scan a whole page at a time,
1552         and merge adjacent free lines into BumpRanges.
1553
1554         (bmalloc::Heap::allocateSmallPage):
1555         (bmalloc::Heap::allocateMediumPage):
1556         (bmalloc::Heap::deallocateSmallLine):
1557         (bmalloc::Heap::deallocateMediumLine): Track pages rather than lines,
1558         since we scan for free memory a page at a time.
1559
1560         (bmalloc::Heap::allocateSmallLineSlowCase): Deleted.
1561         (bmalloc::Heap::allocateMediumLineSlowCase): Deleted. Folded into the
1562         fast path.
1563
1564         * bmalloc/Heap.h:
1565         (bmalloc::Heap::derefSmallLine):
1566         (bmalloc::Heap::derefMediumLine):
1567         (bmalloc::Heap::deallocateSmallLine): Deleted.
1568         (bmalloc::Heap::allocateSmallLine): Deleted.
1569         (bmalloc::Heap::deallocateMediumLine): Deleted.
1570         (bmalloc::Heap::allocateMediumLine): Deleted. Updated for interface changes.
1571
1572         * bmalloc/Line.h:
1573         (bmalloc::Line<Traits>::ref):
1574         (bmalloc::Line<Traits>::deref):
1575         (bmalloc::Line<Traits>::concurrentRef): Deleted. We don't pass a derefCount
1576         anymore, since we only ever deref by 1 now.
1577
1578         * bmalloc/MediumAllocator.h:
1579         (bmalloc::MediumAllocator::isNull): Deleted.
1580         (bmalloc::MediumAllocator::MediumAllocator): Deleted.
1581         (bmalloc::MediumAllocator::line): Deleted.
1582         (bmalloc::MediumAllocator::allocate): Deleted.
1583         (bmalloc::MediumAllocator::derefCount): Deleted.
1584         (bmalloc::MediumAllocator::refill): Deleted.
1585         (bmalloc::MediumAllocator::clear): Deleted. Deleted some code that's
1586         been dead for a while, since it doesn't build anymore with this patch.
1587
1588         * bmalloc/Page.h:
1589         (bmalloc::Page::sizeClass):
1590         (bmalloc::Page::setSizeClass):
1591         (bmalloc::Page::smallSizeClass): Deleted.
1592         (bmalloc::Page::setSmallSizeClass): Deleted. Renamed setSmallSizeClass
1593         to sizeClass, since we use it for medium sizes too.
1594
1595         * bmalloc/Sizes.h:
1596         (bmalloc::Sizes::sizeClass):
1597         (bmalloc::Sizes::objectSize): Shrank line sizes to save memory.
1598
1599         (bmalloc::Sizes::smallSizeClassFor): Deleted.
1600         (bmalloc::Sizes::mediumSizeClassFor): Deleted.
1601
1602         * bmalloc/bmalloc.h:
1603         (bmalloc::api::realloc): Now that we have precise objects sizes, realloc
1604         can be a bit more precise. It also has to be, since we can't guarantee
1605         that an object ends at the end of a line anymore.
1606
1607 2014-09-19  Daniel Bates  <dabates@apple.com>
1608
1609         Always assume internal SDK when building configuration Production
1610         https://bugs.webkit.org/show_bug.cgi?id=136925
1611         <rdar://problem/18362399>
1612
1613         Reviewed by Dan Bernstein.
1614
1615         * Configurations/Base.xcconfig:
1616
1617 2014-09-16  Geoffrey Garen  <ggaren@apple.com>
1618
1619         bmalloc: moved line caches from the deallocator to the allocator
1620         https://bugs.webkit.org/show_bug.cgi?id=136868
1621
1622         Reviewed by Gavin Barraclough.
1623
1624         I did this mostly as a simplification, to make it easier to change the
1625         allocation strategy.
1626
1627         No throughput change on MallocBench. Saves about 50kB.
1628
1629         Since the deallocator needs to lock the heap when freeing lines anyway,
1630         there isn't much benefit to giving the deallocator a local cache of
1631         deallocated lines.
1632
1633         We still give the allocator a local cache of lines because that does
1634         reduce the frequency at which it needs to lock the heap in order to
1635         acquire more lines.
1636
1637         * bmalloc/Allocator.cpp:
1638         (bmalloc::Allocator::scavenge):
1639         (bmalloc::Allocator::allocateSmallLine):
1640         (bmalloc::Allocator::allocateMediumLine):
1641         (bmalloc::Allocator::allocateMedium):
1642         (bmalloc::Allocator::allocateSlowCase):
1643         * bmalloc/Allocator.h:
1644         * bmalloc/Deallocator.cpp:
1645         (bmalloc::Deallocator::Deallocator):
1646         (bmalloc::Deallocator::scavenge):
1647         (bmalloc::Deallocator::processObjectLog):
1648         (bmalloc::Deallocator::deallocateSmallLine): Deleted.
1649         (bmalloc::Deallocator::allocateSmallLine): Deleted.
1650         (bmalloc::Deallocator::deallocateMediumLine): Deleted.
1651         (bmalloc::Deallocator::allocateMediumLine): Deleted.
1652         * bmalloc/Deallocator.h:
1653
1654         * bmalloc/Sizes.h:
1655         * bmalloc/VMAllocate.h: Took the opportunity to make the line cache size
1656         exactly one page in size. That's about what we were shooting for anyway,
1657         and it may make it easier to switch to per-page allocation in future.
1658
1659 2014-09-15  Geoffrey Garen  <ggaren@apple.com>
1660
1661         bmalloc: allocate small and medium objects using the same bump pointer class
1662         https://bugs.webkit.org/show_bug.cgi?id=136843
1663
1664         Reviewed by Gavin Barraclough.
1665
1666         4% speedup on MallocBench.
1667
1668         Now that medium-sized objects have dedicated per-size allocators, they
1669         don't need to use an arbitrary bump pointer allocator. This means that
1670         every allocator knows how many objects it will allocate from the start,
1671         and we don't need a post-processing step to adjust refcounts based on
1672         real allocation count.
1673
1674         * bmalloc.xcodeproj/project.pbxproj: Renamed SmallAllocator to BumpAllocator
1675         since it's used for small and medium objects now.
1676
1677         * bmalloc/Allocator.cpp:
1678         (bmalloc::Allocator::Allocator): Updated to use new interface.
1679         (bmalloc::Allocator::scavenge): To "retire" an allocator, we just need
1680         to make sure that we finish allocating all the objects in it.
1681
1682         (bmalloc::Allocator::allocateMedium):
1683         (bmalloc::Allocator::allocateSlowCase):
1684         (bmalloc::Allocator::retire): Deleted.
1685         (bmalloc::Allocator::processSmallAllocatorLog): Deleted.
1686         (bmalloc::Allocator::processMediumAllocatorLog): Deleted.
1687         * bmalloc/Allocator.h:
1688         (bmalloc::Allocator::allocateFastCase): Removed abstractions and data
1689         used to post-process an allocator based on how many objects it allocated.
1690
1691         * bmalloc/BumpAllocator.h: Copied from Source/bmalloc/bmalloc/SmallAllocator.h.
1692         (bmalloc::BumpAllocator::BumpAllocator):
1693         (bmalloc::BumpAllocator::init):
1694         (bmalloc::BumpAllocator::line):
1695         (bmalloc::BumpAllocator::validate):
1696         (bmalloc::BumpAllocator::allocate):
1697         (bmalloc::BumpAllocator::refill):
1698         (bmalloc::BumpAllocator::clear): Updated these functions to be agnostic
1699         about the kinds of lines they allocate into. In some cases, the line
1700         type must be provided as a template parameter by the caller.
1701
1702         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
1703         (bmalloc::SmallAllocator::line): Deleted.
1704         (bmalloc::SmallAllocator::allocate): Deleted.
1705         (bmalloc::SmallAllocator::objectCount): Deleted.
1706         (bmalloc::SmallAllocator::derefCount): Deleted.
1707         (bmalloc::SmallAllocator::refill): Deleted.
1708         (bmalloc::SmallAllocator::clear): Deleted.
1709
1710         * bmalloc/ObjectType.h:
1711         (bmalloc::isMedium):
1712
1713         * bmalloc/SmallAllocator.h:
1714         (bmalloc::SmallAllocator::isNull): Deleted.
1715         (bmalloc::SmallAllocator::canAllocate): Deleted.
1716         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
1717         (bmalloc::SmallAllocator::line): Deleted.
1718         (bmalloc::SmallAllocator::allocate): Deleted.
1719         (bmalloc::SmallAllocator::objectCount): Deleted.
1720         (bmalloc::SmallAllocator::derefCount): Deleted.
1721         (bmalloc::SmallAllocator::refill): Deleted.
1722         (bmalloc::SmallAllocator::clear): Deleted.
1723
1724 2014-09-12  Geoffrey Garen  <ggaren@apple.com>
1725
1726         Fixed a goof in bmalloc Vector sizing
1727         https://bugs.webkit.org/show_bug.cgi?id=136795
1728
1729         Reviewed by Gavin Barraclough and Sam Weinig.
1730
1731         We want our minimum vector to be page-sized since the OS will give us
1732         a page no matter what -- but we want that many bytes, and not enough
1733         bytes to store that many elements.
1734
1735         * bmalloc/Vector.h: Math is hard.
1736
1737 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
1738
1739         bmalloc should segregate medium-sized objects by line like it does for small-sized objects
1740         https://bugs.webkit.org/show_bug.cgi?id=136693
1741
1742         Reviewed by Gavin Barraclough.
1743
1744         4% reduction in heap size on the MallocBench *_memory_warning benchmarks.
1745
1746         No throughput change.
1747
1748         We keep an array of medium allocators, just like our array of small
1749         allocators.
1750
1751         In future, we can simplify the allocation fast path by merging the small
1752         and medium allocator arrays. For now, this is the simplest change that
1753         gets the win.
1754
1755         * bmalloc/Allocator.cpp:
1756         (bmalloc::Allocator::Allocator):
1757         (bmalloc::Allocator::scavenge):
1758         (bmalloc::Allocator::allocateMedium):
1759         * bmalloc/Allocator.h:
1760         * bmalloc/Sizes.h:
1761         (bmalloc::Sizes::mediumSizeClassFor):
1762
1763 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
1764
1765         Reviewed by Sam Weinig.
1766
1767         Renamed log => retire for clarity.
1768
1769         * bmalloc/Allocator.cpp:
1770         (bmalloc::Allocator::scavenge):
1771         (bmalloc::Allocator::retire):
1772         (bmalloc::Allocator::allocateMedium):
1773         (bmalloc::Allocator::allocateSlowCase):
1774         (bmalloc::Allocator::log): Deleted.
1775         * bmalloc/Allocator.h:
1776
1777 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
1778
1779         bmalloc: eager scavenge leaves behind a bogus allocator
1780         https://bugs.webkit.org/show_bug.cgi?id=136743
1781
1782         Reviewed by Sam Weinig.
1783
1784         Be sure to clear the allocator after logging it in the eager scavenge
1785         case, so that we don't later try to allocate out of the lines that we
1786         have thrown away.
1787
1788         We didn't need to do this previously because scavenge would only happen
1789         at thread exit time, after which no further allocation from the per-thread
1790         cache would take place.
1791
1792         * bmalloc/Allocator.cpp:
1793         (bmalloc::Allocator::scavenge):
1794         * bmalloc/MediumAllocator.h:
1795         (bmalloc::MediumAllocator::clear):
1796         * bmalloc/SmallAllocator.h:
1797         (bmalloc::SmallAllocator::clear):
1798
1799 2014-09-05  Geoffrey Garen  <ggaren@apple.com>
1800
1801         bmalloc should honor the FastMalloc statistics API
1802         https://bugs.webkit.org/show_bug.cgi?id=136592
1803
1804         Reviewed by Gavin Barraclough.
1805
1806         We do this by tracking "size" and "capacity" in the VM heap.
1807
1808         The VM heap's "capacity" is all the VM we ever allocated.
1809
1810         The VM heap's "size" the subset of VM currently held onto by the
1811         VM heap (and therefore not in use by the regular heap).
1812
1813         Somewhat ironically, reducing the process's memory footprint, increases
1814         the size of the VM heap, since the VM heap holds the pages that are
1815         purely virtual and not physical.
1816
1817         * bmalloc/Heap.cpp:
1818         (bmalloc::Heap::size):
1819         (bmalloc::Heap::capacity):
1820         * bmalloc/Heap.h:
1821         * bmalloc/VMHeap.cpp:
1822         (bmalloc::VMHeap::VMHeap):
1823         (bmalloc::VMHeap::allocateSmallChunk):
1824         (bmalloc::VMHeap::allocateMediumChunk):
1825         (bmalloc::VMHeap::allocateLargeChunk):
1826         * bmalloc/VMHeap.h:
1827         (bmalloc::VMHeap::size):
1828         (bmalloc::VMHeap::capacity):
1829         (bmalloc::VMHeap::allocateSmallPage):
1830         (bmalloc::VMHeap::allocateMediumPage):
1831         (bmalloc::VMHeap::allocateLargeRange):
1832         (bmalloc::VMHeap::deallocateSmallPage):
1833         (bmalloc::VMHeap::deallocateMediumPage):
1834         (bmalloc::VMHeap::deallocateLargeRange):
1835         * bmalloc/bmalloc.h:
1836         (bmalloc::api::heapSize):
1837         (bmalloc::api::heapCapacity):
1838
1839 2014-09-02  Geoffrey Garen  <ggaren@apple.com>
1840
1841         bmalloc crashes on the EWS bots (due to bad large object allocation)
1842         https://bugs.webkit.org/show_bug.cgi?id=136469
1843
1844         Reviewed by Andreas Kling.
1845
1846         It's possible to convince bmalloc to perform a bad large object allocation,
1847         through these steps:
1848
1849         (1) Insert object A into freelist F0.
1850
1851         (2) Split, merge and split again A's neighbors such that object B is
1852         inserted into freelist F0, with boundary tag and size equal to object A,
1853         but pointer not completely equal to object A. Put object B at the head of F0.
1854
1855         (3) Allocate some other object from F0, swapping its position in the
1856         freelist with object B, such that object A is now ahead of object B.
1857
1858         --> Now, the next allocation for size A/B will allocate object A, which
1859         has a slightly wrong idea about where the object actually begins.
1860         Immediately, you'll corrupt a little memory, and over time, you'll also
1861         corrupt boundary tag metadata.
1862
1863         The solution is to store the begin pointer in the boundary tag. Luckily,
1864         this doesn't make the tag any bigger, and it's not a noticeable slowdown
1865         on MallocBench.
1866
1867         * bmalloc/Algorithm.h:
1868         (bmalloc::rightShift):
1869         * bmalloc/BeginTag.h:
1870         (bmalloc::BeginTag::isInFreeList): This is the bug fix. Make sure to
1871         validate the start pointer when popping off the free list. Through a
1872         very uncommon set of steps, it is possible to have an item in the free
1873         list that is valid by all accounts except for its start pointer.
1874
1875         * bmalloc/BoundaryTag.h:
1876         (bmalloc::BoundaryTag::compactBegin):
1877         (bmalloc::BoundaryTag::setRange):
1878         (bmalloc::BoundaryTag::setSize): Deleted. Record a compact version of the
1879         start pointer. We don't need the whole pointer -- just the offset, in
1880         largeAlignment increments, into the relevant boundary tag bucket.
1881
1882         * bmalloc/BoundaryTagInlines.h:
1883         (bmalloc::validateNext):
1884         (bmalloc::BoundaryTag::init):
1885         (bmalloc::BoundaryTag::mergeLarge):
1886         (bmalloc::BoundaryTag::splitLarge):
1887         * bmalloc/SegregatedFreeList.cpp:
1888         (bmalloc::SegregatedFreeList::insert):
1889         (bmalloc::SegregatedFreeList::takeGreedy):
1890         (bmalloc::SegregatedFreeList::take): Provide the whole range instead of
1891         the size when establishing a boundary tag, as required by the new
1892         interface.
1893
1894         * bmalloc/Sizes.h:
1895
1896 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
1897
1898         Fixed a bmalloc crash seen on the EWS bot
1899         https://bugs.webkit.org/show_bug.cgi?id=135955
1900
1901         Reviewed by Andreas Kling.
1902
1903         * bmalloc/Syscall.h: Some CG APIs vm_copy their input buffers. If the
1904         input buffer is a malloc region, that region will get marked Copy-On-Write
1905         by the kernel. Calls to madvise() for COW regions fail and return EINVAL
1906         on older OS X's. In 10.10, they still fail, but they do not return
1907         EINVAL.
1908
1909         So, we can only ASSERT that our syscalls succeed starting with 10.10.
1910
1911 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
1912
1913         Fixed the bmalloc build
1914         https://bugs.webkit.org/show_bug.cgi?id=135953
1915
1916         Reviewed by Andreas Kling.
1917
1918         * bmalloc.xcodeproj/project.pbxproj: Marked a few headers as private.
1919         These headers are used, so they must be available outside the project.
1920
1921 2014-08-13  Daniel Bates  <dabates@apple.com>
1922
1923         Attempt to fix the build following <http://trac.webkit.org/changeset/172576>
1924         (https://bugs.webkit.org/show_bug.cgi?id=135895)
1925
1926         Substitute PerThreadStorage<T>::initSharedKeyIfNeeded() for initSharedKeyIfNeeded() in
1927         implementation of PerThread<T>::getFastCase().
1928
1929         * bmalloc/PerThread.h:
1930         (bmalloc::PerThread<T>::getFastCase):
1931
1932 2014-08-13  Daniel Bates  <dabates@apple.com>
1933
1934         Make bmalloc::PerThread work without C++ thread local storage
1935         https://bugs.webkit.org/show_bug.cgi?id=135895
1936
1937         Reviewed by Geoffrey Garen.
1938
1939         Implement support for building bmalloc without C++ thread local storage.
1940
1941         * bmalloc/BPlatform.h: Remove macro define BPLATFORM_IOS_SIMULATOR. Added macro function
1942         BCOMPILER_SUPPORTS() and macro define BCOMPILER_SUPPORTS_CXX_THREAD_LOCAL that can be used
1943         to determine whether the compiler supports C++ thread local storage.
1944         * bmalloc/PerThread.h:
1945         (bmalloc::PerThreadStorage::get): Modified to call pthread_getspecific() when building
1946         without C++ thread local storage.
1947         (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Added.
1948         (bmalloc::PerThreadStorage::init): Moved logic to initialize shared Pthread key from here to
1949         PerThreadStorage::initSharedKeyIfNeeded().
1950         (bmalloc::PerThread<T>::getFastCase): Modified to call PerThreadStorage::initSharedKeyIfNeeded()
1951         before querying PerThreadStorage::get() when building without C++ thread local storage so as to
1952         ensure that the shared key has been initialized.
1953         (_pthread_setspecific_direct): Deleted.
1954         (_pthread_getspecific_direct): Deleted.
1955
1956 2014-08-13  Daniel Bates  <dabates@apple.com>
1957
1958         [iOS] Make JavaScriptCore and bmalloc build with the public SDK
1959         https://bugs.webkit.org/show_bug.cgi?id=135848
1960
1961         Reviewed by Geoffrey Garen.
1962
1963         * bmalloc/BPlatform.h: Added macro BPLATFORM_IOS_SIMULATOR, which evaluates to true
1964         when building for the iOS Simulator.
1965         * bmalloc/PerThread.h: Use pthread_machdep.h code path when building for iOS Simulator
1966         using the public SDK.
1967         (_pthread_setspecific_direct): Added; only defined when building for the iOS Simulator
1968         using the public SDK.
1969         (_pthread_getspecific_direct): Added; only defined when building for the iOS Simulator
1970         using the public SDK.
1971
1972 2014-08-12  Daniel Bates  <dabates@apple.com>
1973
1974         BPLATFORM(IOS) always evaluates to false
1975         https://bugs.webkit.org/show_bug.cgi?id=135843
1976
1977         Reviewed by Geoffrey Garen.
1978
1979         Fix typo in definition of BPLATFORM() and include system header TargetConditionals.h
1980         (when building on an Apple platform) so that BPLATFORM(X) evaluates to true when
1981         building for platform X. In particular, so that BPLATFORM(IOS) evaluates to true when
1982         building for iOS.
1983
1984         As a side effect of this change, the change made in <http://trac.webkit.org/changeset/167289>
1985         will be honored and iOS will assume a VM page size of 16kB (again) instead of 4kB.
1986
1987         * bmalloc/BPlatform.h:
1988
1989 2014-08-11  Andy Estes  <aestes@apple.com>
1990
1991         [iOS] Get rid of iOS.xcconfig
1992         https://bugs.webkit.org/show_bug.cgi?id=135809
1993
1994         Reviewed by Joseph Pecoraro.
1995
1996         All iOS.xcconfig did was include AspenFamily.xcconfig, so there's no need for the indirection.
1997
1998         * Configurations/Base.xcconfig:
1999         * Configurations/iOS.xcconfig: Removed.
2000         * bmalloc.xcodeproj/project.pbxproj:
2001
2002 2014-05-01  Dan Bernstein  <mitz@apple.com>
2003
2004         Fixed production builds for the iOS Simulator.
2005         <rdar://problem/16792221>
2006
2007         * Configurations/bmalloc.xcconfig: Include INSTALL_PATH_PREFIX in
2008         PRIVATE_HEADERS_FOLDER_PATH when installing.
2009
2010 2014-04-20  Geoffrey Garen  <ggaren@apple.com>
2011
2012         bmalloc: Segregate pages by objects size
2013         https://bugs.webkit.org/show_bug.cgi?id=131909
2014
2015         Reviewed by Andreas Kling.
2016
2017         2% reduction in memory-at-end on the Membuster memory_warning benchmarks.
2018
2019         * bmalloc/Allocator.cpp:
2020         (bmalloc::Allocator::allocateSlowCase):
2021         * bmalloc/Allocator.h:
2022         (bmalloc::Allocator::allocateFastCase):
2023         (bmalloc::Allocator::smallAllocatorFor): Use the new shared helper
2024         function for size class calculation.
2025
2026         * bmalloc/Deallocator.cpp:
2027         (bmalloc::Deallocator::Deallocator):
2028         (bmalloc::Deallocator::scavenge):
2029         (bmalloc::Deallocator::deallocateSmallLine):
2030         (bmalloc::Deallocator::allocateSmallLine):
2031         * bmalloc/Deallocator.h: Keep a cache for every size class, since the
2032         cache can't be shared anymore.
2033
2034         * bmalloc/Heap.cpp:
2035         (bmalloc::Heap::allocateSmallLineSlowCase):
2036         * bmalloc/Heap.h:
2037         (bmalloc::Heap::deallocateSmallLine): Ditto.
2038
2039         (bmalloc::Heap::allocateSmallLine): Check size class in addition to
2040         page refcount when allocating a line because we might have deallocated
2041         the page and the recycled it for another size class.
2042
2043         (bmalloc::Heap::deallocateMediumLine):
2044         (bmalloc::Heap::allocateMediumLine):
2045         * bmalloc/Line.h:
2046         (bmalloc::Line::refCount):
2047         * bmalloc/Page.h:
2048         (bmalloc::Page::refCount):
2049         (bmalloc::Page::smallSizeClass):
2050         (bmalloc::Page::setSmallSizeClass):
2051         (bmalloc::Page<Traits>::refCount): Deleted.
2052         * bmalloc/Sizes.h:
2053         (bmalloc::Sizes::smallSizeClassFor): New shared API for computing
2054         an index into an array from a size.
2055
2056 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
2057
2058         bmalloc: Improved alignment in LargeChunk
2059         https://bugs.webkit.org/show_bug.cgi?id=131895
2060
2061         Reviewed by Andreas Kling.
2062
2063         * bmalloc/Chunk.h:
2064         * bmalloc/LargeChunk.h: Align to vmPageSize just like Chunk does.
2065         Technically, the previous alignment was harmless, but I would prefer,
2066         dear reader, not to have to explain the interlocking set of
2067         circumstances that made it so.
2068
2069 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
2070
2071         Rolled out r167502 because it caused a crash on the facebook benchmark.
2072
2073         Unreviewed.
2074
2075             bmalloc: Added an XSmall line size
2076             https://bugs.webkit.org/show_bug.cgi?id=131851
2077
2078             Reviewed by Sam Weinig.
2079
2080 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
2081
2082         bmalloc: Mutex should be harder to use wrong
2083         https://bugs.webkit.org/show_bug.cgi?id=131879
2084
2085         Reviewed by Andreas Kling.
2086
2087         Mutex now has a proper constructor, so you can't deadlock by forgetting
2088         to initialize it.
2089
2090         * bmalloc.xcodeproj/project.pbxproj:
2091         * bmalloc/Allocator.cpp:
2092         (bmalloc::Allocator::processXSmallAllocatorLog):
2093         (bmalloc::Allocator::processSmallAllocatorLog):
2094         (bmalloc::Allocator::processMediumAllocatorLog):
2095         (bmalloc::Allocator::allocateLarge):
2096         (bmalloc::Allocator::allocateXLarge): Global replace Mutex => StaticMutex,
2097         since the Heap mutex is a static.
2098
2099         * bmalloc/AsyncTask.h:
2100         (bmalloc::Function>::AsyncTask): Use Mutex, since we're not static. No
2101         need for explicit initialization anymore.
2102
2103         * bmalloc/Deallocator.cpp:
2104         (bmalloc::Deallocator::scavenge):
2105         (bmalloc::Deallocator::deallocateLarge):
2106         (bmalloc::Deallocator::deallocateXLarge):
2107         (bmalloc::Deallocator::processObjectLog):
2108         (bmalloc::Deallocator::deallocateSmallLine):
2109         (bmalloc::Deallocator::deallocateXSmallLine):
2110         (bmalloc::Deallocator::allocateSmallLine):
2111         (bmalloc::Deallocator::allocateXSmallLine):
2112         (bmalloc::Deallocator::deallocateMediumLine):
2113         (bmalloc::Deallocator::allocateMediumLine):
2114         * bmalloc/Deallocator.h:
2115         * bmalloc/Heap.cpp:
2116         (bmalloc::sleep):
2117         (bmalloc::Heap::Heap):
2118         (bmalloc::Heap::concurrentScavenge):
2119         (bmalloc::Heap::scavenge):
2120         (bmalloc::Heap::scavengeSmallPages):
2121         (bmalloc::Heap::scavengeXSmallPages):
2122         (bmalloc::Heap::scavengeMediumPages):
2123         (bmalloc::Heap::scavengeLargeRanges):
2124         (bmalloc::Heap::allocateXSmallLineSlowCase):
2125         (bmalloc::Heap::allocateSmallLineSlowCase):
2126         (bmalloc::Heap::allocateMediumLineSlowCase):
2127         (bmalloc::Heap::allocateXLarge):
2128         (bmalloc::Heap::deallocateXLarge):
2129         (bmalloc::Heap::allocateLarge):
2130         (bmalloc::Heap::deallocateLarge):
2131         * bmalloc/Heap.h:
2132         (bmalloc::Heap::deallocateXSmallLine):
2133         (bmalloc::Heap::allocateXSmallLine):
2134         (bmalloc::Heap::deallocateSmallLine):
2135         (bmalloc::Heap::allocateSmallLine):
2136         (bmalloc::Heap::deallocateMediumLine):
2137         (bmalloc::Heap::allocateMediumLine):
2138         * bmalloc/Line.h:
2139         (bmalloc::Line<Traits>::deref):
2140         * bmalloc/Mutex.cpp: Removed.
2141         * bmalloc/Mutex.h:
2142         (bmalloc::Mutex::Mutex):
2143         (bmalloc::Mutex::init): Deleted.
2144         (bmalloc::Mutex::try_lock): Deleted.
2145         (bmalloc::Mutex::lock): Deleted.
2146         (bmalloc::Mutex::unlock): Deleted.
2147         * bmalloc/Page.h:
2148         (bmalloc::Page<Traits>::ref):
2149         (bmalloc::Page<Traits>::deref):
2150         (bmalloc::Page<Traits>::refCount):
2151         * bmalloc/PerProcess.h:
2152         (bmalloc::PerProcess::mutex):
2153         (bmalloc::PerProcess<T>::getSlowCase):
2154         * bmalloc/StaticMutex.cpp: Added.
2155         (bmalloc::StaticMutex::lockSlowCase):
2156         * bmalloc/StaticMutex.h: Added.
2157         (bmalloc::StaticMutex::init):
2158         (bmalloc::StaticMutex::try_lock):
2159         (bmalloc::StaticMutex::lock):
2160         (bmalloc::StaticMutex::unlock):
2161         * bmalloc/VMHeap.h:
2162         (bmalloc::VMHeap::deallocateXSmallPage):
2163         (bmalloc::VMHeap::deallocateSmallPage):
2164         (bmalloc::VMHeap::deallocateMediumPage):
2165         (bmalloc::VMHeap::deallocateLargeRange):
2166         * bmalloc/bmalloc.h:
2167         (bmalloc::api::scavenge): Global replace Mutex => StaticMutex,
2168         since the Heap mutex is a static.
2169
2170 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
2171
2172         bmalloc: AsyncTask should use Mutex instead of std::mutex
2173         https://bugs.webkit.org/show_bug.cgi?id=131865
2174
2175         Reviewed by Gavin Barraclough.
2176
2177         std::mutex is so slow that it makes parallelizing simple tasks through
2178         AsyncTask a net regression. Mutex fixes this.
2179
2180         * bmalloc/AsyncTask.h:
2181         (bmalloc::Function>::AsyncTask):
2182         (bmalloc::Function>::join):
2183         (bmalloc::Function>::runSlowCase):
2184         (bmalloc::Function>::entryPoint):
2185         * bmalloc/Mutex.h:
2186         (bmalloc::Mutex::init):
2187
2188 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
2189
2190         bmalloc: Added an XSmall line size
2191         https://bugs.webkit.org/show_bug.cgi?id=131851
2192
2193         Reviewed by Sam Weinig.
2194
2195         Reduces malloc footprint on Membuster recordings by 10%.
2196
2197         This is a throughput regression, but we're still way ahead of TCMalloc.
2198         I have some ideas for how to recover the regression -- but I wanted to
2199         get this win in first.
2200
2201         Full set of benchmark results:
2202
2203                 bmalloc> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks --measure-heap nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
2204                                                                                                 
2205                                                                nopatch                      patch                                Δ
2206                 Peak Memory:
2207                     reddit_memory_warning                      7,896kB                    7,532kB                  ^ 1.05x smaller
2208                     flickr_memory_warning                     12,968kB                   12,324kB                  ^ 1.05x smaller
2209                     theverge_memory_warning                   16,672kB                   15,200kB                   ^ 1.1x smaller
2210
2211                     <geometric mean>                          11,952kB                   11,216kB                  ^ 1.07x smaller
2212                     <arithmetic mean>                         12,512kB                   11,685kB                  ^ 1.07x smaller
2213                     <harmonic mean>                           11,375kB                   10,726kB                  ^ 1.06x smaller
2214
2215                 Memory at End:
2216                     reddit_memory_warning                      7,320kB                    6,856kB                  ^ 1.07x smaller
2217                     flickr_memory_warning                     10,848kB                    9,692kB                  ^ 1.12x smaller
2218                     theverge_memory_warning                   16,380kB                   14,872kB                   ^ 1.1x smaller
2219
2220                     <geometric mean>                          10,916kB                    9,961kB                   ^ 1.1x smaller
2221                     <arithmetic mean>                         11,516kB                   10,473kB                   ^ 1.1x smaller
2222                     <harmonic mean>                           10,350kB                    9,485kB                  ^ 1.09x smaller
2223
2224                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
2225                                                                                                 
2226                                                    nopatch                patch                         Δ
2227                 Execution Time:
2228                     churn                            127ms                151ms            ! 1.19x slower
2229                     list_allocate                    130ms                164ms            ! 1.26x slower
2230                     tree_allocate                    109ms                127ms            ! 1.17x slower
2231                     tree_churn                       115ms                120ms            ! 1.04x slower
2232                     facebook                         240ms                259ms            ! 1.08x slower
2233                     fragment                          91ms                131ms            ! 1.44x slower
2234                     fragment_iterate                 105ms                106ms            ! 1.01x slower
2235                     message_one                      260ms                259ms             ^ 1.0x faster
2236                     message_many                     149ms                154ms            ! 1.03x slower
2237                     medium                           194ms                248ms            ! 1.28x slower
2238                     big                              157ms                160ms            ! 1.02x slower
2239
2240                     <geometric mean>                 144ms                163ms            ! 1.13x slower
2241                     <arithmetic mean>                152ms                171ms            ! 1.12x slower
2242                     <harmonic mean>                  137ms                156ms            ! 1.14x slower
2243
2244                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
2245                                                                                                 
2246                                                                        nopatch                          patch                                     Δ
2247                 Execution Time:
2248                     churn                                                126ms                          148ms                        ! 1.17x slower
2249                     churn --parallel                                      62ms                           76ms                        ! 1.23x slower
2250                     list_allocate                                        130ms                          164ms                        ! 1.26x slower
2251                     list_allocate --parallel                             120ms                          175ms                        ! 1.46x slower
2252                     tree_allocate                                        111ms                          127ms                        ! 1.14x slower
2253                     tree_allocate --parallel                              95ms                          135ms                        ! 1.42x slower
2254                     tree_churn                                           115ms                          124ms                        ! 1.08x slower
2255                     tree_churn --parallel                                107ms                          126ms                        ! 1.18x slower
2256                     facebook                                             240ms                          276ms                        ! 1.15x slower
2257                     facebook --parallel                                  802ms                        1,088ms                        ! 1.36x slower
2258                     fragment                                              92ms                          130ms                        ! 1.41x slower
2259                     fragment --parallel                                   66ms                          124ms                        ! 1.88x slower
2260                     fragment_iterate                                     109ms                          127ms                        ! 1.17x slower
2261                     fragment_iterate --parallel                           55ms                           64ms                        ! 1.16x slower
2262                     message_one                                          260ms                          260ms                                      
2263                     message_many                                         170ms                          238ms                         ! 1.4x slower
2264                     medium                                               185ms                          250ms                        ! 1.35x slower
2265                     medium --parallel                                    210ms                          334ms                        ! 1.59x slower
2266                     big                                                  150ms                          169ms                        ! 1.13x slower
2267                     big --parallel                                       138ms                          144ms                        ! 1.04x slower
2268
2269                     <geometric mean>                                     135ms                          170ms                        ! 1.26x slower
2270                     <arithmetic mean>                                    167ms                          214ms                        ! 1.28x slower
2271                     <harmonic mean>                                      117ms                          148ms                        ! 1.26x slower
2272
2273                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
2274
2275                                                                     TC                      patch                                Δ
2276                 Peak Memory:
2277                     reddit_memory_warning                     13,836kB                   13,436kB                  ^ 1.03x smaller
2278                     flickr_memory_warning                     24,868kB                   25,188kB                   ! 1.01x bigger
2279                     theverge_memory_warning                   24,504kB                   26,636kB                   ! 1.09x bigger
2280
2281                     <geometric mean>                          20,353kB                   20,812kB                   ! 1.02x bigger
2282                     <arithmetic mean>                         21,069kB                   21,753kB                   ! 1.03x bigger
2283                     <harmonic mean>                           19,570kB                   19,780kB                   ! 1.01x bigger
2284
2285                 Memory at End:
2286                     reddit_memory_warning                      8,656kB                   10,016kB                   ! 1.16x bigger
2287                     flickr_memory_warning                     11,844kB                   13,784kB                   ! 1.16x bigger
2288                     theverge_memory_warning                   18,516kB                   22,748kB                   ! 1.23x bigger
2289
2290                     <geometric mean>                          12,382kB                   14,644kB                   ! 1.18x bigger
2291                     <arithmetic mean>                         13,005kB                   15,516kB                   ! 1.19x bigger
2292                     <harmonic mean>                           11,813kB                   13,867kB                   ! 1.17x bigger
2293
2294                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
2295                                                                                                 
2296                                                         TC                patch                         Δ
2297                 Execution Time:
2298                     churn                            416ms                148ms            ^ 2.81x faster
2299                     list_allocate                    463ms                164ms            ^ 2.82x faster
2300                     tree_allocate                    292ms                127ms             ^ 2.3x faster
2301                     tree_churn                       157ms                120ms            ^ 1.31x faster
2302                     facebook                         327ms                276ms            ^ 1.18x faster
2303                     fragment                         335ms                129ms             ^ 2.6x faster
2304                     fragment_iterate                 344ms                108ms            ^ 3.19x faster
2305                     message_one                      386ms                258ms             ^ 1.5x faster
2306                     message_many                     410ms                154ms            ^ 2.66x faster
2307                     medium                           391ms                245ms             ^ 1.6x faster
2308                     big                              261ms                167ms            ^ 1.56x faster
2309
2310                     <geometric mean>                 332ms                164ms            ^ 2.02x faster
2311                     <arithmetic mean>                344ms                172ms            ^ 1.99x faster
2312                     <harmonic mean>                  317ms                157ms            ^ 2.02x faster
2313
2314         * bmalloc.xcodeproj/project.pbxproj:
2315         * bmalloc/Allocator.cpp:
2316         (bmalloc::Allocator::Allocator): Don't assume that each allocator's
2317         index corresponds with its size. Instead, use the size selection function
2318         explicitly. Now that we have XSmall, some small allocator entries are
2319         unused.
2320
2321         (bmalloc::Allocator::scavenge):
2322         (bmalloc::Allocator::log):
2323         (bmalloc::Allocator::processXSmallAllocatorLog):
2324         (bmalloc::Allocator::allocateSlowCase):
2325         * bmalloc/Allocator.h:
2326         (bmalloc::Allocator::xSmallAllocatorFor):
2327         (bmalloc::Allocator::allocateFastCase):
2328         * bmalloc/Chunk.h:
2329         * bmalloc/Deallocator.cpp:
2330         (bmalloc::Deallocator::scavenge):
2331         (bmalloc::Deallocator::processObjectLog):
2332         (bmalloc::Deallocator::deallocateSlowCase):
2333         (bmalloc::Deallocator::deallocateXSmallLine):
2334         (bmalloc::Deallocator::allocateXSmallLine):
2335         * bmalloc/Deallocator.h:
2336         (bmalloc::Deallocator::deallocateFastCase):
2337         * bmalloc/Heap.cpp:
2338         (bmalloc::Heap::scavenge):
2339         (bmalloc::Heap::scavengeXSmallPages):
2340         (bmalloc::Heap::allocateXSmallLineSlowCase):
2341         * bmalloc/Heap.h:
2342         (bmalloc::Heap::deallocateXSmallLine):
2343         (bmalloc::Heap::allocateXSmallLine):
2344         * bmalloc/LargeChunk.h:
2345         (bmalloc::LargeChunk::get):
2346         (bmalloc::LargeChunk::endTag):
2347         * bmalloc/Line.h:
2348         * bmalloc/MediumAllocator.h:
2349         (bmalloc::MediumAllocator::allocate):
2350         (bmalloc::MediumAllocator::refill):
2351         * bmalloc/ObjectType.cpp:
2352         (bmalloc::objectType):
2353         * bmalloc/ObjectType.h:
2354         (bmalloc::isXSmall):
2355         (bmalloc::isSmall):
2356         (bmalloc::isMedium):
2357         (bmalloc::isLarge):
2358         (bmalloc::isSmallOrMedium): Deleted.
2359         * bmalloc/SegregatedFreeList.h: I boiler-plate copied existing code for
2360         handling small objects. There's probably a reasonable way to share this
2361         code in the future -- I'll look into that once it's stopped changing.
2362
2363         * bmalloc/Sizes.h: Tweaked size classes to make Membuster happy. This
2364         is the main reason things got slower.
2365
2366         * bmalloc/SmallAllocator.h:
2367         (bmalloc::SmallAllocator::allocate):
2368         * bmalloc/SmallTraits.h:
2369         * bmalloc/VMHeap.cpp:
2370         (bmalloc::VMHeap::allocateXSmallChunk):
2371         * bmalloc/VMHeap.h:
2372         (bmalloc::VMHeap::allocateXSmallPage):
2373         (bmalloc::VMHeap::deallocateXSmallPage):
2374         * bmalloc/XSmallAllocator.h: Added.
2375         (bmalloc::XSmallAllocator::isNull):
2376         (bmalloc::XSmallAllocator::canAllocate):
2377         (bmalloc::XSmallAllocator::XSmallAllocator):
2378         (bmalloc::XSmallAllocator::line):
2379         (bmalloc::XSmallAllocator::allocate):
2380         (bmalloc::XSmallAllocator::objectCount):
2381         (bmalloc::XSmallAllocator::derefCount):
2382         (bmalloc::XSmallAllocator::refill):
2383         * bmalloc/XSmallChunk.h: Added.
2384         * bmalloc/XSmallLine.h: Added.
2385         * bmalloc/XSmallPage.h: Added.
2386         * bmalloc/XSmallTraits.h: Added.
2387         * bmalloc/bmalloc.h:
2388         (bmalloc::api::realloc): Boiler-plate copy, as above.
2389
2390 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
2391
2392         MallocBench should scavenge explicitly instead of waiting
2393         https://bugs.webkit.org/show_bug.cgi?id=131661
2394
2395         Reviewed by Andreas Kling.
2396
2397         Added explicit scavenge support to bmalloc. This isn't a memory win,
2398         since bmalloc's per-thread cache is so small. But it makes testing
2399         simpler.
2400
2401         * bmalloc/Allocator.cpp:
2402         (bmalloc::Allocator::~Allocator):
2403         (bmalloc::Allocator::scavenge):
2404         * bmalloc/Allocator.h:
2405         * bmalloc/Cache.cpp:
2406         (bmalloc::Cache::operator new):
2407         (bmalloc::Cache::operator delete):
2408         (bmalloc::Cache::Cache):
2409         (bmalloc::Cache::scavenge):
2410         * bmalloc/Cache.h:
2411         * bmalloc/Deallocator.cpp:
2412         (bmalloc::Deallocator::~Deallocator):
2413         (bmalloc::Deallocator::scavenge):
2414         * bmalloc/Deallocator.h: Factored existing scavenging code into helper
2415         functions, for reuse.
2416
2417         * bmalloc/Heap.cpp:
2418         (bmalloc::sleep):
2419         (bmalloc::Heap::concurrentScavenge):
2420         (bmalloc::Heap::scavenge):
2421         (bmalloc::Heap::scavengeSmallPages):
2422         (bmalloc::Heap::scavengeMediumPages):
2423         (bmalloc::Heap::scavengeLargeRanges):
2424         * bmalloc/Heap.h: Made scavenge sleep duration a parameter. Forced
2425         scavenging -- in response to a benchmark or a low memory warning --
2426         wants to complete as soon as possible, so its sleep duration is 0.
2427
2428         * bmalloc/bmalloc.h:
2429         (bmalloc::api::scavenge):
2430         * bmalloc/mbmalloc.cpp: Exported the scavenge API for MallocBench's use.
2431
2432 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
2433
2434         Use 4kB pages on Mac
2435         https://bugs.webkit.org/show_bug.cgi?id=131658
2436
2437         Reviewed by Sam Weinig.
2438
2439         This reduces memory use a lot on Membuster:
2440
2441                                                                   base                      patch                                Δ
2442                 Execution Time:
2443                     reddit_memory_warning                         18ms                       17ms                   ^ 1.06x faster
2444                     flickr_memory_warning                         34ms                       36ms                   ! 1.06x slower
2445                     theverge_memory_warning                       39ms                       41ms                   ! 1.05x slower
2446
2447                     <geometric mean>                              29ms                       29ms                   ! 1.02x slower
2448                     <arithmetic mean>                             30ms                       31ms                   ! 1.03x slower
2449                     <harmonic mean>                               27ms                       27ms                    ^ 1.0x faster
2450
2451                 Peak Memory:
2452                     reddit_memory_warning                     16,412kB                   16,436kB                    ! 1.0x bigger
2453                     flickr_memory_warning                     30,120kB                   30,184kB                    ! 1.0x bigger
2454                     theverge_memory_warning                   33,408kB                   33,420kB                    ! 1.0x bigger
2455
2456                     <geometric mean>                          25,466kB                   25,499kB                    ! 1.0x bigger
2457                     <arithmetic mean>                         26,647kB                   26,680kB                    ! 1.0x bigger
2458                     <harmonic mean>                           24,181kB                   24,214kB                    ! 1.0x bigger
2459
2460                 Memory at End:
2461                     reddit_memory_warning                      2,404kB                    1,920kB                  ^ 1.25x smaller
2462                     flickr_memory_warning                      3,764kB                    3,072kB                  ^ 1.23x smaller
2463                     theverge_memory_warning                    3,648kB                    3,132kB                  ^ 1.16x smaller
2464
2465                     <geometric mean>                           3,208kB                    2,644kB                  ^ 1.21x smaller
2466                     <arithmetic mean>                          3,272kB                    2,708kB                  ^ 1.21x smaller
2467                     <harmonic mean>                            3,139kB                    2,574kB                  ^ 1.22x smaller
2468
2469
2470         * bmalloc.xcodeproj/project.pbxproj:
2471         * bmalloc/BPlatform.h: Added.
2472         * bmalloc/VMAllocate.h: Only use 16kB pages on iOS because the page size
2473         is 4kB on Mac.
2474
2475 2014-04-14  Alexey Proskuryakov  <ap@apple.com>
2476
2477         Fixed svn:ignore on bmalloc.xcodeproj, it had erroneous leading spaces.
2478
2479         * bmalloc.xcodeproj: Modified property svn:ignore.
2480
2481 2014-04-13  Geoffrey Garen  <ggaren@apple.com>
2482
2483         Fixed some mbmalloc exports
2484         https://bugs.webkit.org/show_bug.cgi?id=131599
2485
2486         Reviewed by Ryosuke Niwa.
2487
2488         * bmalloc.xcodeproj/project.pbxproj: Made some headers a private part
2489         of the project, so we can call them from API.
2490
2491         * bmalloc/mbmalloc.cpp: Marked the mbmalloc functions with default
2492         visibility, so they show up as exported in the .dylib.
2493
2494 2014-04-09  Geoffrey Garen  <ggaren@apple.com>
2495
2496         Put bmalloc headers in the right place
2497         https://bugs.webkit.org/show_bug.cgi?id=131464
2498
2499         Reviewed by Mark Rowe.
2500
2501         * Configurations/bmalloc.xcconfig: Set PRIVATE_HEADERS_FOLDER_PATH to
2502         specify that we don't just want to dump all of our generically-named
2503         headers into /usr/local/include.
2504
2505 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
2506
2507         Made bmalloc more #include friendly
2508         https://bugs.webkit.org/show_bug.cgi?id=131386
2509
2510         Reviewed by Andreas Kling.
2511
2512         Marked a bunch of headers private so they can be used from client code
2513         that #includes bmalloc.h.
2514
2515         Renamed ASSERT macros to BASSERT. This matches their header, which already
2516         had to be renamed, and fixes conflicts with WTF's ASSERT macros.
2517
2518         * bmalloc.xcodeproj/project.pbxproj:
2519         * bmalloc/Allocator.cpp:
2520         (bmalloc::Allocator::allocateSlowCase):
2521         * bmalloc/AsyncTask.h:
2522         (bmalloc::Function>::runSlowCase):
2523         * bmalloc/BAssert.h:
2524         * bmalloc/BoundaryTag.h:
2525         (bmalloc::BoundaryTag::setSize):
2526         * bmalloc/BoundaryTagInlines.h:
2527         (bmalloc::validate):
2528         (bmalloc::BoundaryTag::init):
2529         (bmalloc::BoundaryTag::deallocate):
2530         (bmalloc::BoundaryTag::splitLarge):
2531         (bmalloc::BoundaryTag::allocate):
2532         * bmalloc/Chunk.h:
2533         * bmalloc/Deallocator.cpp:
2534         (bmalloc::Deallocator::processObjectLog):
2535         (bmalloc::Deallocator::deallocateSlowCase):
2536         * bmalloc/Deallocator.h:
2537         (bmalloc::Deallocator::deallocateFastCase):
2538         * bmalloc/FixedVector.h:
2539         (bmalloc::Capacity>::operator):
2540         (bmalloc::Capacity>::push):
2541         (bmalloc::Capacity>::pop):
2542         (bmalloc::Capacity>::shrink):
2543         * bmalloc/Heap.cpp:
2544         (bmalloc::Heap::allocateLarge):
2545         * bmalloc/LargeChunk.h:
2546         (bmalloc::LargeChunk::get):
2547         (bmalloc::LargeChunk::endTag):
2548         * bmalloc/Line.h:
2549         (bmalloc::Line<Traits>::concurrentRef):
2550         (bmalloc::Line<Traits>::deref):
2551         * bmalloc/MediumAllocator.h:
2552         (bmalloc::MediumAllocator::allocate):
2553         * bmalloc/ObjectType.h:
2554         (bmalloc::isSmall):
2555         * bmalloc/Page.h:
2556         (bmalloc::Page<Traits>::ref):
2557         (bmalloc::Page<Traits>::deref):
2558         * bmalloc/PerThread.h:
2559         (bmalloc::PerThread<T>::getSlowCase):
2560         * bmalloc/SegregatedFreeList.cpp:
2561         (bmalloc::SegregatedFreeList::SegregatedFreeList):
2562         (bmalloc::SegregatedFreeList::insert):
2563         * bmalloc/SmallAllocator.h:
2564         (bmalloc::SmallAllocator::allocate):
2565         (bmalloc::SmallAllocator::refill):
2566         * bmalloc/Syscall.h:
2567         * bmalloc/VMAllocate.h:
2568         (bmalloc::vmValidate):
2569         (bmalloc::vmAllocate):
2570         (bmalloc::vmDeallocatePhysicalPagesSloppy):
2571         * bmalloc/Vector.h:
2572         (bmalloc::Vector<T>::operator):
2573         (bmalloc::Vector<T>::pop):
2574         (bmalloc::Vector<T>::shrink):
2575         * bmalloc/XLargeChunk.h:
2576         (bmalloc::XLargeChunk::range):
2577         (bmalloc::XLargeChunk::size):
2578
2579 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
2580
2581         Removed an unused file.
2582
2583         Unreviewed.
2584
2585         * bmalloc/AsyncTask.cpp: Removed.
2586
2587 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
2588
2589         Build bmalloc on Mac
2590         https://bugs.webkit.org/show_bug.cgi?id=131333
2591
2592         Reviewed by Mark Rowe.
2593
2594         * Makefile: Added. For make clients.
2595
2596         These files are required for building any project in WebKit. I copied
2597         them from WTF:
2598         * Configurations: Added.
2599         * Configurations/Base.xcconfig: Added.
2600         * Configurations/DebugRelease.xcconfig: Added.
2601         * Configurations/bmalloc.xcconfig: Added.
2602         * Configurations/iOS.xcconfig: Added.
2603         * Configurations/mbmalloc.xcconfig: Added.
2604
2605         * bmalloc.xcodeproj/project.pbxproj: I removed per-project-file stuff
2606         from here because everything is in .xcconfig files now.
2607
2608         I had to fix a bunch of minor warnings, since they're enabled in our
2609         .xcconfig files:
2610
2611         * bmalloc/AsyncTask.h:
2612         (bmalloc::Function>::AsyncTask):
2613         * bmalloc/BAssert.h:
2614         * bmalloc/BoundaryTagInlines.h:
2615         (bmalloc::validate):
2616         * bmalloc/Heap.cpp:
2617         (bmalloc::Heap::Heap):
2618         (bmalloc::Heap::allocateLarge):
2619         (bmalloc::Heap::deallocateLarge):
2620         * bmalloc/Mutex.h:
2621         (bmalloc::Mutex::Mutex): Deleted.
2622         * bmalloc/VMAllocate.h:
2623         (bmalloc::vmValidate):
2624         * bmalloc/mbmalloc.cpp:
2625
2626 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
2627
2628         bmalloc: Fixed a leak in the per-thread cache
2629         https://bugs.webkit.org/show_bug.cgi?id=131330
2630
2631         Reviewed by Andreas Kling.
2632
2633         Remember to deallocate our line caches upon thread exit.
2634
2635         * bmalloc/Deallocator.cpp:
2636         (bmalloc::Deallocator::~Deallocator):
2637
2638 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
2639
2640         bmalloc: rolled out the tryLock experiment
2641         https://bugs.webkit.org/show_bug.cgi?id=131328
2642
2643         Reviewed by Andreas Kling.
2644
2645         It wasn't a speedup.
2646
2647         * bmalloc.xcodeproj/project.pbxproj:
2648         * bmalloc/Allocator.cpp:
2649         (bmalloc::Allocator::processSmallAllocatorLog):
2650         (bmalloc::Allocator::processMediumAllocatorLog):
2651         * bmalloc/Deallocator.cpp:
2652         (bmalloc::Deallocator::processObjectLog):
2653         (bmalloc::Deallocator::deallocateSlowCase):
2654         (bmalloc::Deallocator::deallocateSmallLine):
2655         (bmalloc::Deallocator::deallocateMediumLine):
2656         * bmalloc/Deallocator.h:
2657         (bmalloc::Deallocator::deallocateFastCase):
2658         * bmalloc/Heap.h:
2659         (bmalloc::Heap::deallocateSmallLine):
2660         (bmalloc::Heap::deallocateMediumLine):
2661         * bmalloc/Line.h:
2662         (bmalloc::Line<Traits>::deref):
2663         * bmalloc/Page.h:
2664         (bmalloc::Page<Traits>::deref):
2665
2666 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
2667
2668         bmalloc
2669         https://bugs.webkit.org/show_bug.cgi?id=131170
2670
2671         Reviewed by Andreas Kling.
2672
2673         Initial commit.
2674
2675         * bmalloc: Added.
2676         * bmalloc.xcodeproj: Added.
2677         * bmalloc.xcodeproj/project.pbxproj: Added.
2678         * bmalloc/Algorithm.h: Added.
2679         (bmalloc::max):
2680         (bmalloc::min):
2681         (bmalloc::mask):
2682         (bmalloc::test):
2683         (bmalloc::roundUpToMultipleOf):
2684         (bmalloc::roundDownToMultipleOf):
2685         (bmalloc::sizeOf):
2686         (bmalloc::bitCount):
2687         (bmalloc::isPowerOfTwo):
2688         * bmalloc/Allocator.cpp: Added.
2689         (bmalloc::Allocator::Allocator):
2690         (bmalloc::Allocator::~Allocator):
2691         (bmalloc::Allocator::log):
2692         (bmalloc::Allocator::processSmallAllocatorLog):
2693         (bmalloc::Allocator::processMediumAllocatorLog):
2694         (bmalloc::Allocator::allocateLarge):
2695         (bmalloc::Allocator::allocateXLarge):
2696         (bmalloc::Allocator::allocateMedium):
2697         (bmalloc::Allocator::allocateSlowCase):
2698         * bmalloc/Allocator.h: Added.
2699         (bmalloc::Allocator::smallAllocatorFor):
2700         (bmalloc::Allocator::allocateFastCase):
2701         (bmalloc::Allocator::allocate):
2702         * bmalloc/AsyncTask.cpp: Added.
2703         (bmalloc::AsyncTask<Function>::runSlowCase):
2704         (bmalloc::AsyncTask<Function>::pthreadEntryPoint):
2705         (bmalloc::AsyncTask<Function>::entryPoint):
2706         * bmalloc/AsyncTask.h: Added.
2707         (bmalloc::Function>::AsyncTask):
2708         (bmalloc::Function>::join):
2709         (bmalloc::Function>::run):
2710         (bmalloc::Function>::runSlowCase):
2711         (bmalloc::Function>::pthreadEntryPoint):
2712         (bmalloc::Function>::entryPoint):
2713         * bmalloc/BAssert.h: Added.
2714         * bmalloc/BeginTag.h: Added.
2715         (bmalloc::BeginTag::isInFreeList):
2716         * bmalloc/BoundaryTag.h: Added.
2717         (bmalloc::BoundaryTag::isXLarge):
2718         (bmalloc::BoundaryTag::setXLarge):
2719         (bmalloc::BoundaryTag::isFree):
2720         (bmalloc::BoundaryTag::setFree):
2721         (bmalloc::BoundaryTag::isEnd):
2722         (bmalloc::BoundaryTag::setEnd):
2723         (bmalloc::BoundaryTag::hasPhysicalPages):
2724         (bmalloc::BoundaryTag::setHasPhysicalPages):
2725         (bmalloc::BoundaryTag::isNull):
2726         (bmalloc::BoundaryTag::clear):
2727         (bmalloc::BoundaryTag::size):
2728         (bmalloc::BoundaryTag::setSize):
2729         (bmalloc::BoundaryTag::prev):
2730         (bmalloc::BoundaryTag::next):
2731         * bmalloc/BoundaryTagInlines.h: Added.
2732         (bmalloc::validate):
2733         (bmalloc::validatePrev):
2734         (bmalloc::validateNext):
2735         (bmalloc::BoundaryTag::init):
2736         (bmalloc::BoundaryTag::mergeLargeLeft):
2737         (bmalloc::BoundaryTag::mergeLargeRight):
2738         (bmalloc::BoundaryTag::mergeLarge):
2739         (bmalloc::BoundaryTag::deallocate):
2740         (bmalloc::BoundaryTag::splitLarge):
2741         (bmalloc::BoundaryTag::allocate):
2742         * bmalloc/Cache.cpp: Added.
2743         (bmalloc::Cache::operator new):
2744         (bmalloc::Cache::operator delete):
2745         (bmalloc::Cache::Cache):
2746         (bmalloc::Cache::allocateSlowCase):
2747         (bmalloc::Cache::allocateSlowCaseNullCache):
2748         (bmalloc::Cache::deallocateSlowCase):
2749         (bmalloc::Cache::deallocateSlowCaseNullCache):
2750         * bmalloc/Cache.h: Added.
2751         (bmalloc::Cache::allocator):
2752         (bmalloc::Cache::deallocator):
2753         (bmalloc::Cache::allocateFastCase):
2754         (bmalloc::Cache::deallocateFastCase):
2755         (bmalloc::Cache::allocate):
2756         (bmalloc::Cache::deallocate):
2757         * bmalloc/Chunk.h: Added.
2758         (bmalloc::Chunk::begin):
2759         (bmalloc::Chunk::end):
2760         (bmalloc::Chunk::lines):
2761         (bmalloc::Chunk::pages):
2762         * bmalloc/Deallocator.cpp: Added.
2763         (bmalloc::Deallocator::Deallocator):
2764         (bmalloc::Deallocator::~Deallocator):
2765         (bmalloc::Deallocator::deallocateLarge):
2766         (bmalloc::Deallocator::deallocateXLarge):
2767         (bmalloc::Deallocator::processObjectLog):
2768         (bmalloc::Deallocator::deallocateSlowCase):
2769         (bmalloc::Deallocator::deallocateSmallLine):
2770         (bmalloc::Deallocator::allocateSmallLine):
2771         (bmalloc::Deallocator::deallocateMediumLine):
2772         (bmalloc::Deallocator::allocateMediumLine):
2773         * bmalloc/Deallocator.h: Added.
2774         (bmalloc::Deallocator::deallocateFastCase):
2775         (bmalloc::Deallocator::deallocate):
2776         * bmalloc/EndTag.h: Added.
2777         (bmalloc::EndTag::operator=):
2778         * bmalloc/FixedVector.h: Added.
2779         (bmalloc::FixedVector::begin):
2780         (bmalloc::FixedVector::end):
2781         (bmalloc::FixedVector::size):
2782         (bmalloc::FixedVector::capacity):
2783         (bmalloc::FixedVector::clear):
2784         (bmalloc::FixedVector::isEmpty):
2785         (bmalloc::Capacity>::FixedVector):
2786         (bmalloc::Capacity>::operator):
2787         (bmalloc::Capacity>::push):
2788         (bmalloc::Capacity>::pop):
2789         (bmalloc::Capacity>::shrink):
2790         * bmalloc/Heap.cpp: Added.
2791         (bmalloc::sleep):
2792         (bmalloc::Heap::Heap):
2793         (bmalloc::Heap::concurrentScavenge):
2794         (bmalloc::Heap::scavengeSmallPages):
2795         (bmalloc::Heap::scavengeMediumPages):
2796         (bmalloc::Heap::scavengeLargeRanges):
2797         (bmalloc::Heap::allocateSmallLineSlowCase):
2798         (bmalloc::Heap::allocateMediumLineSlowCase):
2799         (bmalloc::Heap::allocateXLarge):
2800         (bmalloc::Heap::deallocateXLarge):
2801         (bmalloc::Heap::allocateLarge):
2802         (bmalloc::Heap::deallocateLarge):
2803         * bmalloc/Heap.h: Added.
2804         (bmalloc::Heap::deallocateSmallLine):
2805         (bmalloc::Heap::allocateSmallLine):
2806         (bmalloc::Heap::deallocateMediumLine):
2807         (bmalloc::Heap::allocateMediumLine):
2808         * bmalloc/Inline.h: Added.
2809         * bmalloc/LargeChunk.h: Added.
2810         (bmalloc::LargeChunk::begin):
2811         (bmalloc::LargeChunk::end):
2812         (bmalloc::LargeChunk::create):
2813         (bmalloc::LargeChunk::get):
2814         (bmalloc::LargeChunk::beginTag):
2815         (bmalloc::LargeChunk::endTag):
2816         * bmalloc/Line.h: Added.
2817         (bmalloc::Line<Traits>::begin):
2818         (bmalloc::Line<Traits>::end):
2819         (bmalloc::Line<Traits>::concurrentRef):
2820         (bmalloc::Line<Traits>::deref):
2821         * bmalloc/MediumAllocator.h: Added.
2822         (bmalloc::MediumAllocator::isNull):
2823         (bmalloc::MediumAllocator::MediumAllocator):
2824         (bmalloc::MediumAllocator::line):
2825         (bmalloc::MediumAllocator::allocate):
2826         (bmalloc::MediumAllocator::derefCount):
2827         (bmalloc::MediumAllocator::refill):
2828         * bmalloc/MediumChunk.h: Added.
2829         * bmalloc/MediumLine.h: Added.
2830         * bmalloc/MediumPage.h: Added.
2831         * bmalloc/MediumTraits.h: Added.
2832         * bmalloc/Mutex.cpp: Added.
2833         (bmalloc::Mutex::lockSlowCase):
2834         * bmalloc/Mutex.h: Added.
2835         (bmalloc::Mutex::Mutex):
2836         (bmalloc::Mutex::try_lock):
2837         (bmalloc::Mutex::lock):
2838         (bmalloc::Mutex::unlock):
2839         * bmalloc/ObjectType.cpp: Added.
2840         (bmalloc::objectType):
2841         * bmalloc/ObjectType.h: Added.
2842         (bmalloc::isSmallOrMedium):
2843         (bmalloc::isSmall):
2844         * bmalloc/Page.h: Added.
2845         (bmalloc::Page<Traits>::ref):
2846         (bmalloc::Page<Traits>::deref):
2847         (bmalloc::Page<Traits>::refCount):
2848         * bmalloc/PerProcess.h: Added.
2849         (bmalloc::PerProcess::mutex):
2850         (bmalloc::PerProcess<T>::getFastCase):
2851         (bmalloc::PerProcess<T>::get):
2852         (bmalloc::PerProcess<T>::getSlowCase):
2853         * bmalloc/PerThread.h: Added.
2854         (bmalloc::PerThreadStorage<Cache>::get):
2855         (bmalloc::PerThreadStorage<Cache>::init):
2856         (bmalloc::PerThreadStorage::get):
2857         (bmalloc::PerThreadStorage::init):
2858         (bmalloc::PerThread<T>::getFastCase):
2859         (bmalloc::PerThread<T>::get):
2860         (bmalloc::PerThread<T>::destructor):
2861         (bmalloc::PerThread<T>::getSlowCase):
2862         * bmalloc/Range.h: Added.
2863         (bmalloc::Range::Range):
2864         (bmalloc::Range::begin):
2865         (bmalloc::Range::end):
2866         (bmalloc::Range::size):
2867         (bmalloc::Range::operator!):
2868         (bmalloc::Range::operator<):
2869         * bmalloc/SegregatedFreeList.cpp: Added.
2870         (bmalloc::SegregatedFreeList::SegregatedFreeList):
2871         (bmalloc::SegregatedFreeList::insert):
2872         (bmalloc::SegregatedFreeList::takeGreedy):
2873         (bmalloc::SegregatedFreeList::take):
2874         * bmalloc/SegregatedFreeList.h: Added.
2875         * bmalloc/Sizes.h: Added.
2876         * bmalloc/SmallAllocator.h: Added.
2877         (bmalloc::SmallAllocator::isNull):
2878         (bmalloc::SmallAllocator::canAllocate):
2879         (bmalloc::SmallAllocator::SmallAllocator):
2880         (bmalloc::SmallAllocator::line):
2881         (bmalloc::SmallAllocator::allocate):
2882         (bmalloc::SmallAllocator::objectCount):
2883         (bmalloc::SmallAllocator::derefCount):
2884         (bmalloc::SmallAllocator::refill):
2885         * bmalloc/SmallChunk.h: Added.
2886         * bmalloc/SmallLine.h: Added.
2887         * bmalloc/SmallPage.h: Added.
2888         * bmalloc/SmallTraits.h: Added.
2889         * bmalloc/Syscall.h: Added.
2890         * bmalloc/VMAllocate.h: Added.
2891         (bmalloc::vmSize):
2892         (bmalloc::vmValidate):
2893         (bmalloc::vmAllocate):
2894         (bmalloc::vmDeallocate):
2895         (bmalloc::vmDeallocatePhysicalPages):
2896         (bmalloc::vmAllocatePhysicalPages):
2897         (bmalloc::vmDeallocatePhysicalPagesSloppy):
2898         (bmalloc::vmAllocatePhysicalPagesSloppy):
2899         * bmalloc/VMHeap.cpp: Added.
2900         (bmalloc::VMHeap::VMHeap):
2901         (bmalloc::VMHeap::allocateSmallChunk):
2902         (bmalloc::VMHeap::allocateMediumChunk):
2903         (bmalloc::VMHeap::allocateLargeChunk):
2904         * bmalloc/VMHeap.h: Added.
2905         (bmalloc::VMHeap::allocateSmallPage):
2906         (bmalloc::VMHeap::allocateMediumPage):
2907         (bmalloc::VMHeap::allocateLargeRange):
2908         (bmalloc::VMHeap::deallocateSmallPage):
2909         (bmalloc::VMHeap::deallocateMediumPage):
2910         (bmalloc::VMHeap::deallocateLargeRange):
2911         * bmalloc/Vector.h: Added.
2912         (bmalloc::Vector::begin):
2913         (bmalloc::Vector::end):
2914         (bmalloc::Vector::size):
2915         (bmalloc::Vector::capacity):
2916         (bmalloc::Vector::last):
2917         (bmalloc::Vector::pop):
2918         (bmalloc::Vector<T>::Vector):
2919         (bmalloc::Vector<T>::~Vector):
2920         (bmalloc::Vector<T>::operator):
2921         (bmalloc::Vector<T>::push):
2922         (bmalloc::Vector<T>::pop):
2923         (bmalloc::Vector<T>::shrink):
2924         (bmalloc::Vector<T>::reallocateBuffer):
2925         (bmalloc::Vector<T>::shrinkCapacity):
2926         (bmalloc::Vector<T>::growCapacity):
2927         * bmalloc/XLargeChunk.h: Added.
2928         (bmalloc::XLargeChunk::get):
2929         (bmalloc::XLargeChunk::begin):
2930         (bmalloc::XLargeChunk::XLargeChunk):
2931         (bmalloc::XLargeChunk::create):
2932         (bmalloc::XLargeChunk::destroy):
2933         (bmalloc::XLargeChunk::range):
2934         (bmalloc::XLargeChunk::size):
2935         * bmalloc/bmalloc.h: Added.
2936         (bmalloc::api::malloc):
2937         (bmalloc::api::free):
2938         (bmalloc::api::realloc):
2939         * bmalloc/mbmalloc.cpp: Added.
2940