2015-02-26 Geoffrey Garen <ggaren@apple.com>
[WebKit-https.git] / Source / bmalloc / ChangeLog
1 2015-02-26  Geoffrey Garen  <ggaren@apple.com>
2
3         Try to fix the Mac build.
4
5         Unreviewed.
6
7         * bmalloc.xcodeproj/project.pbxproj: Make FreeList.h available.
8
9 2015-02-26  Geoffrey Garen  <ggaren@apple.com>
10
11         bmalloc: Refactored SegregatedFreeList and BoundaryTag::init
12         https://bugs.webkit.org/show_bug.cgi?id=142049
13
14         Reviewed by Anders Carlsson.
15
16         Split out a FreeList class from SegregatedFreeList. This will make it
17         easier to add behaviors on free list insertion and removal -- and it's
18         probably how I should have designed things at the start.
19
20         Moved BoundaryTag::init into LargeObject, since all the related logic
21         lives in LargeObject now too, and this allows us to remove BoundaryTagInlines.h.
22
23         * bmalloc.xcodeproj/project.pbxproj:
24         * bmalloc/BoundaryTagInlines.h: Removed.
25         * bmalloc/FreeList.cpp: Copied from Source/bmalloc/bmalloc/SegregatedFreeList.cpp.
26         (bmalloc::FreeList::takeGreedy):
27         (bmalloc::FreeList::take):
28         (bmalloc::SegregatedFreeList::SegregatedFreeList): Deleted.
29         (bmalloc::SegregatedFreeList::insert): Deleted.
30         (bmalloc::SegregatedFreeList::takeGreedy): Deleted.
31         (bmalloc::SegregatedFreeList::take): Deleted.
32         * bmalloc/FreeList.h: Copied from Source/bmalloc/bmalloc/SegregatedFreeList.h.
33         (bmalloc::FreeList::push):
34         * bmalloc/LargeObject.h:
35         (bmalloc::LargeObject::init):
36         * bmalloc/SegregatedFreeList.cpp:
37         (bmalloc::SegregatedFreeList::SegregatedFreeList):
38         (bmalloc::SegregatedFreeList::insert):
39         (bmalloc::SegregatedFreeList::takeGreedy):
40         (bmalloc::SegregatedFreeList::take):
41         * bmalloc/SegregatedFreeList.h:
42         * bmalloc/Sizes.h:
43         * bmalloc/VMHeap.cpp:
44         (bmalloc::VMHeap::grow):
45
46 2015-02-26  Geoffrey Garen  <ggaren@apple.com>
47
48         bmalloc: free up a bit in BoundaryTag
49         https://bugs.webkit.org/show_bug.cgi?id=142048
50
51         Reviewed by Brady Eidson.
52
53         We were wasting a bit by accident, and I need one now.
54
55         * bmalloc/Algorithm.h:
56         (bmalloc::rightShift): Deleted. Not needed, now that I've simplified
57         the math.
58
59         * bmalloc/BoundaryTag.h: Since each boundary tag bucket is 1024 bytes
60         long, the maximum offset into a bucket is 1023.
61
62         You need 5 bits to count up to 1024, but only 4 to count up to 1023.
63         
64         Math is hard.
65
66         (bmalloc::BoundaryTag::compactBegin): Switched to division because it
67         is simpler, and easier to match up with our ASSERT. The compiler will
68         turn division by constant power of two into a shift for us.
69
70         (bmalloc::BoundaryTag::setRange): Added an ASSERT for compactBegin
71         because we do encode it, so we should ASSERT that encoding did not
72         lose information.
73
74         * bmalloc/Sizes.h: Shifting is no longer used since we use division
75         instead.
76
77 2015-02-24  Stephanie Lewis  <slewis@apple.com>
78
79         Rolling out http://trac.webkit.org/changeset/180430 as it causes the PLT to crash.
80         <rdar://problem/19948015>
81
82         Unreviewed.
83
84         * bmalloc/VMHeap.cpp:
85         (bmalloc::VMHeap::grow):
86         * bmalloc/VMHeap.h:
87         * bmalloc/Zone.cpp:
88         (bmalloc::Zone::Zone):
89         (bmalloc::Zone::size): Deleted.
90         * bmalloc/Zone.h:
91
92 2015-02-24  Geoffrey Garen  <ggaren@apple.com>
93
94         bmalloc: Added a little more abstraction for large objects
95         https://bugs.webkit.org/show_bug.cgi?id=141978
96
97         Reviewed by Sam Weinig.
98
99         Previously, each client needed to manage the boundary tags of
100         a large object using free functions. This patch introduces a LargeObject
101         class that does things a little more automatically.
102
103         * bmalloc.xcodeproj/project.pbxproj:
104
105         * bmalloc/Allocator.cpp:
106         (bmalloc::Allocator::reallocate): Use the new LargeObject class.
107
108         * bmalloc/BeginTag.h:
109         (bmalloc::BeginTag::isInFreeList): Deleted. Moved this logic into the
110         LargeObject class.
111
112         * bmalloc/BoundaryTag.h:
113         (bmalloc::BoundaryTag::isSentinel):
114         (bmalloc::BoundaryTag::compactBegin):
115         (bmalloc::BoundaryTag::setRange):
116         (bmalloc::BoundaryTag::initSentinel): Added an explicit API for sentinels,
117         which we used to create and test for implicitly.
118
119         * bmalloc/BoundaryTagInlines.h:
120         (bmalloc::BoundaryTag::init):
121         (bmalloc::validate): Deleted.
122         (bmalloc::validatePrev): Deleted.
123         (bmalloc::validateNext): Deleted.
124         (bmalloc::BoundaryTag::mergeLeft): Deleted.
125         (bmalloc::BoundaryTag::mergeRight): Deleted.
126         (bmalloc::BoundaryTag::merge): Deleted.
127         (bmalloc::BoundaryTag::deallocate): Deleted.
128         (bmalloc::BoundaryTag::split): Deleted.
129         (bmalloc::BoundaryTag::allocate): Deleted. Moved this logic into the
130         LargeObject class.
131
132         * bmalloc/EndTag.h:
133         (bmalloc::EndTag::init):
134         (bmalloc::EndTag::operator=): Deleted. Re-reading this code, I found
135         special behavior in the assignment operator to be a surprising API.
136         So, I replaced the assignment operation with an explicit initializing
137         function.
138
139         * bmalloc/Heap.cpp:
140         (bmalloc::Heap::scavengeLargeRanges):
141         (bmalloc::Heap::allocateXLarge):
142         (bmalloc::Heap::findXLarge):
143         (bmalloc::Heap::deallocateXLarge):
144         (bmalloc::Heap::allocateLarge):
145         (bmalloc::Heap::deallocateLarge):
146         * bmalloc/Heap.h: No behavior changes here -- just adopting the
147         LargeObject interface.
148
149         * bmalloc/LargeObject.h: Added.
150         (bmalloc::LargeObject::operator!):
151         (bmalloc::LargeObject::begin):
152         (bmalloc::LargeObject::size):
153         (bmalloc::LargeObject::range):
154         (bmalloc::LargeObject::LargeObject):
155         (bmalloc::LargeObject::setFree):
156         (bmalloc::LargeObject::isFree):
157         (bmalloc::LargeObject::hasPhysicalPages):
158         (bmalloc::LargeObject::setHasPhysicalPages):
159         (bmalloc::LargeObject::isValidAndFree):
160         (bmalloc::LargeObject::merge):
161         (bmalloc::LargeObject::split):
162         (bmalloc::LargeObject::validateSelf):
163         (bmalloc::LargeObject::validate): Moved this code into a class, out of
164         BoundaryTag free functions.
165
166         New to the class are these features:
167
168             (1) Every reference to an object is validated upon creation and use.
169
170             (2) There's an explicit API for "This is a reference to an object
171             that might be stale (the DoNotValidate API)".
172
173             (3) The begin and end tags are kept in sync automatically.
174
175         * bmalloc/SegregatedFreeList.cpp:
176         (bmalloc::SegregatedFreeList::insert):
177         (bmalloc::SegregatedFreeList::takeGreedy):
178         (bmalloc::SegregatedFreeList::take):
179         * bmalloc/SegregatedFreeList.h: Adopt the LargeObject interface.
180
181         * bmalloc/VMHeap.cpp:
182         (bmalloc::VMHeap::grow):
183         * bmalloc/VMHeap.h:
184         (bmalloc::VMHeap::allocateLargeRange):
185         (bmalloc::VMHeap::deallocateLargeRange): Adopt the LargeObject interface.
186
187 2015-02-20  Geoffrey Garen  <ggaren@apple.com>
188
189         bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
190         https://bugs.webkit.org/show_bug.cgi?id=141802
191
192         Reviewed by Andreas Kling.
193
194         Rolling back in with a fix for a crash seen while using GuardMalloc.
195
196         * bmalloc/VMHeap.cpp:
197         (bmalloc::VMHeap::grow):
198         * bmalloc/VMHeap.h:
199         * bmalloc/Zone.cpp: Re-land the old patch.
200
201         (bmalloc::Zone::size): Be sure to implement the size() function since
202         it's accessible indirectly via the malloc_zone_from_ptr public API --
203         and GuardMalloc calls it all the time.
204
205         (bmalloc::Zone::Zone):
206         * bmalloc/Zone.h: Re-land the old patch.
207
208 2015-02-19  Commit Queue  <commit-queue@webkit.org>
209
210         Unreviewed, rolling out r180363.
211         https://bugs.webkit.org/show_bug.cgi?id=141814
212
213         Caused >50 crashes when running LayoutTests in GuardMalloc or
214         ASAN modes. (Requested by jernoble on #webkit).
215
216         Reverted changeset:
217
218         "bmalloc should implement malloc introspection (to stop false-
219         positive leaks when MallocStackLogging is off)"
220         https://bugs.webkit.org/show_bug.cgi?id=141802
221         http://trac.webkit.org/changeset/180363
222
223 2015-02-19  Geoffrey Garen  <ggaren@apple.com>
224
225         bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
226         https://bugs.webkit.org/show_bug.cgi?id=141802
227
228         Reviewed by Andreas Kling.
229
230         Fixed a last-minute type.
231
232         The macro is OS, not PLATFORM.
233
234         * bmalloc/VMHeap.cpp:
235         (bmalloc::VMHeap::grow):
236         * bmalloc/VMHeap.h:
237         * bmalloc/Zone.h:
238
239 2015-02-19  Geoffrey Garen  <ggaren@apple.com>
240
241         bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
242         https://bugs.webkit.org/show_bug.cgi?id=141802
243
244         Reviewed by Andreas Kling.
245
246         This patch does the bare minimum to stop false positive leaks from
247         being reported by the Darwin leaks tool. We register each super chunk
248         as a single object, and then request that the leaks tool scan it.
249
250         * bmalloc.xcodeproj/project.pbxproj: Added an abstraction for the malloc
251         zone introspection API.
252
253         * bmalloc/Algorithm.h: Missing #include.
254
255         * bmalloc/VMHeap.cpp:
256         (bmalloc::VMHeap::grow):
257         * bmalloc/VMHeap.h: Adopt the new abstraction.
258
259         * bmalloc/Zone.cpp: Added.
260         (bmalloc::remoteRead): Helper for reading an object out of another process.
261         (bmalloc::Zone::enumerator):
262         (bmalloc::Zone::Zone): Register a malloc zone so that we will participate
263         in introspection.
264
265         * bmalloc/Zone.h: Added.
266         (bmalloc::Zone::superChunks):
267         (bmalloc::Zone::addSuperChunk): Use a non-dynamically-allocated vector
268         since our dynamic allocations will not be scanned by leaks since they
269         will have the malloc VM tag.
270
271 2015-02-18  Geoffrey Garen  <ggaren@apple.com>
272
273         bmalloc: VMHeap should keep a record of all of its VM ranges (for malloc introspection)
274         https://bugs.webkit.org/show_bug.cgi?id=141759
275
276         Reviewed by Andreas Kling.
277
278         * bmalloc.xcodeproj/project.pbxproj:
279         * bmalloc/SuperChunk.h: Added.
280         (bmalloc::SuperChunk::create):
281         (bmalloc::SuperChunk::SuperChunk):
282         (bmalloc::SuperChunk::smallChunk):
283         (bmalloc::SuperChunk::mediumChunk):
284         (bmalloc::SuperChunk::largeChunk): Factored out super chunk creation
285         into a separate class, for clarity and type safety.
286
287         * bmalloc/VMHeap.cpp:
288         (bmalloc::VMHeap::grow):
289         (bmalloc::VMHeap::allocateSuperChunk): Renamed "allocateSuperChunk" to
290         "grow" because Andreas found "allocateSuperChunk" to be unclear.
291
292         * bmalloc/VMHeap.h: Track all our VM ranges. We will use this information
293         for malloc introspection.
294
295         (bmalloc::VMHeap::allocateSmallPage):
296         (bmalloc::VMHeap::allocateMediumPage):
297         (bmalloc::VMHeap::allocateLargeRange): Updated for renames.
298
299 2015-02-18  Zan Dobersek  <zdobersek@igalia.com>
300
301         Build bmalloc through CMake as a static library. It's then linked either
302         into the WTF library (if built as a shared library) or into the JSC and
303         WebKit2 libraries. There's no need to build it as a standalone shared library.
304
305         Rubber-stamped by Carlos Garcia Campos.
306
307         * CMakeLists.txt:
308
309 2015-02-13  Gyuyoung Kim  <gyuyoung.kim@samsung.com>
310
311         [BMalloc] Add a FIXME comment for memory alignas
312         https://bugs.webkit.org/show_bug.cgi?id=141556
313
314         Reviewed by Csaba Osztrogonác.
315
316         * bmalloc/Chunk.h: Add a FIXME comment.
317         * bmalloc/LargeChunk.h: ditto.
318
319 2015-02-11  Csaba Osztrogonác  <ossy@webkit.org>
320
321         bmalloc buildfix on 32 bit Linux (x86/ARM)
322         https://bugs.webkit.org/show_bug.cgi?id=141472
323
324         Reviewed by Gyuyoung Kim.
325
326         * bmalloc/Algorithm.h:
327         (bmalloc::roundUpToMultipleOf):
328         * bmalloc/FixedVector.h:
329         (bmalloc::FixedVector::clear):
330         * bmalloc/Sizes.h:
331         (bmalloc::Sizes::sizeClass):
332
333 2015-02-11  Gyuyoung Kim  <gyuyoung.kim@samsung.com>
334
335         [EFL][GTK] Use bmalloc instead of tcmalloc
336         https://bugs.webkit.org/show_bug.cgi?id=140162
337
338         Reviewed by Carlos Garcia Campos.
339
340         Support to use bmalloc on EFL and GTK ports.
341
342         * CMakeLists.txt: Added.
343         * bmalloc/Allocator.cpp:
344         (bmalloc::Allocator::allocate):
345             Fix unused return value caused by posix_memalign().
346         * bmalloc/AsyncTask.h:
347         * bmalloc/BoundaryTag.h:
348         (bmalloc::BoundaryTag::clear):
349         * bmalloc/Chunk.h:
350             Change Traits::Page with Traits::PageType in order to fix
351             -fpermitive build error on EFL and GTK port. 
352         * bmalloc/EndTag.h:
353         (bmalloc::EndTag::operator=):
354         * bmalloc/Line.h: ditto.
355         * bmalloc/MediumTraits.h:
356         * bmalloc/Page.h: ditto.
357         * bmalloc/PerThread.h:
358             EFL port doesn't support __has_include definition yet.
359             Define HAVE_PTHREAD_MACHDEP_H according to check if __has_include is supported.
360         * bmalloc/SmallTraits.h: ditto.
361         * bmalloc/VMAllocate.h:
362         (bmalloc::vmDeallocatePhysicalPages):
363         (bmalloc::vmAllocatePhysicalPages):
364         * bmalloc/Vector.h:
365         (bmalloc::Vector<T>::push):
366         (bmalloc::Vector<T>::reallocateBuffer):
367
368 2015-01-31  Sam Weinig  <sam@webkit.org>
369
370         Remove even more Mountain Lion support
371         https://bugs.webkit.org/show_bug.cgi?id=141124
372
373         Reviewed by Alexey Proskuryakov.
374
375         * Configurations/Base.xcconfig:
376         * Configurations/DebugRelease.xcconfig:
377
378 2015-01-30  Geoffrey Garen  <ggaren@apple.com>
379
380         GC marking threads should clear malloc caches
381         https://bugs.webkit.org/show_bug.cgi?id=141097
382
383         Reviewed by Andreas Kling.
384
385         Split the scavenging API into per-thread vs global, so that you can
386         request to scavenge your own thread without scavenging the whole heap.
387
388         * bmalloc/Cache.cpp:
389         (bmalloc::Cache::scavenge):
390         * bmalloc/bmalloc.h:
391         (bmalloc::api::scavengeThisThread):
392         (bmalloc::api::scavenge):
393
394 2015-01-28  Dana Burkart  <dburkart@apple.com>
395
396         Move ASan flag settings from DebugRelease.xcconfig to Base.xcconfig
397         https://bugs.webkit.org/show_bug.cgi?id=136765
398
399         Reviewed by Alexey Proskuryakov.
400
401         * Configurations/Base.xcconfig:
402         * Configurations/DebugRelease.xcconfig:
403
404 2015-01-21  Geoffrey Garen  <ggaren@apple.com>
405
406         bmalloc: support aligned allocation
407         https://bugs.webkit.org/show_bug.cgi?id=140732
408
409         Reviewed by Andreas Kling.
410
411         * bmalloc/Allocator.cpp:
412         (bmalloc::Allocator::allocate): New function for aligned allocation.
413
414         Small and medium requests just allocate and free until they find an
415         aligned pointer. This is slightly inefficient in the worst case, but
416         still constant-time with little-to-no space overhead.
417
418         Large requests use a new API that requires the client to specify both
419         its ideal size and alignment, and the worst-case size you would have to
420         allocate in order to produce some interior pointer of the requested size
421         and alignment. We put the burden of this calculation on the client
422         because it simplifies things if we guarantee that allocation won't fail.
423
424         XLarge requests are easy: we just forward them to vmAllocate, which
425         already supported aligned requests.
426
427         * bmalloc/BoundaryTag.h:
428         * bmalloc/BoundaryTagInlines.h:
429         (bmalloc::BoundaryTag::mergeLeft):
430         (bmalloc::BoundaryTag::mergeRight):
431         (bmalloc::BoundaryTag::merge):
432         (bmalloc::BoundaryTag::deallocate):
433         (bmalloc::BoundaryTag::split):
434         (bmalloc::BoundaryTag::allocate): No behavior change here. I just
435         refactored the interface to remove some reference out parameters in
436         order to clarify what changes and what doesn't.
437
438         * bmalloc/Heap.cpp:
439         (bmalloc::Heap::allocateXLarge): Added an alignment API.
440
441         (bmalloc::Heap::allocateLarge):
442         * bmalloc/Heap.h: Added an alignment API. I split out allocateLarge into
443         a few variants, so aligned and unaligned allocation could share some code.
444
445         * bmalloc/SegregatedFreeList.cpp:
446         (bmalloc::SegregatedFreeList::take):
447         * bmalloc/SegregatedFreeList.h: Changed to use a separate, explicit API
448         for aligned allocation. It turns out that the aligned path is pretty
449         different, since it ends up searching for two potential ways to satisfy
450         an allocation: either large enough and aligned, or large enough to split
451         into something not aligned and something large enough and aligned.
452
453         * bmalloc/VMAllocate.h:
454         (bmalloc::vmAllocate): Switched alignment to come before size because
455         that's how the memalign API specifies it.
456
457         * bmalloc/VMHeap.h:
458         (bmalloc::VMHeap::allocateLargeRange): Added an alignment API.
459
460 2015-01-20  Geoffrey Garen  <ggaren@apple.com>
461
462         bmalloc: a little bit of cleanup
463         https://bugs.webkit.org/show_bug.cgi?id=140687
464
465         Reviewed by Anders Carlsson.
466
467         * bmalloc/Algorithm.h:
468         (bmalloc::isPowerOfTwo): Added a check for 0, since 0 would break a lot
469         of code.
470
471         * bmalloc/BoundaryTag.h:
472         * bmalloc/BoundaryTagInlines.h:
473         (bmalloc::BoundaryTag::mergeLeft):
474         (bmalloc::BoundaryTag::mergeRight):
475         (bmalloc::BoundaryTag::merge):
476         (bmalloc::BoundaryTag::deallocate):
477         (bmalloc::BoundaryTag::split):
478         (bmalloc::BoundaryTag::allocate):
479         (bmalloc::BoundaryTag::mergeLargeLeft): Deleted.
480         (bmalloc::BoundaryTag::mergeLargeRight): Deleted.
481         (bmalloc::BoundaryTag::mergeLarge): Deleted.
482         (bmalloc::BoundaryTag::splitLarge): Deleted. Removed the word "Large"
483         from all these functions, since boundary tags always pertain to large
484         objects, and putting the word "Large" everywhere wasn't helping to
485         explain that.
486
487         * bmalloc/Heap.cpp:
488         (bmalloc::Heap::allocateXLarge):
489         (bmalloc::Heap::findXLarge):
490         (bmalloc::Heap::deallocateXLarge):
491         * bmalloc/Heap.h:
492         * bmalloc/VMHeap.h:
493         (bmalloc::VMHeap::allocateXLarge): Deleted.
494         (bmalloc::VMHeap::findXLarge): Deleted.
495         (bmalloc::VMHeap::deallocateXLarge): Deleted. Moved XLarge allocation
496         from VMHeap to Heap. Since the purpose of the VMHeap is to cache VM
497         ranges, and the VMHeap never caches any XLarge ranges, it doesn't
498         really make sense for the VMHeap to be involved.
499
500 2015-01-16  Geoffrey Garen  <ggaren@apple.com>
501
502         bmalloc: refactored XLarge allocation for better alignment
503         https://bugs.webkit.org/show_bug.cgi?id=140582
504
505         Reviewed by Andreas Kling.
506
507         XLarge objects used to be Large objects with an extra bit of metadata
508         that said "actually, I'm not large -- I'm extra large".
509
510         The metadata header in an XLarge allocation made it impossible for the
511         XLarge object to honor a very large alignment request.
512
513         The solution is to stop using a metadata header for XLarge objects, and
514         instead to store explicit metadata on the side.
515
516         This is a bit less astonishing, which is also nice.
517
518         Finding XLarge metadata is now a linear search. That's probably OK, since
519         it was always so in TCMalloc, and the usual number of XLarge allocations
520         in a process is 0.
521
522         This design makes it possible for the heap to cache XLarge allocations
523         with and/or without physical pages. I haven't actually done that yet
524         because the tradeoffs are subtle, so I don't want to do anything without
525         a motivating test case.
526
527         * bmalloc.xcodeproj/project.pbxproj:
528         * bmalloc/Allocator.cpp:
529         (bmalloc::Allocator::reallocate): Removed the concept of an XLargeChunk,
530         since an XLarge allocation is now just a naked buffer without a header.
531
532         (bmalloc::Allocator::allocateXLarge): Added an explicit qualifier for
533         XLarge alignment, since XLargeChunk won't give this to us implicitly
534         anymore.
535
536         * bmalloc/BoundaryTag.h:
537         (bmalloc::BoundaryTag::setRange):
538         (bmalloc::BoundaryTag::isXLarge): Deleted.
539         (bmalloc::BoundaryTag::setXLarge): Deleted.
540         * bmalloc/BoundaryTagInlines.h:
541         (bmalloc::validate):
542         (bmalloc::BoundaryTag::deallocate): Removed the XLarge hacks from Large allocations.
543
544         * bmalloc/Deallocator.cpp:
545         (bmalloc::Deallocator::deallocateXLarge):
546         (bmalloc::Deallocator::deallocateSlowCase):
547         * bmalloc/Heap.cpp:
548         (bmalloc::Heap::findXLarge):
549         (bmalloc::Heap::allocateXLarge):
550         (bmalloc::Heap::deallocateXLarge):
551         * bmalloc/Heap.h: Updated for interface changes.
552
553         * bmalloc/ObjectType.cpp:
554         (bmalloc::objectType):
555         * bmalloc/ObjectType.h:
556         (bmalloc::isXLarge): We can now tell if a pointer is XLarge just by
557         examining its bit pattern -- just like we do for other kinds of
558         allocations -- which is nice.
559
560         * bmalloc/Sizes.h:
561         * bmalloc/VMHeap.h:
562         (bmalloc::VMHeap::allocateXLarge):
563         (bmalloc::VMHeap::findXLarge):
564         (bmalloc::VMHeap::deallocateXLarge): Keep an explicit vector of metadata
565         for XLarge allocations.
566
567         * bmalloc/XLargeChunk.h: Removed.
568
569 2015-01-16  Geoffrey Garen  <ggaren@apple.com>
570
571         bmalloc: added some infrastructure for aligned allocation
572         https://bugs.webkit.org/show_bug.cgi?id=140572
573
574         Reviewed by Andreas Kling.
575
576         * bmalloc/Algorithm.h:
577         (bmalloc::isPowerOfTwo):
578         (bmalloc::roundUpToMultipleOf):
579         (bmalloc::roundDownToMultipleOf): Refactored some duplicate code to use our
580         isPowerOfTwo helper function.
581
582         * bmalloc/Allocator.cpp:
583         (bmalloc::Allocator::allocate):
584         * bmalloc/Allocator.h: Stubbed out an implementation of aligned allocation.
585         Doesn't do anything yet, but does correctly forward to system malloc
586         when bmalloc is disabled.
587
588         * bmalloc/Cache.cpp:
589         (bmalloc::Cache::allocateSlowCaseNullCache):
590         * bmalloc/Cache.h:
591         (bmalloc::Cache::allocate):
592         * bmalloc/bmalloc.h:
593         (bmalloc::api::memalign):
594         * bmalloc/mbmalloc.cpp: Stubbed out an API for aligned allocation.
595
596 2015-01-13  Geoffrey Garen  <ggaren@apple.com>
597
598         Consider alignment when allocating from a SegregatedFreeList
599         https://bugs.webkit.org/show_bug.cgi?id=140408
600
601         Reviewed by Sam Weinig.
602
603         In preparation for supporting aligned allocation.
604
605         No performance change.
606
607         Since this is just one extra branch in an already expensive function,
608         I decided not to duplicate the function just to avoid the branch in
609         the un-aligned case.
610
611         * bmalloc/SegregatedFreeList.cpp:
612         (bmalloc::SegregatedFreeList::take):
613         * bmalloc/SegregatedFreeList.h:
614
615 2015-01-13  Geoffrey Garen  <ggaren@apple.com>
616
617         Renamed minimum to size in SegregatedFreeList
618         https://bugs.webkit.org/show_bug.cgi?id=140406
619
620         Reviewed by Sam Weinig.
621
622         In preparation for supporting aligned allocation.
623
624         * bmalloc/SegregatedFreeList.cpp:
625         (bmalloc::SegregatedFreeList::takeGreedy):
626         (bmalloc::SegregatedFreeList::take): Every size passed to malloc is
627         really just a minimum. Let's not imply that this value is special.
628
629 2015-01-11  Dan Bernstein  <mitz@apple.com>
630
631         Geoff is organized, but he is not an organization.
632
633         Rubber-stamped by Anders Carlsson.
634
635         * bmalloc.xcodeproj/project.pbxproj: Removed the ORGANIZATIONNAME project attribute.
636
637 2015-01-07  Geoffrey Garen  <ggaren@apple.com>
638
639         Make bmalloc work with ASan
640         https://bugs.webkit.org/show_bug.cgi?id=140194
641
642         Reviewed by Mark Lam.
643
644         * bmalloc/BPlatform.h: Added a way to detect Darwin OSes, since we need
645         an OS-specific API to test for loaded runtime libraries.
646
647         * bmalloc/Environment.cpp:
648         (bmalloc::isASanEnabled):
649         (bmalloc::Environment::computeIsBmallocEnabled): Disabled bmalloc if
650         ASan is enabled, since system malloc has the Asan hooks we need.
651
652         You could check for the ASan compile-time flag instead, but doing this
653         check at runtime prepares bmalloc for a world where it is a dynamic
654         library that might be loaded into projects it did not compile with.
655
656 2015-01-05  Geoffrey Garen  <ggaren@apple.com>
657
658         Fix up bmalloc's PerThread for use on Linux
659         https://bugs.webkit.org/show_bug.cgi?id=139804
660
661         Reviewed by Anders Carlsson.
662
663         The previous implementation was a bit slow.
664
665         * bmalloc/PerThread.h:
666         (bmalloc::PerThreadStorage<Cache>::get):
667         (bmalloc::PerThreadStorage::get):
668         (bmalloc::PerThreadStorage::init): Added a catch-all cross-platform Unix
669         way to do fast per-thread access without taking a lock every time. This
670         probably works on all the platforms we care about, and it matches other
671         techniques we use elsewhere in WebKit.
672
673         (bmalloc::PerThread<T>::getFastCase): Removed the conditional from
674         this class because PerThreadStorage now encapsulates everything that
675         needs to be conditional.
676
677         (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Deleted.
678
679 2014-12-26  Dan Bernstein  <mitz@apple.com>
680
681         <rdar://problem/19348208> REGRESSION (r177027): iOS builds use the wrong toolchain
682         https://bugs.webkit.org/show_bug.cgi?id=139950
683
684         Reviewed by David Kilzer.
685
686         * Configurations/Base.xcconfig: Only define TOOLCHAINS when building for OS X, doing so
687         in a manner that works with Xcode 5.1.1.
688
689 2014-12-15  Geoffrey Garen  <ggaren@apple.com>
690
691         Safari crashes when you set Malloc environment variables
692         https://bugs.webkit.org/show_bug.cgi?id=139656
693
694         Reviewed by Michael Saboff.
695
696         I forgot to cover the realloc() case. Whoops. (OoPS?)
697
698         This time around, I ran the full MallocBench test suite in Malloc=1
699         mode, and it passed.
700
701         * bmalloc/Allocator.cpp:
702         (bmalloc::Allocator::reallocate):
703         * bmalloc/Allocator.h: Pushed realloc() logic down into the allocator.
704         It needs to be down there so that we can do the short-circuiting check
705         for whether bmalloc is enabled first.
706
707         Also added the check.
708
709         * bmalloc/Cache.cpp:
710         (bmalloc::Cache::scavenge):
711         (bmalloc::Cache::Cache):
712         (bmalloc::Cache::reallocateSlowCaseNullCache):
713         * bmalloc/Cache.h:
714         (bmalloc::Cache::deallocator):
715         (bmalloc::Cache::reallocate): Ditto.
716
717         * bmalloc/bmalloc.h:
718         (bmalloc::api::free):
719         (bmalloc::api::realloc): Ditto.
720
721         (bmalloc::api::scavenge): Pushed this down into Cache to match the
722         surrounding functions.
723
724 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
725
726         bmalloc should support system memory analysis tools (part 2)
727         https://bugs.webkit.org/show_bug.cgi?id=139565
728
729         Reviewed by Mark Lam.
730
731         This patch actually queries the environment to see if memory analysis
732         tools have been enabled.
733
734         * bmalloc/Deallocator.cpp:
735         (bmalloc::Deallocator::scavenge): Don't process the object log if
736         we've disabled bmalloc because it will be full of invalid nullptrs.
737
738         * bmalloc/Environment.cpp:
739         (bmalloc::isMallocEnvironmentVariableSet): Test for the list of known
740         Malloc debugging flags. I also added a plain "Malloc" catch-all for
741         when you want to disable bmalloc without enabling any kind of funny
742         business.
743
744         It would be slightly nicer just to iterate the list of environment
745         variables and strstr them, but getenv is the more portable option,
746         and performance here doesn't really matter.
747
748         (bmalloc::isLibgmallocEnabled): Test for the libgmalloc insertion
749         environment variable.
750
751         (bmalloc::Environment::computeIsBmallocEnabled):
752
753 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
754
755         Try to fix the iOS simulator build.
756
757         #include the declaration of malloc / free.
758
759         * bmalloc/Allocator.cpp:
760         * bmalloc/Deallocator.cpp:
761
762 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
763
764         Try to fix the build.
765
766         * bmalloc.xcodeproj/project.pbxproj: Marked a header exported.
767
768 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
769
770         bmalloc should support system memory analysis tools (part 1)
771         https://bugs.webkit.org/show_bug.cgi?id=139559
772
773         Reviewed by Mark Lam.
774
775         This patch adds the hooks to disable bmalloc at runtime if certain
776         environment variables are set, but doesn't actually read from the
777         environment yet.
778
779         No performance change.
780
781         * bmalloc.xcodeproj/project.pbxproj: Added the Environment class, which
782         we'll use to read environment variables and see if memory analysis tools
783         have been enabled.
784
785         * bmalloc/Allocator.cpp:
786         (bmalloc::Allocator::Allocator):
787         (bmalloc::Allocator::allocateSlowCase): Added a hook to disable bmalloc
788         on the allocation path. We cache the setting to make the check fast.
789
790         * bmalloc/Allocator.h: Interface changes.
791
792         * bmalloc/Cache.cpp:
793         (bmalloc::Cache::Cache): Pass a heap pointer through to our allocator
794         and deallocator. This main purpose is to enable them to query the
795         environment for whether bmalloc is enabled; but this is also a slightly
796         cleaner way to guarantee to them that the Heap has been pre-initialized.
797
798         * bmalloc/Deallocator.cpp:
799         (bmalloc::Deallocator::Deallocator): If bmalloc is disable, artificially
800         fill the object log to force us to take the slow path on all deallocations.
801
802         (bmalloc::Deallocator::deallocateSlowCase): Do the disabled check.
803
804         * bmalloc/Deallocator.h: Interface changes.
805
806         * bmalloc/Environment.cpp: Added.
807         (bmalloc::Environment::Environment):
808         (bmalloc::Environment::computeIsBmallocEnabled):
809         * bmalloc/Environment.h: Added.
810         (bmalloc::Environment::isBmallocEnabled): This is the class that will
811         encapsulate looking for environment variables that turn on heap
812         analysis tools.
813
814         * bmalloc/Heap.h:
815         (bmalloc::Heap::environment):
816
817         * bmalloc/Mutex.h:
818         (bmalloc::Mutex::Mutex):
819         * bmalloc/StaticMutex.h: A little refactoring to clarify these comments,
820         since I got super confused about them while writing this patch.
821
822         * bmalloc/VMHeap.cpp: Fixed an #include.
823
824 2014-12-09  David Kilzer  <ddkilzer@apple.com>
825
826         Switch from using PLATFORM_NAME to SDK selectors in ANGLE, bmalloc, gtest, JavaScriptCore, WTF
827         <http://webkit.org/b/139212>
828
829         Reviewed by Joseph Pecoraro.
830
831         * Configurations/Base.xcconfig:
832         - Only set GCC_ENABLE_OBJC_GC, GCC_MODEL_TUNING and TOOLCHAINS
833           on OS X.
834         * Configurations/DebugRelease.xcconfig:
835         - Only set MACOSX_DEPLOYMENT_TARGET and SDKROOT on OS X.
836
837 2014-11-07  Geoffrey Garen  <ggaren@apple.com>
838
839         bmalloc uses 8X more virtual memory than necessary
840         https://bugs.webkit.org/show_bug.cgi?id=138495
841
842         Reviewed by Mark Lam.
843
844         iOS has a per-process virtual memory cap around 1GB, so there's some
845         value to not going totally ham with virtual memory.
846
847         We currently use about 8X the necessary amount:
848             - 2X to align our VM allocation
849             - 4X to reserve small / medium / (2) large chunk VM ranges per superchunk
850
851         We can cut that down:
852             - Return the unaligned portion of our VM allocation (-2X)
853             - Use all the chunks in a superchunk, instead of allocating one
854               chunk per superchunk (-4X)
855
856         * bmalloc/Algorithm.h:
857         (bmalloc::roundUpToMultipleOf): Added a non-constant version of this
858         function so we can call it with getpagesize() at runtime.
859
860         * bmalloc/Chunk.h:
861         * bmalloc/LargeChunk.h:
862         (bmalloc::LargeChunk::create): Deleted. Instead of each chunk allocating
863         its own VM, VMHeap allocates the superchunk and all the chunks in it at a time.
864
865         * bmalloc/VMAllocate.h:
866         (bmalloc::vmValidate):
867         (bmalloc::vmAllocate): ASSERT that mmap succeeds to make crashes clearer
868         if it does not succeed. Allocate precisely, and give back the extra.
869
870         * bmalloc/VMHeap.cpp:
871         (bmalloc::VMHeap::allocateSuperChunk):
872         (bmalloc::VMHeap::allocateSmallChunk): Deleted.
873         (bmalloc::VMHeap::allocateMediumChunk): Deleted.
874         (bmalloc::VMHeap::allocateLargeChunk): Deleted. Use all the chunks
875         in a superchunk, instead of just one.
876
877         * bmalloc/VMHeap.h:
878         (bmalloc::VMHeap::allocateSmallPage):
879         (bmalloc::VMHeap::allocateMediumPage):
880         (bmalloc::VMHeap::allocateLargeRange):
881         * bmalloc/XLargeChunk.h:
882         (bmalloc::XLargeChunk::create): Updated to match changes above.
883
884 2014-11-01  David Kilzer  <ddkilzer@apple.com>
885
886         JavaScriptCore is missing debug info for bmalloc because libbmalloc.a is stripped
887         <https://webkit.org/b/138286>
888         <rdar://problem/18847087>
889
890         Reviewed by Dan Bernstein.
891
892         * Configurations/bmalloc.xcconfig: Set STRIP_INSTALLED_PRODUCT
893         to NO for the target that produces libbmalloc.a so that the
894         debug symbols will be linked into JavaScriptCore and end up in
895         its dSYM file.
896
897 2014-10-30  Dana Burkart  <dburkart@apple.com>
898
899         <rdar://problem/18821260> Prepare for the mysterious future
900
901         Reviewed by Lucas Forschler.
902
903         * Configurations/Base.xcconfig:
904         * Configurations/DebugRelease.xcconfig:
905
906 2014-09-24  Geoffrey Garen  <ggaren@apple.com>
907
908         bmalloc: cleaned up fast path vs slow path
909         https://bugs.webkit.org/show_bug.cgi?id=137081
910
911         Reviewed by Sam Weinig.
912
913         Might be a 1% speedup on MallocBench. Also cleans up the code a bit.
914
915         * bmalloc/Allocator.cpp:
916         (bmalloc::Allocator::Allocator): Merged the small and medium range
917         caches, just like the small and medium allocators. Ranges are abstract
918         objects that don't really care whether they hold small or medium objects,
919         so they don't need to be segregated.
920
921         (bmalloc::Allocator::scavenge): Ditto.
922
923         (bmalloc::Allocator::allocateBumpRangeSlowCase):
924         (bmalloc::Allocator::allocateBumpRange): Same thing here, except that
925         we do care a tiny bit, because we need to specify small vs medium when
926         allocating new ranges from the heap, to ensure that the heap allocates
927         from the right segment of VM.
928
929         (bmalloc::Allocator::allocateLarge):
930         (bmalloc::Allocator::allocateXLarge): NO_INLINE because this was clouding
931         up the fast path. Large allocation performance is dominated by allocation
932         logic and initialization, so inlining it doesn't help.
933
934         (bmalloc::Allocator::allocateSlowCase): Slow path got a bit cleaner since
935         it doesn't need to distinguish small vs medium objects.
936
937         (bmalloc::Allocator::allocateSmallBumpRange): Deleted.
938         (bmalloc::Allocator::allocateMediumBumpRange): Deleted.
939
940         * bmalloc/Allocator.h:
941         * bmalloc/BumpRange.h:
942
943         * bmalloc/Cache.cpp:
944         (bmalloc::Cache::allocateSlowCase): Deleted.
945         (bmalloc::Cache::deallocateSlowCase): Deleted.
946         * bmalloc/Cache.h:
947         (bmalloc::Cache::allocate):
948         (bmalloc::Cache::deallocate):
949         (bmalloc::Cache::allocateFastCase): Deleted.
950         (bmalloc::Cache::deallocateFastCase): Deleted. Removed the Cache slow
951         paths. The downside to this change is that the fast path branches to two
952         distinct failure cases instead of one. The upside is that the slow path
953         doesn't need to re-read the segment register, which is not as cheap as a
954         normal register, and it doesn't need to do an extra level of function 
955         call. Seems to be worth it.
956
957         * bmalloc/Deallocator.h:
958         * bmalloc/Heap.cpp:
959         (bmalloc::Heap::refillSmallBumpRangeCache):
960         (bmalloc::Heap::refillMediumBumpRangeCache):
961         * bmalloc/Heap.h: Updated for interface changes.
962
963         * bmalloc/Sizes.h: The most ranges a cache will hold is the number of
964         small lines in a page / 2, since any other free lines will coalesce
965         with their neighbors.
966
967 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
968
969         Rolled out r173346.
970
971             bmalloc should honor the FastMalloc statistics API
972             https://bugs.webkit.org/show_bug.cgi?id=136592
973
974         This didn't really work. Because we allow ranges with and without
975         physical pages to merge, and we allow double-committing and
976         double-decommitting, we can't rely on commit actions to track memory
977         footprint.
978
979         * bmalloc/Heap.cpp:
980         (bmalloc::Heap::size): Deleted.
981         (bmalloc::Heap::capacity): Deleted.
982         * bmalloc/Heap.h:
983         * bmalloc/VMHeap.cpp:
984         (bmalloc::VMHeap::VMHeap):
985         (bmalloc::VMHeap::allocateSmallChunk):
986         (bmalloc::VMHeap::allocateMediumChunk):
987         (bmalloc::VMHeap::allocateLargeChunk):
988         * bmalloc/VMHeap.h:
989         (bmalloc::VMHeap::allocateSmallPage):
990         (bmalloc::VMHeap::allocateMediumPage):
991         (bmalloc::VMHeap::allocateLargeRange):
992         (bmalloc::VMHeap::deallocateSmallPage):
993         (bmalloc::VMHeap::deallocateMediumPage):
994         (bmalloc::VMHeap::deallocateLargeRange):
995         (bmalloc::VMHeap::size): Deleted.
996         (bmalloc::VMHeap::capacity): Deleted.
997         * bmalloc/bmalloc.h:
998         (bmalloc::api::heapSize): Deleted.
999         (bmalloc::api::heapCapacity): Deleted.
1000
1001 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
1002
1003         bmalloc: Allocation should be more precise
1004         https://bugs.webkit.org/show_bug.cgi?id=136993
1005
1006         Reviewed by Gavin Barraclough.
1007
1008         13% reduction in heap size on the MallocBench *_memory_warning benchmarks.
1009
1010         This patch teaches the allocator to merge adjacent free lines into a
1011         single allocatable range. This allows us to shrink the size of an
1012         individual line without increasing fragmentation or the rate of allocator
1013         slow paths.
1014
1015         We'll only take more slow paths when available memory is sparse, which
1016         is exactly when it's worth it. When available memory is dense, we'll
1017         take fewer slow paths.
1018
1019         * bmalloc.xcodeproj/project.pbxproj:
1020         * bmalloc/Algorithm.h:
1021         (bmalloc::divideRoundingUp):
1022
1023         * bmalloc/Allocator.cpp:
1024         (bmalloc::Allocator::Allocator): Updated for interface changes.
1025
1026         (bmalloc::Allocator::scavenge): Scavenge by object instead of by line.
1027         Now that we merge lines, it's not convenient to scavenge by line.
1028
1029         (bmalloc::Allocator::allocateSmallBumpRange):
1030         (bmalloc::Allocator::allocateMediumBumpRange): Allocate whole ranges
1031         instead of individual lines.
1032
1033         (bmalloc::Allocator::allocateSlowCase):
1034         (bmalloc::Allocator::allocateSmallLine): Deleted.
1035         (bmalloc::Allocator::allocateMediumLine): Deleted.
1036         (bmalloc::Allocator::allocateMedium): Deleted.
1037         * bmalloc/Allocator.h:
1038         (bmalloc::Allocator::allocateFastCase): Folded medium allocations
1039         into the standard fast path with small allocations. Since a BumpAllocator
1040         just allocates out of an arbitrary range, it doesn't need to distinguish
1041         between small and medium lines.
1042
1043         * bmalloc/BumpAllocator.h:
1044         (bmalloc::BumpAllocator::size):
1045         (bmalloc::BumpAllocator::BumpAllocator):
1046         (bmalloc::BumpAllocator::init):
1047         (bmalloc::BumpAllocator::refill):
1048         (bmalloc::BumpAllocator::line): Deleted. No need to track line information
1049         anymore: the heap just gives us a pointer and a pre-computed number of
1050         objects, and we allocate them.
1051
1052         * bmalloc/Deallocator.cpp:
1053         (bmalloc::Deallocator::processObjectLog): Updated for interface changes.
1054
1055         * bmalloc/Heap.cpp:
1056         (bmalloc::Heap::Heap):
1057         (bmalloc::Heap::initializeLineMetadata): Pre-compute precise metadata
1058         detailing where all objects will lie in memory. After we merge two lines,
1059         we might allocate an object that spans from one line to the next. This
1060         metadata details which bits of memory overlap in that way, and how they
1061         overlap.
1062
1063         (bmalloc::Heap::refillSmallBumpRangeCache):
1064         (bmalloc::Heap::refillMediumBumpRangeCache): Scan a whole page at a time,
1065         and merge adjacent free lines into BumpRanges.
1066
1067         (bmalloc::Heap::allocateSmallPage):
1068         (bmalloc::Heap::allocateMediumPage):
1069         (bmalloc::Heap::deallocateSmallLine):
1070         (bmalloc::Heap::deallocateMediumLine): Track pages rather than lines,
1071         since we scan for free memory a page at a time.
1072
1073         (bmalloc::Heap::allocateSmallLineSlowCase): Deleted.
1074         (bmalloc::Heap::allocateMediumLineSlowCase): Deleted. Folded into the
1075         fast path.
1076
1077         * bmalloc/Heap.h:
1078         (bmalloc::Heap::derefSmallLine):
1079         (bmalloc::Heap::derefMediumLine):
1080         (bmalloc::Heap::deallocateSmallLine): Deleted.
1081         (bmalloc::Heap::allocateSmallLine): Deleted.
1082         (bmalloc::Heap::deallocateMediumLine): Deleted.
1083         (bmalloc::Heap::allocateMediumLine): Deleted. Updated for interface changes.
1084
1085         * bmalloc/Line.h:
1086         (bmalloc::Line<Traits>::ref):
1087         (bmalloc::Line<Traits>::deref):
1088         (bmalloc::Line<Traits>::concurrentRef): Deleted. We don't pass a derefCount
1089         anymore, since we only ever deref by 1 now.
1090
1091         * bmalloc/MediumAllocator.h:
1092         (bmalloc::MediumAllocator::isNull): Deleted.
1093         (bmalloc::MediumAllocator::MediumAllocator): Deleted.
1094         (bmalloc::MediumAllocator::line): Deleted.
1095         (bmalloc::MediumAllocator::allocate): Deleted.
1096         (bmalloc::MediumAllocator::derefCount): Deleted.
1097         (bmalloc::MediumAllocator::refill): Deleted.
1098         (bmalloc::MediumAllocator::clear): Deleted. Deleted some code that's
1099         been dead for a while, since it doesn't build anymore with this patch.
1100
1101         * bmalloc/Page.h:
1102         (bmalloc::Page::sizeClass):
1103         (bmalloc::Page::setSizeClass):
1104         (bmalloc::Page::smallSizeClass): Deleted.
1105         (bmalloc::Page::setSmallSizeClass): Deleted. Renamed setSmallSizeClass
1106         to sizeClass, since we use it for medium sizes too.
1107
1108         * bmalloc/Sizes.h:
1109         (bmalloc::Sizes::sizeClass):
1110         (bmalloc::Sizes::objectSize): Shrank line sizes to save memory.
1111
1112         (bmalloc::Sizes::smallSizeClassFor): Deleted.
1113         (bmalloc::Sizes::mediumSizeClassFor): Deleted.
1114
1115         * bmalloc/bmalloc.h:
1116         (bmalloc::api::realloc): Now that we have precise objects sizes, realloc
1117         can be a bit more precise. It also has to be, since we can't guarantee
1118         that an object ends at the end of a line anymore.
1119
1120 2014-09-19  Daniel Bates  <dabates@apple.com>
1121
1122         Always assume internal SDK when building configuration Production
1123         https://bugs.webkit.org/show_bug.cgi?id=136925
1124         <rdar://problem/18362399>
1125
1126         Reviewed by Dan Bernstein.
1127
1128         * Configurations/Base.xcconfig:
1129
1130 2014-09-16  Geoffrey Garen  <ggaren@apple.com>
1131
1132         bmalloc: moved line caches from the deallocator to the allocator
1133         https://bugs.webkit.org/show_bug.cgi?id=136868
1134
1135         Reviewed by Gavin Barraclough.
1136
1137         I did this mostly as a simplification, to make it easier to change the
1138         allocation strategy.
1139
1140         No throughput change on MallocBench. Saves about 50kB.
1141
1142         Since the deallocator needs to lock the heap when freeing lines anyway,
1143         there isn't much benefit to giving the deallocator a local cache of
1144         deallocated lines.
1145
1146         We still give the allocator a local cache of lines because that does
1147         reduce the frequency at which it needs to lock the heap in order to
1148         acquire more lines.
1149
1150         * bmalloc/Allocator.cpp:
1151         (bmalloc::Allocator::scavenge):
1152         (bmalloc::Allocator::allocateSmallLine):
1153         (bmalloc::Allocator::allocateMediumLine):
1154         (bmalloc::Allocator::allocateMedium):
1155         (bmalloc::Allocator::allocateSlowCase):
1156         * bmalloc/Allocator.h:
1157         * bmalloc/Deallocator.cpp:
1158         (bmalloc::Deallocator::Deallocator):
1159         (bmalloc::Deallocator::scavenge):
1160         (bmalloc::Deallocator::processObjectLog):
1161         (bmalloc::Deallocator::deallocateSmallLine): Deleted.
1162         (bmalloc::Deallocator::allocateSmallLine): Deleted.
1163         (bmalloc::Deallocator::deallocateMediumLine): Deleted.
1164         (bmalloc::Deallocator::allocateMediumLine): Deleted.
1165         * bmalloc/Deallocator.h:
1166
1167         * bmalloc/Sizes.h:
1168         * bmalloc/VMAllocate.h: Took the opportunity to make the line cache size
1169         exactly one page in size. That's about what we were shooting for anyway,
1170         and it may make it easier to switch to per-page allocation in future.
1171
1172 2014-09-15  Geoffrey Garen  <ggaren@apple.com>
1173
1174         bmalloc: allocate small and medium objects using the same bump pointer class
1175         https://bugs.webkit.org/show_bug.cgi?id=136843
1176
1177         Reviewed by Gavin Barraclough.
1178
1179         4% speedup on MallocBench.
1180
1181         Now that medium-sized objects have dedicated per-size allocators, they
1182         don't need to use an arbitrary bump pointer allocator. This means that
1183         every allocator knows how many objects it will allocate from the start,
1184         and we don't need a post-processing step to adjust refcounts based on
1185         real allocation count.
1186
1187         * bmalloc.xcodeproj/project.pbxproj: Renamed SmallAllocator to BumpAllocator
1188         since it's used for small and medium objects now.
1189
1190         * bmalloc/Allocator.cpp:
1191         (bmalloc::Allocator::Allocator): Updated to use new interface.
1192         (bmalloc::Allocator::scavenge): To "retire" an allocator, we just need
1193         to make sure that we finish allocating all the objects in it.
1194
1195         (bmalloc::Allocator::allocateMedium):
1196         (bmalloc::Allocator::allocateSlowCase):
1197         (bmalloc::Allocator::retire): Deleted.
1198         (bmalloc::Allocator::processSmallAllocatorLog): Deleted.
1199         (bmalloc::Allocator::processMediumAllocatorLog): Deleted.
1200         * bmalloc/Allocator.h:
1201         (bmalloc::Allocator::allocateFastCase): Removed abstractions and data
1202         used to post-process an allocator based on how many objects it allocated.
1203
1204         * bmalloc/BumpAllocator.h: Copied from Source/bmalloc/bmalloc/SmallAllocator.h.
1205         (bmalloc::BumpAllocator::BumpAllocator):
1206         (bmalloc::BumpAllocator::init):
1207         (bmalloc::BumpAllocator::line):
1208         (bmalloc::BumpAllocator::validate):
1209         (bmalloc::BumpAllocator::allocate):
1210         (bmalloc::BumpAllocator::refill):
1211         (bmalloc::BumpAllocator::clear): Updated these functions to be agnostic
1212         about the kinds of lines they allocate into. In some cases, the line
1213         type must be provided as a template parameter by the caller.
1214
1215         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
1216         (bmalloc::SmallAllocator::line): Deleted.
1217         (bmalloc::SmallAllocator::allocate): Deleted.
1218         (bmalloc::SmallAllocator::objectCount): Deleted.
1219         (bmalloc::SmallAllocator::derefCount): Deleted.
1220         (bmalloc::SmallAllocator::refill): Deleted.
1221         (bmalloc::SmallAllocator::clear): Deleted.
1222
1223         * bmalloc/ObjectType.h:
1224         (bmalloc::isMedium):
1225
1226         * bmalloc/SmallAllocator.h:
1227         (bmalloc::SmallAllocator::isNull): Deleted.
1228         (bmalloc::SmallAllocator::canAllocate): Deleted.
1229         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
1230         (bmalloc::SmallAllocator::line): Deleted.
1231         (bmalloc::SmallAllocator::allocate): Deleted.
1232         (bmalloc::SmallAllocator::objectCount): Deleted.
1233         (bmalloc::SmallAllocator::derefCount): Deleted.
1234         (bmalloc::SmallAllocator::refill): Deleted.
1235         (bmalloc::SmallAllocator::clear): Deleted.
1236
1237 2014-09-12  Geoffrey Garen  <ggaren@apple.com>
1238
1239         Fixed a goof in bmalloc Vector sizing
1240         https://bugs.webkit.org/show_bug.cgi?id=136795
1241
1242         Reviewed by Gavin Barraclough and Sam Weinig.
1243
1244         We want our minimum vector to be page-sized since the OS will give us
1245         a page no matter what -- but we want that many bytes, and not enough
1246         bytes to store that many elements.
1247
1248         * bmalloc/Vector.h: Math is hard.
1249
1250 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
1251
1252         bmalloc should segregate medium-sized objects by line like it does for small-sized objects
1253         https://bugs.webkit.org/show_bug.cgi?id=136693
1254
1255         Reviewed by Gavin Barraclough.
1256
1257         4% reduction in heap size on the MallocBench *_memory_warning benchmarks.
1258
1259         No throughput change.
1260
1261         We keep an array of medium allocators, just like our array of small
1262         allocators.
1263
1264         In future, we can simplify the allocation fast path by merging the small
1265         and medium allocator arrays. For now, this is the simplest change that
1266         gets the win.
1267
1268         * bmalloc/Allocator.cpp:
1269         (bmalloc::Allocator::Allocator):
1270         (bmalloc::Allocator::scavenge):
1271         (bmalloc::Allocator::allocateMedium):
1272         * bmalloc/Allocator.h:
1273         * bmalloc/Sizes.h:
1274         (bmalloc::Sizes::mediumSizeClassFor):
1275
1276 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
1277
1278         Reviewed by Sam Weinig.
1279
1280         Renamed log => retire for clarity.
1281
1282         * bmalloc/Allocator.cpp:
1283         (bmalloc::Allocator::scavenge):
1284         (bmalloc::Allocator::retire):
1285         (bmalloc::Allocator::allocateMedium):
1286         (bmalloc::Allocator::allocateSlowCase):
1287         (bmalloc::Allocator::log): Deleted.
1288         * bmalloc/Allocator.h:
1289
1290 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
1291
1292         bmalloc: eager scavenge leaves behind a bogus allocator
1293         https://bugs.webkit.org/show_bug.cgi?id=136743
1294
1295         Reviewed by Sam Weinig.
1296
1297         Be sure to clear the allocator after logging it in the eager scavenge
1298         case, so that we don't later try to allocate out of the lines that we
1299         have thrown away.
1300
1301         We didn't need to do this previously because scavenge would only happen
1302         at thread exit time, after which no further allocation from the per-thread
1303         cache would take place.
1304
1305         * bmalloc/Allocator.cpp:
1306         (bmalloc::Allocator::scavenge):
1307         * bmalloc/MediumAllocator.h:
1308         (bmalloc::MediumAllocator::clear):
1309         * bmalloc/SmallAllocator.h:
1310         (bmalloc::SmallAllocator::clear):
1311
1312 2014-09-05  Geoffrey Garen  <ggaren@apple.com>
1313
1314         bmalloc should honor the FastMalloc statistics API
1315         https://bugs.webkit.org/show_bug.cgi?id=136592
1316
1317         Reviewed by Gavin Barraclough.
1318
1319         We do this by tracking "size" and "capacity" in the VM heap.
1320
1321         The VM heap's "capacity" is all the VM we ever allocated.
1322
1323         The VM heap's "size" the subset of VM currently held onto by the
1324         VM heap (and therefore not in use by the regular heap).
1325
1326         Somewhat ironically, reducing the process's memory footprint, increases
1327         the size of the VM heap, since the VM heap holds the pages that are
1328         purely virtual and not physical.
1329
1330         * bmalloc/Heap.cpp:
1331         (bmalloc::Heap::size):
1332         (bmalloc::Heap::capacity):
1333         * bmalloc/Heap.h:
1334         * bmalloc/VMHeap.cpp:
1335         (bmalloc::VMHeap::VMHeap):
1336         (bmalloc::VMHeap::allocateSmallChunk):
1337         (bmalloc::VMHeap::allocateMediumChunk):
1338         (bmalloc::VMHeap::allocateLargeChunk):
1339         * bmalloc/VMHeap.h:
1340         (bmalloc::VMHeap::size):
1341         (bmalloc::VMHeap::capacity):
1342         (bmalloc::VMHeap::allocateSmallPage):
1343         (bmalloc::VMHeap::allocateMediumPage):
1344         (bmalloc::VMHeap::allocateLargeRange):
1345         (bmalloc::VMHeap::deallocateSmallPage):
1346         (bmalloc::VMHeap::deallocateMediumPage):
1347         (bmalloc::VMHeap::deallocateLargeRange):
1348         * bmalloc/bmalloc.h:
1349         (bmalloc::api::heapSize):
1350         (bmalloc::api::heapCapacity):
1351
1352 2014-09-02  Geoffrey Garen  <ggaren@apple.com>
1353
1354         bmalloc crashes on the EWS bots (due to bad large object allocation)
1355         https://bugs.webkit.org/show_bug.cgi?id=136469
1356
1357         Reviewed by Andreas Kling.
1358
1359         It's possible to convince bmalloc to perform a bad large object allocation,
1360         through these steps:
1361
1362         (1) Insert object A into freelist F0.
1363
1364         (2) Split, merge and split again A's neighbors such that object B is
1365         inserted into freelist F0, with boundary tag and size equal to object A,
1366         but pointer not completely equal to object A. Put object B at the head of F0.
1367
1368         (3) Allocate some other object from F0, swapping its position in the
1369         freelist with object B, such that object A is now ahead of object B.
1370
1371         --> Now, the next allocation for size A/B will allocate object A, which
1372         has a slightly wrong idea about where the object actually begins.
1373         Immediately, you'll corrupt a little memory, and over time, you'll also
1374         corrupt boundary tag metadata.
1375
1376         The solution is to store the begin pointer in the boundary tag. Luckily,
1377         this doesn't make the tag any bigger, and it's not a noticeable slowdown
1378         on MallocBench.
1379
1380         * bmalloc/Algorithm.h:
1381         (bmalloc::rightShift):
1382         * bmalloc/BeginTag.h:
1383         (bmalloc::BeginTag::isInFreeList): This is the bug fix. Make sure to
1384         validate the start pointer when popping off the free list. Through a
1385         very uncommon set of steps, it is possible to have an item in the free
1386         list that is valid by all accounts except for its start pointer.
1387
1388         * bmalloc/BoundaryTag.h:
1389         (bmalloc::BoundaryTag::compactBegin):
1390         (bmalloc::BoundaryTag::setRange):
1391         (bmalloc::BoundaryTag::setSize): Deleted. Record a compact version of the
1392         start pointer. We don't need the whole pointer -- just the offset, in
1393         largeAlignment increments, into the relevant boundary tag bucket.
1394
1395         * bmalloc/BoundaryTagInlines.h:
1396         (bmalloc::validateNext):
1397         (bmalloc::BoundaryTag::init):
1398         (bmalloc::BoundaryTag::mergeLarge):
1399         (bmalloc::BoundaryTag::splitLarge):
1400         * bmalloc/SegregatedFreeList.cpp:
1401         (bmalloc::SegregatedFreeList::insert):
1402         (bmalloc::SegregatedFreeList::takeGreedy):
1403         (bmalloc::SegregatedFreeList::take): Provide the whole range instead of
1404         the size when establishing a boundary tag, as required by the new
1405         interface.
1406
1407         * bmalloc/Sizes.h:
1408
1409 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
1410
1411         Fixed a bmalloc crash seen on the EWS bot
1412         https://bugs.webkit.org/show_bug.cgi?id=135955
1413
1414         Reviewed by Andreas Kling.
1415
1416         * bmalloc/Syscall.h: Some CG APIs vm_copy their input buffers. If the
1417         input buffer is a malloc region, that region will get marked Copy-On-Write
1418         by the kernel. Calls to madvise() for COW regions fail and return EINVAL
1419         on older OS X's. In 10.10, they still fail, but they do not return
1420         EINVAL.
1421
1422         So, we can only ASSERT that our syscalls succeed starting with 10.10.
1423
1424 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
1425
1426         Fixed the bmalloc build
1427         https://bugs.webkit.org/show_bug.cgi?id=135953
1428
1429         Reviewed by Andreas Kling.
1430
1431         * bmalloc.xcodeproj/project.pbxproj: Marked a few headers as private.
1432         These headers are used, so they must be available outside the project.
1433
1434 2014-08-13  Daniel Bates  <dabates@apple.com>
1435
1436         Attempt to fix the build following <http://trac.webkit.org/changeset/172576>
1437         (https://bugs.webkit.org/show_bug.cgi?id=135895)
1438
1439         Substitute PerThreadStorage<T>::initSharedKeyIfNeeded() for initSharedKeyIfNeeded() in
1440         implementation of PerThread<T>::getFastCase().
1441
1442         * bmalloc/PerThread.h:
1443         (bmalloc::PerThread<T>::getFastCase):
1444
1445 2014-08-13  Daniel Bates  <dabates@apple.com>
1446
1447         Make bmalloc::PerThread work without C++ thread local storage
1448         https://bugs.webkit.org/show_bug.cgi?id=135895
1449
1450         Reviewed by Geoffrey Garen.
1451
1452         Implement support for building bmalloc without C++ thread local storage.
1453
1454         * bmalloc/BPlatform.h: Remove macro define BPLATFORM_IOS_SIMULATOR. Added macro function
1455         BCOMPILER_SUPPORTS() and macro define BCOMPILER_SUPPORTS_CXX_THREAD_LOCAL that can be used
1456         to determine whether the compiler supports C++ thread local storage.
1457         * bmalloc/PerThread.h:
1458         (bmalloc::PerThreadStorage::get): Modified to call pthread_getspecific() when building
1459         without C++ thread local storage.
1460         (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Added.
1461         (bmalloc::PerThreadStorage::init): Moved logic to initialize shared Pthread key from here to
1462         PerThreadStorage::initSharedKeyIfNeeded().
1463         (bmalloc::PerThread<T>::getFastCase): Modified to call PerThreadStorage::initSharedKeyIfNeeded()
1464         before querying PerThreadStorage::get() when building without C++ thread local storage so as to
1465         ensure that the shared key has been initialized.
1466         (_pthread_setspecific_direct): Deleted.
1467         (_pthread_getspecific_direct): Deleted.
1468
1469 2014-08-13  Daniel Bates  <dabates@apple.com>
1470
1471         [iOS] Make JavaScriptCore and bmalloc build with the public SDK
1472         https://bugs.webkit.org/show_bug.cgi?id=135848
1473
1474         Reviewed by Geoffrey Garen.
1475
1476         * bmalloc/BPlatform.h: Added macro BPLATFORM_IOS_SIMULATOR, which evaluates to true
1477         when building for the iOS Simulator.
1478         * bmalloc/PerThread.h: Use pthread_machdep.h code path when building for iOS Simulator
1479         using the public SDK.
1480         (_pthread_setspecific_direct): Added; only defined when building for the iOS Simulator
1481         using the public SDK.
1482         (_pthread_getspecific_direct): Added; only defined when building for the iOS Simulator
1483         using the public SDK.
1484
1485 2014-08-12  Daniel Bates  <dabates@apple.com>
1486
1487         BPLATFORM(IOS) always evaluates to false
1488         https://bugs.webkit.org/show_bug.cgi?id=135843
1489
1490         Reviewed by Geoffrey Garen.
1491
1492         Fix typo in definition of BPLATFORM() and include system header TargetConditionals.h
1493         (when building on an Apple platform) so that BPLATFORM(X) evaluates to true when
1494         building for platform X. In particular, so that BPLATFORM(IOS) evaluates to true when
1495         building for iOS.
1496
1497         As a side effect of this change, the change made in <http://trac.webkit.org/changeset/167289>
1498         will be honored and iOS will assume a VM page size of 16kB (again) instead of 4kB.
1499
1500         * bmalloc/BPlatform.h:
1501
1502 2014-08-11  Andy Estes  <aestes@apple.com>
1503
1504         [iOS] Get rid of iOS.xcconfig
1505         https://bugs.webkit.org/show_bug.cgi?id=135809
1506
1507         Reviewed by Joseph Pecoraro.
1508
1509         All iOS.xcconfig did was include AspenFamily.xcconfig, so there's no need for the indirection.
1510
1511         * Configurations/Base.xcconfig:
1512         * Configurations/iOS.xcconfig: Removed.
1513         * bmalloc.xcodeproj/project.pbxproj:
1514
1515 2014-05-01  Dan Bernstein  <mitz@apple.com>
1516
1517         Fixed production builds for the iOS Simulator.
1518         <rdar://problem/16792221>
1519
1520         * Configurations/bmalloc.xcconfig: Include INSTALL_PATH_PREFIX in
1521         PRIVATE_HEADERS_FOLDER_PATH when installing.
1522
1523 2014-04-20  Geoffrey Garen  <ggaren@apple.com>
1524
1525         bmalloc: Segregate pages by objects size
1526         https://bugs.webkit.org/show_bug.cgi?id=131909
1527
1528         Reviewed by Andreas Kling.
1529
1530         2% reduction in memory-at-end on the Membuster memory_warning benchmarks.
1531
1532         * bmalloc/Allocator.cpp:
1533         (bmalloc::Allocator::allocateSlowCase):
1534         * bmalloc/Allocator.h:
1535         (bmalloc::Allocator::allocateFastCase):
1536         (bmalloc::Allocator::smallAllocatorFor): Use the new shared helper
1537         function for size class calculation.
1538
1539         * bmalloc/Deallocator.cpp:
1540         (bmalloc::Deallocator::Deallocator):
1541         (bmalloc::Deallocator::scavenge):
1542         (bmalloc::Deallocator::deallocateSmallLine):
1543         (bmalloc::Deallocator::allocateSmallLine):
1544         * bmalloc/Deallocator.h: Keep a cache for every size class, since the
1545         cache can't be shared anymore.
1546
1547         * bmalloc/Heap.cpp:
1548         (bmalloc::Heap::allocateSmallLineSlowCase):
1549         * bmalloc/Heap.h:
1550         (bmalloc::Heap::deallocateSmallLine): Ditto.
1551
1552         (bmalloc::Heap::allocateSmallLine): Check size class in addition to
1553         page refcount when allocating a line because we might have deallocated
1554         the page and the recycled it for another size class.
1555
1556         (bmalloc::Heap::deallocateMediumLine):
1557         (bmalloc::Heap::allocateMediumLine):
1558         * bmalloc/Line.h:
1559         (bmalloc::Line::refCount):
1560         * bmalloc/Page.h:
1561         (bmalloc::Page::refCount):
1562         (bmalloc::Page::smallSizeClass):
1563         (bmalloc::Page::setSmallSizeClass):
1564         (bmalloc::Page<Traits>::refCount): Deleted.
1565         * bmalloc/Sizes.h:
1566         (bmalloc::Sizes::smallSizeClassFor): New shared API for computing
1567         an index into an array from a size.
1568
1569 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1570
1571         bmalloc: Improved alignment in LargeChunk
1572         https://bugs.webkit.org/show_bug.cgi?id=131895
1573
1574         Reviewed by Andreas Kling.
1575
1576         * bmalloc/Chunk.h:
1577         * bmalloc/LargeChunk.h: Align to vmPageSize just like Chunk does.
1578         Technically, the previous alignment was harmless, but I would prefer,
1579         dear reader, not to have to explain the interlocking set of
1580         circumstances that made it so.
1581
1582 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1583
1584         Rolled out r167502 because it caused a crash on the facebook benchmark.
1585
1586         Unreviewed.
1587
1588             bmalloc: Added an XSmall line size
1589             https://bugs.webkit.org/show_bug.cgi?id=131851
1590
1591             Reviewed by Sam Weinig.
1592
1593 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1594
1595         bmalloc: Mutex should be harder to use wrong
1596         https://bugs.webkit.org/show_bug.cgi?id=131879
1597
1598         Reviewed by Andreas Kling.
1599
1600         Mutex now has a proper constructor, so you can't deadlock by forgetting
1601         to initialize it.
1602
1603         * bmalloc.xcodeproj/project.pbxproj:
1604         * bmalloc/Allocator.cpp:
1605         (bmalloc::Allocator::processXSmallAllocatorLog):
1606         (bmalloc::Allocator::processSmallAllocatorLog):
1607         (bmalloc::Allocator::processMediumAllocatorLog):
1608         (bmalloc::Allocator::allocateLarge):
1609         (bmalloc::Allocator::allocateXLarge): Global replace Mutex => StaticMutex,
1610         since the Heap mutex is a static.
1611
1612         * bmalloc/AsyncTask.h:
1613         (bmalloc::Function>::AsyncTask): Use Mutex, since we're not static. No
1614         need for explicit initialization anymore.
1615
1616         * bmalloc/Deallocator.cpp:
1617         (bmalloc::Deallocator::scavenge):
1618         (bmalloc::Deallocator::deallocateLarge):
1619         (bmalloc::Deallocator::deallocateXLarge):
1620         (bmalloc::Deallocator::processObjectLog):
1621         (bmalloc::Deallocator::deallocateSmallLine):
1622         (bmalloc::Deallocator::deallocateXSmallLine):
1623         (bmalloc::Deallocator::allocateSmallLine):
1624         (bmalloc::Deallocator::allocateXSmallLine):
1625         (bmalloc::Deallocator::deallocateMediumLine):
1626         (bmalloc::Deallocator::allocateMediumLine):
1627         * bmalloc/Deallocator.h:
1628         * bmalloc/Heap.cpp:
1629         (bmalloc::sleep):
1630         (bmalloc::Heap::Heap):
1631         (bmalloc::Heap::concurrentScavenge):
1632         (bmalloc::Heap::scavenge):
1633         (bmalloc::Heap::scavengeSmallPages):
1634         (bmalloc::Heap::scavengeXSmallPages):
1635         (bmalloc::Heap::scavengeMediumPages):
1636         (bmalloc::Heap::scavengeLargeRanges):
1637         (bmalloc::Heap::allocateXSmallLineSlowCase):
1638         (bmalloc::Heap::allocateSmallLineSlowCase):
1639         (bmalloc::Heap::allocateMediumLineSlowCase):
1640         (bmalloc::Heap::allocateXLarge):
1641         (bmalloc::Heap::deallocateXLarge):
1642         (bmalloc::Heap::allocateLarge):
1643         (bmalloc::Heap::deallocateLarge):
1644         * bmalloc/Heap.h:
1645         (bmalloc::Heap::deallocateXSmallLine):
1646         (bmalloc::Heap::allocateXSmallLine):
1647         (bmalloc::Heap::deallocateSmallLine):
1648         (bmalloc::Heap::allocateSmallLine):
1649         (bmalloc::Heap::deallocateMediumLine):
1650         (bmalloc::Heap::allocateMediumLine):
1651         * bmalloc/Line.h:
1652         (bmalloc::Line<Traits>::deref):
1653         * bmalloc/Mutex.cpp: Removed.
1654         * bmalloc/Mutex.h:
1655         (bmalloc::Mutex::Mutex):
1656         (bmalloc::Mutex::init): Deleted.
1657         (bmalloc::Mutex::try_lock): Deleted.
1658         (bmalloc::Mutex::lock): Deleted.
1659         (bmalloc::Mutex::unlock): Deleted.
1660         * bmalloc/Page.h:
1661         (bmalloc::Page<Traits>::ref):
1662         (bmalloc::Page<Traits>::deref):
1663         (bmalloc::Page<Traits>::refCount):
1664         * bmalloc/PerProcess.h:
1665         (bmalloc::PerProcess::mutex):
1666         (bmalloc::PerProcess<T>::getSlowCase):
1667         * bmalloc/StaticMutex.cpp: Added.
1668         (bmalloc::StaticMutex::lockSlowCase):
1669         * bmalloc/StaticMutex.h: Added.
1670         (bmalloc::StaticMutex::init):
1671         (bmalloc::StaticMutex::try_lock):
1672         (bmalloc::StaticMutex::lock):
1673         (bmalloc::StaticMutex::unlock):
1674         * bmalloc/VMHeap.h:
1675         (bmalloc::VMHeap::deallocateXSmallPage):
1676         (bmalloc::VMHeap::deallocateSmallPage):
1677         (bmalloc::VMHeap::deallocateMediumPage):
1678         (bmalloc::VMHeap::deallocateLargeRange):
1679         * bmalloc/bmalloc.h:
1680         (bmalloc::api::scavenge): Global replace Mutex => StaticMutex,
1681         since the Heap mutex is a static.
1682
1683 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
1684
1685         bmalloc: AsyncTask should use Mutex instead of std::mutex
1686         https://bugs.webkit.org/show_bug.cgi?id=131865
1687
1688         Reviewed by Gavin Barraclough.
1689
1690         std::mutex is so slow that it makes parallelizing simple tasks through
1691         AsyncTask a net regression. Mutex fixes this.
1692
1693         * bmalloc/AsyncTask.h:
1694         (bmalloc::Function>::AsyncTask):
1695         (bmalloc::Function>::join):
1696         (bmalloc::Function>::runSlowCase):
1697         (bmalloc::Function>::entryPoint):
1698         * bmalloc/Mutex.h:
1699         (bmalloc::Mutex::init):
1700
1701 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
1702
1703         bmalloc: Added an XSmall line size
1704         https://bugs.webkit.org/show_bug.cgi?id=131851
1705
1706         Reviewed by Sam Weinig.
1707
1708         Reduces malloc footprint on Membuster recordings by 10%.
1709
1710         This is a throughput regression, but we're still way ahead of TCMalloc.
1711         I have some ideas for how to recover the regression -- but I wanted to
1712         get this win in first.
1713
1714         Full set of benchmark results:
1715
1716                 bmalloc> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks --measure-heap nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1717                                                                                                 
1718                                                                nopatch                      patch                                Δ
1719                 Peak Memory:
1720                     reddit_memory_warning                      7,896kB                    7,532kB                  ^ 1.05x smaller
1721                     flickr_memory_warning                     12,968kB                   12,324kB                  ^ 1.05x smaller
1722                     theverge_memory_warning                   16,672kB                   15,200kB                   ^ 1.1x smaller
1723
1724                     <geometric mean>                          11,952kB                   11,216kB                  ^ 1.07x smaller
1725                     <arithmetic mean>                         12,512kB                   11,685kB                  ^ 1.07x smaller
1726                     <harmonic mean>                           11,375kB                   10,726kB                  ^ 1.06x smaller
1727
1728                 Memory at End:
1729                     reddit_memory_warning                      7,320kB                    6,856kB                  ^ 1.07x smaller
1730                     flickr_memory_warning                     10,848kB                    9,692kB                  ^ 1.12x smaller
1731                     theverge_memory_warning                   16,380kB                   14,872kB                   ^ 1.1x smaller
1732
1733                     <geometric mean>                          10,916kB                    9,961kB                   ^ 1.1x smaller
1734                     <arithmetic mean>                         11,516kB                   10,473kB                   ^ 1.1x smaller
1735                     <harmonic mean>                           10,350kB                    9,485kB                  ^ 1.09x smaller
1736
1737                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1738                                                                                                 
1739                                                    nopatch                patch                         Δ
1740                 Execution Time:
1741                     churn                            127ms                151ms            ! 1.19x slower
1742                     list_allocate                    130ms                164ms            ! 1.26x slower
1743                     tree_allocate                    109ms                127ms            ! 1.17x slower
1744                     tree_churn                       115ms                120ms            ! 1.04x slower
1745                     facebook                         240ms                259ms            ! 1.08x slower
1746                     fragment                          91ms                131ms            ! 1.44x slower
1747                     fragment_iterate                 105ms                106ms            ! 1.01x slower
1748                     message_one                      260ms                259ms             ^ 1.0x faster
1749                     message_many                     149ms                154ms            ! 1.03x slower
1750                     medium                           194ms                248ms            ! 1.28x slower
1751                     big                              157ms                160ms            ! 1.02x slower
1752
1753                     <geometric mean>                 144ms                163ms            ! 1.13x slower
1754                     <arithmetic mean>                152ms                171ms            ! 1.12x slower
1755                     <harmonic mean>                  137ms                156ms            ! 1.14x slower
1756
1757                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1758                                                                                                 
1759                                                                        nopatch                          patch                                     Δ
1760                 Execution Time:
1761                     churn                                                126ms                          148ms                        ! 1.17x slower
1762                     churn --parallel                                      62ms                           76ms                        ! 1.23x slower
1763                     list_allocate                                        130ms                          164ms                        ! 1.26x slower
1764                     list_allocate --parallel                             120ms                          175ms                        ! 1.46x slower
1765                     tree_allocate                                        111ms                          127ms                        ! 1.14x slower
1766                     tree_allocate --parallel                              95ms                          135ms                        ! 1.42x slower
1767                     tree_churn                                           115ms                          124ms                        ! 1.08x slower
1768                     tree_churn --parallel                                107ms                          126ms                        ! 1.18x slower
1769                     facebook                                             240ms                          276ms                        ! 1.15x slower
1770                     facebook --parallel                                  802ms                        1,088ms                        ! 1.36x slower
1771                     fragment                                              92ms                          130ms                        ! 1.41x slower
1772                     fragment --parallel                                   66ms                          124ms                        ! 1.88x slower
1773                     fragment_iterate                                     109ms                          127ms                        ! 1.17x slower
1774                     fragment_iterate --parallel                           55ms                           64ms                        ! 1.16x slower
1775                     message_one                                          260ms                          260ms                                      
1776                     message_many                                         170ms                          238ms                         ! 1.4x slower
1777                     medium                                               185ms                          250ms                        ! 1.35x slower
1778                     medium --parallel                                    210ms                          334ms                        ! 1.59x slower
1779                     big                                                  150ms                          169ms                        ! 1.13x slower
1780                     big --parallel                                       138ms                          144ms                        ! 1.04x slower
1781
1782                     <geometric mean>                                     135ms                          170ms                        ! 1.26x slower
1783                     <arithmetic mean>                                    167ms                          214ms                        ! 1.28x slower
1784                     <harmonic mean>                                      117ms                          148ms                        ! 1.26x slower
1785
1786                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
1787
1788                                                                     TC                      patch                                Δ
1789                 Peak Memory:
1790                     reddit_memory_warning                     13,836kB                   13,436kB                  ^ 1.03x smaller
1791                     flickr_memory_warning                     24,868kB                   25,188kB                   ! 1.01x bigger
1792                     theverge_memory_warning                   24,504kB                   26,636kB                   ! 1.09x bigger
1793
1794                     <geometric mean>                          20,353kB                   20,812kB                   ! 1.02x bigger
1795                     <arithmetic mean>                         21,069kB                   21,753kB                   ! 1.03x bigger
1796                     <harmonic mean>                           19,570kB                   19,780kB                   ! 1.01x bigger
1797
1798                 Memory at End:
1799                     reddit_memory_warning                      8,656kB                   10,016kB                   ! 1.16x bigger
1800                     flickr_memory_warning                     11,844kB                   13,784kB                   ! 1.16x bigger
1801                     theverge_memory_warning                   18,516kB                   22,748kB                   ! 1.23x bigger
1802
1803                     <geometric mean>                          12,382kB                   14,644kB                   ! 1.18x bigger
1804                     <arithmetic mean>                         13,005kB                   15,516kB                   ! 1.19x bigger
1805                     <harmonic mean>                           11,813kB                   13,867kB                   ! 1.17x bigger
1806
1807                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
1808                                                                                                 
1809                                                         TC                patch                         Δ
1810                 Execution Time:
1811                     churn                            416ms                148ms            ^ 2.81x faster
1812                     list_allocate                    463ms                164ms            ^ 2.82x faster
1813                     tree_allocate                    292ms                127ms             ^ 2.3x faster
1814                     tree_churn                       157ms                120ms            ^ 1.31x faster
1815                     facebook                         327ms                276ms            ^ 1.18x faster
1816                     fragment                         335ms                129ms             ^ 2.6x faster
1817                     fragment_iterate                 344ms                108ms            ^ 3.19x faster
1818                     message_one                      386ms                258ms             ^ 1.5x faster
1819                     message_many                     410ms                154ms            ^ 2.66x faster
1820                     medium                           391ms                245ms             ^ 1.6x faster
1821                     big                              261ms                167ms            ^ 1.56x faster
1822
1823                     <geometric mean>                 332ms                164ms            ^ 2.02x faster
1824                     <arithmetic mean>                344ms                172ms            ^ 1.99x faster
1825                     <harmonic mean>                  317ms                157ms            ^ 2.02x faster
1826
1827         * bmalloc.xcodeproj/project.pbxproj:
1828         * bmalloc/Allocator.cpp:
1829         (bmalloc::Allocator::Allocator): Don't assume that each allocator's
1830         index corresponds with its size. Instead, use the size selection function
1831         explicitly. Now that we have XSmall, some small allocator entries are
1832         unused.
1833
1834         (bmalloc::Allocator::scavenge):
1835         (bmalloc::Allocator::log):
1836         (bmalloc::Allocator::processXSmallAllocatorLog):
1837         (bmalloc::Allocator::allocateSlowCase):
1838         * bmalloc/Allocator.h:
1839         (bmalloc::Allocator::xSmallAllocatorFor):
1840         (bmalloc::Allocator::allocateFastCase):
1841         * bmalloc/Chunk.h:
1842         * bmalloc/Deallocator.cpp:
1843         (bmalloc::Deallocator::scavenge):
1844         (bmalloc::Deallocator::processObjectLog):
1845         (bmalloc::Deallocator::deallocateSlowCase):
1846         (bmalloc::Deallocator::deallocateXSmallLine):
1847         (bmalloc::Deallocator::allocateXSmallLine):
1848         * bmalloc/Deallocator.h:
1849         (bmalloc::Deallocator::deallocateFastCase):
1850         * bmalloc/Heap.cpp:
1851         (bmalloc::Heap::scavenge):
1852         (bmalloc::Heap::scavengeXSmallPages):
1853         (bmalloc::Heap::allocateXSmallLineSlowCase):
1854         * bmalloc/Heap.h:
1855         (bmalloc::Heap::deallocateXSmallLine):
1856         (bmalloc::Heap::allocateXSmallLine):
1857         * bmalloc/LargeChunk.h:
1858         (bmalloc::LargeChunk::get):
1859         (bmalloc::LargeChunk::endTag):
1860         * bmalloc/Line.h:
1861         * bmalloc/MediumAllocator.h:
1862         (bmalloc::MediumAllocator::allocate):
1863         (bmalloc::MediumAllocator::refill):
1864         * bmalloc/ObjectType.cpp:
1865         (bmalloc::objectType):
1866         * bmalloc/ObjectType.h:
1867         (bmalloc::isXSmall):
1868         (bmalloc::isSmall):
1869         (bmalloc::isMedium):
1870         (bmalloc::isLarge):
1871         (bmalloc::isSmallOrMedium): Deleted.
1872         * bmalloc/SegregatedFreeList.h: I boiler-plate copied existing code for
1873         handling small objects. There's probably a reasonable way to share this
1874         code in the future -- I'll look into that once it's stopped changing.
1875
1876         * bmalloc/Sizes.h: Tweaked size classes to make Membuster happy. This
1877         is the main reason things got slower.
1878
1879         * bmalloc/SmallAllocator.h:
1880         (bmalloc::SmallAllocator::allocate):
1881         * bmalloc/SmallTraits.h:
1882         * bmalloc/VMHeap.cpp:
1883         (bmalloc::VMHeap::allocateXSmallChunk):
1884         * bmalloc/VMHeap.h:
1885         (bmalloc::VMHeap::allocateXSmallPage):
1886         (bmalloc::VMHeap::deallocateXSmallPage):
1887         * bmalloc/XSmallAllocator.h: Added.
1888         (bmalloc::XSmallAllocator::isNull):
1889         (bmalloc::XSmallAllocator::canAllocate):
1890         (bmalloc::XSmallAllocator::XSmallAllocator):
1891         (bmalloc::XSmallAllocator::line):
1892         (bmalloc::XSmallAllocator::allocate):
1893         (bmalloc::XSmallAllocator::objectCount):
1894         (bmalloc::XSmallAllocator::derefCount):
1895         (bmalloc::XSmallAllocator::refill):
1896         * bmalloc/XSmallChunk.h: Added.
1897         * bmalloc/XSmallLine.h: Added.
1898         * bmalloc/XSmallPage.h: Added.
1899         * bmalloc/XSmallTraits.h: Added.
1900         * bmalloc/bmalloc.h:
1901         (bmalloc::api::realloc): Boiler-plate copy, as above.
1902
1903 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
1904
1905         MallocBench should scavenge explicitly instead of waiting
1906         https://bugs.webkit.org/show_bug.cgi?id=131661
1907
1908         Reviewed by Andreas Kling.
1909
1910         Added explicit scavenge support to bmalloc. This isn't a memory win,
1911         since bmalloc's per-thread cache is so small. But it makes testing
1912         simpler.
1913
1914         * bmalloc/Allocator.cpp:
1915         (bmalloc::Allocator::~Allocator):
1916         (bmalloc::Allocator::scavenge):
1917         * bmalloc/Allocator.h:
1918         * bmalloc/Cache.cpp:
1919         (bmalloc::Cache::operator new):
1920         (bmalloc::Cache::operator delete):
1921         (bmalloc::Cache::Cache):
1922         (bmalloc::Cache::scavenge):
1923         * bmalloc/Cache.h:
1924         * bmalloc/Deallocator.cpp:
1925         (bmalloc::Deallocator::~Deallocator):
1926         (bmalloc::Deallocator::scavenge):
1927         * bmalloc/Deallocator.h: Factored existing scavenging code into helper
1928         functions, for reuse.
1929
1930         * bmalloc/Heap.cpp:
1931         (bmalloc::sleep):
1932         (bmalloc::Heap::concurrentScavenge):
1933         (bmalloc::Heap::scavenge):
1934         (bmalloc::Heap::scavengeSmallPages):
1935         (bmalloc::Heap::scavengeMediumPages):
1936         (bmalloc::Heap::scavengeLargeRanges):
1937         * bmalloc/Heap.h: Made scavenge sleep duration a parameter. Forced
1938         scavenging -- in response to a benchmark or a low memory warning --
1939         wants to complete as soon as possible, so its sleep duration is 0.
1940
1941         * bmalloc/bmalloc.h:
1942         (bmalloc::api::scavenge):
1943         * bmalloc/mbmalloc.cpp: Exported the scavenge API for MallocBench's use.
1944
1945 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
1946
1947         Use 4kB pages on Mac
1948         https://bugs.webkit.org/show_bug.cgi?id=131658
1949
1950         Reviewed by Sam Weinig.
1951
1952         This reduces memory use a lot on Membuster:
1953
1954                                                                   base                      patch                                Δ
1955                 Execution Time:
1956                     reddit_memory_warning                         18ms                       17ms                   ^ 1.06x faster
1957                     flickr_memory_warning                         34ms                       36ms                   ! 1.06x slower
1958                     theverge_memory_warning                       39ms                       41ms                   ! 1.05x slower
1959
1960                     <geometric mean>                              29ms                       29ms                   ! 1.02x slower
1961                     <arithmetic mean>                             30ms                       31ms                   ! 1.03x slower
1962                     <harmonic mean>                               27ms                       27ms                    ^ 1.0x faster
1963
1964                 Peak Memory:
1965                     reddit_memory_warning                     16,412kB                   16,436kB                    ! 1.0x bigger
1966                     flickr_memory_warning                     30,120kB                   30,184kB                    ! 1.0x bigger
1967                     theverge_memory_warning                   33,408kB                   33,420kB                    ! 1.0x bigger
1968
1969                     <geometric mean>                          25,466kB                   25,499kB                    ! 1.0x bigger
1970                     <arithmetic mean>                         26,647kB                   26,680kB                    ! 1.0x bigger
1971                     <harmonic mean>                           24,181kB                   24,214kB                    ! 1.0x bigger
1972
1973                 Memory at End:
1974                     reddit_memory_warning                      2,404kB                    1,920kB                  ^ 1.25x smaller
1975                     flickr_memory_warning                      3,764kB                    3,072kB                  ^ 1.23x smaller
1976                     theverge_memory_warning                    3,648kB                    3,132kB                  ^ 1.16x smaller
1977
1978                     <geometric mean>                           3,208kB                    2,644kB                  ^ 1.21x smaller
1979                     <arithmetic mean>                          3,272kB                    2,708kB                  ^ 1.21x smaller
1980                     <harmonic mean>                            3,139kB                    2,574kB                  ^ 1.22x smaller
1981
1982
1983         * bmalloc.xcodeproj/project.pbxproj:
1984         * bmalloc/BPlatform.h: Added.
1985         * bmalloc/VMAllocate.h: Only use 16kB pages on iOS because the page size
1986         is 4kB on Mac.
1987
1988 2014-04-14  Alexey Proskuryakov  <ap@apple.com>
1989
1990         Fixed svn:ignore on bmalloc.xcodeproj, it had erroneous leading spaces.
1991
1992         * bmalloc.xcodeproj: Modified property svn:ignore.
1993
1994 2014-04-13  Geoffrey Garen  <ggaren@apple.com>
1995
1996         Fixed some mbmalloc exports
1997         https://bugs.webkit.org/show_bug.cgi?id=131599
1998
1999         Reviewed by Ryosuke Niwa.
2000
2001         * bmalloc.xcodeproj/project.pbxproj: Made some headers a private part
2002         of the project, so we can call them from API.
2003
2004         * bmalloc/mbmalloc.cpp: Marked the mbmalloc functions with default
2005         visibility, so they show up as exported in the .dylib.
2006
2007 2014-04-09  Geoffrey Garen  <ggaren@apple.com>
2008
2009         Put bmalloc headers in the right place
2010         https://bugs.webkit.org/show_bug.cgi?id=131464
2011
2012         Reviewed by Mark Rowe.
2013
2014         * Configurations/bmalloc.xcconfig: Set PRIVATE_HEADERS_FOLDER_PATH to
2015         specify that we don't just want to dump all of our generically-named
2016         headers into /usr/local/include.
2017
2018 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
2019
2020         Made bmalloc more #include friendly
2021         https://bugs.webkit.org/show_bug.cgi?id=131386
2022
2023         Reviewed by Andreas Kling.
2024
2025         Marked a bunch of headers private so they can be used from client code
2026         that #includes bmalloc.h.
2027
2028         Renamed ASSERT macros to BASSERT. This matches their header, which already
2029         had to be renamed, and fixes conflicts with WTF's ASSERT macros.
2030
2031         * bmalloc.xcodeproj/project.pbxproj:
2032         * bmalloc/Allocator.cpp:
2033         (bmalloc::Allocator::allocateSlowCase):
2034         * bmalloc/AsyncTask.h:
2035         (bmalloc::Function>::runSlowCase):
2036         * bmalloc/BAssert.h:
2037         * bmalloc/BoundaryTag.h:
2038         (bmalloc::BoundaryTag::setSize):
2039         * bmalloc/BoundaryTagInlines.h:
2040         (bmalloc::validate):
2041         (bmalloc::BoundaryTag::init):
2042         (bmalloc::BoundaryTag::deallocate):
2043         (bmalloc::BoundaryTag::splitLarge):
2044         (bmalloc::BoundaryTag::allocate):
2045         * bmalloc/Chunk.h:
2046         * bmalloc/Deallocator.cpp:
2047         (bmalloc::Deallocator::processObjectLog):
2048         (bmalloc::Deallocator::deallocateSlowCase):
2049         * bmalloc/Deallocator.h:
2050         (bmalloc::Deallocator::deallocateFastCase):
2051         * bmalloc/FixedVector.h:
2052         (bmalloc::Capacity>::operator):
2053         (bmalloc::Capacity>::push):
2054         (bmalloc::Capacity>::pop):
2055         (bmalloc::Capacity>::shrink):
2056         * bmalloc/Heap.cpp:
2057         (bmalloc::Heap::allocateLarge):
2058         * bmalloc/LargeChunk.h:
2059         (bmalloc::LargeChunk::get):
2060         (bmalloc::LargeChunk::endTag):
2061         * bmalloc/Line.h:
2062         (bmalloc::Line<Traits>::concurrentRef):
2063         (bmalloc::Line<Traits>::deref):
2064         * bmalloc/MediumAllocator.h:
2065         (bmalloc::MediumAllocator::allocate):
2066         * bmalloc/ObjectType.h:
2067         (bmalloc::isSmall):
2068         * bmalloc/Page.h:
2069         (bmalloc::Page<Traits>::ref):
2070         (bmalloc::Page<Traits>::deref):
2071         * bmalloc/PerThread.h:
2072         (bmalloc::PerThread<T>::getSlowCase):
2073         * bmalloc/SegregatedFreeList.cpp:
2074         (bmalloc::SegregatedFreeList::SegregatedFreeList):
2075         (bmalloc::SegregatedFreeList::insert):
2076         * bmalloc/SmallAllocator.h:
2077         (bmalloc::SmallAllocator::allocate):
2078         (bmalloc::SmallAllocator::refill):
2079         * bmalloc/Syscall.h:
2080         * bmalloc/VMAllocate.h:
2081         (bmalloc::vmValidate):
2082         (bmalloc::vmAllocate):
2083         (bmalloc::vmDeallocatePhysicalPagesSloppy):
2084         * bmalloc/Vector.h:
2085         (bmalloc::Vector<T>::operator):
2086         (bmalloc::Vector<T>::pop):
2087         (bmalloc::Vector<T>::shrink):
2088         * bmalloc/XLargeChunk.h:
2089         (bmalloc::XLargeChunk::range):
2090         (bmalloc::XLargeChunk::size):
2091
2092 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
2093
2094         Removed an unused file.
2095
2096         Unreviewed.
2097
2098         * bmalloc/AsyncTask.cpp: Removed.
2099
2100 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
2101
2102         Build bmalloc on Mac
2103         https://bugs.webkit.org/show_bug.cgi?id=131333
2104
2105         Reviewed by Mark Rowe.
2106
2107         * Makefile: Added. For make clients.
2108
2109         These files are required for building any project in WebKit. I copied
2110         them from WTF:
2111         * Configurations: Added.
2112         * Configurations/Base.xcconfig: Added.
2113         * Configurations/DebugRelease.xcconfig: Added.
2114         * Configurations/bmalloc.xcconfig: Added.
2115         * Configurations/iOS.xcconfig: Added.
2116         * Configurations/mbmalloc.xcconfig: Added.
2117
2118         * bmalloc.xcodeproj/project.pbxproj: I removed per-project-file stuff
2119         from here because everything is in .xcconfig files now.
2120
2121         I had to fix a bunch of minor warnings, since they're enabled in our
2122         .xcconfig files:
2123
2124         * bmalloc/AsyncTask.h:
2125         (bmalloc::Function>::AsyncTask):
2126         * bmalloc/BAssert.h:
2127         * bmalloc/BoundaryTagInlines.h:
2128         (bmalloc::validate):
2129         * bmalloc/Heap.cpp:
2130         (bmalloc::Heap::Heap):
2131         (bmalloc::Heap::allocateLarge):
2132         (bmalloc::Heap::deallocateLarge):
2133         * bmalloc/Mutex.h:
2134         (bmalloc::Mutex::Mutex): Deleted.
2135         * bmalloc/VMAllocate.h:
2136         (bmalloc::vmValidate):
2137         * bmalloc/mbmalloc.cpp:
2138
2139 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
2140
2141         bmalloc: Fixed a leak in the per-thread cache
2142         https://bugs.webkit.org/show_bug.cgi?id=131330
2143
2144         Reviewed by Andreas Kling.
2145
2146         Remember to deallocate our line caches upon thread exit.
2147
2148         * bmalloc/Deallocator.cpp:
2149         (bmalloc::Deallocator::~Deallocator):
2150
2151 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
2152
2153         bmalloc: rolled out the tryLock experiment
2154         https://bugs.webkit.org/show_bug.cgi?id=131328
2155
2156         Reviewed by Andreas Kling.
2157
2158         It wasn't a speedup.
2159
2160         * bmalloc.xcodeproj/project.pbxproj:
2161         * bmalloc/Allocator.cpp:
2162         (bmalloc::Allocator::processSmallAllocatorLog):
2163         (bmalloc::Allocator::processMediumAllocatorLog):
2164         * bmalloc/Deallocator.cpp:
2165         (bmalloc::Deallocator::processObjectLog):
2166         (bmalloc::Deallocator::deallocateSlowCase):
2167         (bmalloc::Deallocator::deallocateSmallLine):
2168         (bmalloc::Deallocator::deallocateMediumLine):
2169         * bmalloc/Deallocator.h:
2170         (bmalloc::Deallocator::deallocateFastCase):
2171         * bmalloc/Heap.h:
2172         (bmalloc::Heap::deallocateSmallLine):
2173         (bmalloc::Heap::deallocateMediumLine):
2174         * bmalloc/Line.h:
2175         (bmalloc::Line<Traits>::deref):
2176         * bmalloc/Page.h:
2177         (bmalloc::Page<Traits>::deref):
2178
2179 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
2180
2181         bmalloc
2182         https://bugs.webkit.org/show_bug.cgi?id=131170
2183
2184         Reviewed by Andreas Kling.
2185
2186         Initial commit.
2187
2188         * bmalloc: Added.
2189         * bmalloc.xcodeproj: Added.
2190         * bmalloc.xcodeproj/project.pbxproj: Added.
2191         * bmalloc/Algorithm.h: Added.
2192         (bmalloc::max):
2193         (bmalloc::min):
2194         (bmalloc::mask):
2195         (bmalloc::test):
2196         (bmalloc::roundUpToMultipleOf):
2197         (bmalloc::roundDownToMultipleOf):
2198         (bmalloc::sizeOf):
2199         (bmalloc::bitCount):
2200         (bmalloc::isPowerOfTwo):
2201         * bmalloc/Allocator.cpp: Added.
2202         (bmalloc::Allocator::Allocator):
2203         (bmalloc::Allocator::~Allocator):
2204         (bmalloc::Allocator::log):
2205         (bmalloc::Allocator::processSmallAllocatorLog):
2206         (bmalloc::Allocator::processMediumAllocatorLog):
2207         (bmalloc::Allocator::allocateLarge):
2208         (bmalloc::Allocator::allocateXLarge):
2209         (bmalloc::Allocator::allocateMedium):
2210         (bmalloc::Allocator::allocateSlowCase):
2211         * bmalloc/Allocator.h: Added.
2212         (bmalloc::Allocator::smallAllocatorFor):
2213         (bmalloc::Allocator::allocateFastCase):
2214         (bmalloc::Allocator::allocate):
2215         * bmalloc/AsyncTask.cpp: Added.
2216         (bmalloc::AsyncTask<Function>::runSlowCase):
2217         (bmalloc::AsyncTask<Function>::pthreadEntryPoint):
2218         (bmalloc::AsyncTask<Function>::entryPoint):
2219         * bmalloc/AsyncTask.h: Added.
2220         (bmalloc::Function>::AsyncTask):
2221         (bmalloc::Function>::join):
2222         (bmalloc::Function>::run):
2223         (bmalloc::Function>::runSlowCase):
2224         (bmalloc::Function>::pthreadEntryPoint):
2225         (bmalloc::Function>::entryPoint):
2226         * bmalloc/BAssert.h: Added.
2227         * bmalloc/BeginTag.h: Added.
2228         (bmalloc::BeginTag::isInFreeList):
2229         * bmalloc/BoundaryTag.h: Added.
2230         (bmalloc::BoundaryTag::isXLarge):
2231         (bmalloc::BoundaryTag::setXLarge):
2232         (bmalloc::BoundaryTag::isFree):
2233         (bmalloc::BoundaryTag::setFree):
2234         (bmalloc::BoundaryTag::isEnd):
2235         (bmalloc::BoundaryTag::setEnd):
2236         (bmalloc::BoundaryTag::hasPhysicalPages):
2237         (bmalloc::BoundaryTag::setHasPhysicalPages):
2238         (bmalloc::BoundaryTag::isNull):
2239         (bmalloc::BoundaryTag::clear):
2240         (bmalloc::BoundaryTag::size):
2241         (bmalloc::BoundaryTag::setSize):
2242         (bmalloc::BoundaryTag::prev):
2243         (bmalloc::BoundaryTag::next):
2244         * bmalloc/BoundaryTagInlines.h: Added.
2245         (bmalloc::validate):
2246         (bmalloc::validatePrev):
2247         (bmalloc::validateNext):
2248         (bmalloc::BoundaryTag::init):
2249         (bmalloc::BoundaryTag::mergeLargeLeft):
2250         (bmalloc::BoundaryTag::mergeLargeRight):
2251         (bmalloc::BoundaryTag::mergeLarge):
2252         (bmalloc::BoundaryTag::deallocate):
2253         (bmalloc::BoundaryTag::splitLarge):
2254         (bmalloc::BoundaryTag::allocate):
2255         * bmalloc/Cache.cpp: Added.
2256         (bmalloc::Cache::operator new):
2257         (bmalloc::Cache::operator delete):
2258         (bmalloc::Cache::Cache):
2259         (bmalloc::Cache::allocateSlowCase):
2260         (bmalloc::Cache::allocateSlowCaseNullCache):
2261         (bmalloc::Cache::deallocateSlowCase):
2262         (bmalloc::Cache::deallocateSlowCaseNullCache):
2263         * bmalloc/Cache.h: Added.
2264         (bmalloc::Cache::allocator):
2265         (bmalloc::Cache::deallocator):
2266         (bmalloc::Cache::allocateFastCase):
2267         (bmalloc::Cache::deallocateFastCase):
2268         (bmalloc::Cache::allocate):
2269         (bmalloc::Cache::deallocate):
2270         * bmalloc/Chunk.h: Added.
2271         (bmalloc::Chunk::begin):
2272         (bmalloc::Chunk::end):
2273         (bmalloc::Chunk::lines):
2274         (bmalloc::Chunk::pages):
2275         * bmalloc/Deallocator.cpp: Added.
2276         (bmalloc::Deallocator::Deallocator):
2277         (bmalloc::Deallocator::~Deallocator):
2278         (bmalloc::Deallocator::deallocateLarge):
2279         (bmalloc::Deallocator::deallocateXLarge):
2280         (bmalloc::Deallocator::processObjectLog):
2281         (bmalloc::Deallocator::deallocateSlowCase):
2282         (bmalloc::Deallocator::deallocateSmallLine):
2283         (bmalloc::Deallocator::allocateSmallLine):
2284         (bmalloc::Deallocator::deallocateMediumLine):
2285         (bmalloc::Deallocator::allocateMediumLine):
2286         * bmalloc/Deallocator.h: Added.
2287         (bmalloc::Deallocator::deallocateFastCase):
2288         (bmalloc::Deallocator::deallocate):
2289         * bmalloc/EndTag.h: Added.
2290         (bmalloc::EndTag::operator=):
2291         * bmalloc/FixedVector.h: Added.
2292         (bmalloc::FixedVector::begin):
2293         (bmalloc::FixedVector::end):
2294         (bmalloc::FixedVector::size):
2295         (bmalloc::FixedVector::capacity):
2296         (bmalloc::FixedVector::clear):
2297         (bmalloc::FixedVector::isEmpty):
2298         (bmalloc::Capacity>::FixedVector):
2299         (bmalloc::Capacity>::operator):
2300         (bmalloc::Capacity>::push):
2301         (bmalloc::Capacity>::pop):
2302         (bmalloc::Capacity>::shrink):
2303         * bmalloc/Heap.cpp: Added.
2304         (bmalloc::sleep):
2305         (bmalloc::Heap::Heap):
2306         (bmalloc::Heap::concurrentScavenge):
2307         (bmalloc::Heap::scavengeSmallPages):
2308         (bmalloc::Heap::scavengeMediumPages):
2309         (bmalloc::Heap::scavengeLargeRanges):
2310         (bmalloc::Heap::allocateSmallLineSlowCase):
2311         (bmalloc::Heap::allocateMediumLineSlowCase):
2312         (bmalloc::Heap::allocateXLarge):
2313         (bmalloc::Heap::deallocateXLarge):
2314         (bmalloc::Heap::allocateLarge):
2315         (bmalloc::Heap::deallocateLarge):
2316         * bmalloc/Heap.h: Added.
2317         (bmalloc::Heap::deallocateSmallLine):
2318         (bmalloc::Heap::allocateSmallLine):
2319         (bmalloc::Heap::deallocateMediumLine):
2320         (bmalloc::Heap::allocateMediumLine):
2321         * bmalloc/Inline.h: Added.
2322         * bmalloc/LargeChunk.h: Added.
2323         (bmalloc::LargeChunk::begin):
2324         (bmalloc::LargeChunk::end):
2325         (bmalloc::LargeChunk::create):
2326         (bmalloc::LargeChunk::get):
2327         (bmalloc::LargeChunk::beginTag):
2328         (bmalloc::LargeChunk::endTag):
2329         * bmalloc/Line.h: Added.
2330         (bmalloc::Line<Traits>::begin):
2331         (bmalloc::Line<Traits>::end):
2332         (bmalloc::Line<Traits>::concurrentRef):
2333         (bmalloc::Line<Traits>::deref):
2334         * bmalloc/MediumAllocator.h: Added.
2335         (bmalloc::MediumAllocator::isNull):
2336         (bmalloc::MediumAllocator::MediumAllocator):
2337         (bmalloc::MediumAllocator::line):
2338         (bmalloc::MediumAllocator::allocate):
2339         (bmalloc::MediumAllocator::derefCount):
2340         (bmalloc::MediumAllocator::refill):
2341         * bmalloc/MediumChunk.h: Added.
2342         * bmalloc/MediumLine.h: Added.
2343         * bmalloc/MediumPage.h: Added.
2344         * bmalloc/MediumTraits.h: Added.
2345         * bmalloc/Mutex.cpp: Added.
2346         (bmalloc::Mutex::lockSlowCase):
2347         * bmalloc/Mutex.h: Added.
2348         (bmalloc::Mutex::Mutex):
2349         (bmalloc::Mutex::try_lock):
2350         (bmalloc::Mutex::lock):
2351         (bmalloc::Mutex::unlock):
2352         * bmalloc/ObjectType.cpp: Added.
2353         (bmalloc::objectType):
2354         * bmalloc/ObjectType.h: Added.
2355         (bmalloc::isSmallOrMedium):
2356         (bmalloc::isSmall):
2357         * bmalloc/Page.h: Added.
2358         (bmalloc::Page<Traits>::ref):
2359         (bmalloc::Page<Traits>::deref):
2360         (bmalloc::Page<Traits>::refCount):
2361         * bmalloc/PerProcess.h: Added.
2362         (bmalloc::PerProcess::mutex):
2363         (bmalloc::PerProcess<T>::getFastCase):
2364         (bmalloc::PerProcess<T>::get):
2365         (bmalloc::PerProcess<T>::getSlowCase):
2366         * bmalloc/PerThread.h: Added.
2367         (bmalloc::PerThreadStorage<Cache>::get):
2368         (bmalloc::PerThreadStorage<Cache>::init):
2369         (bmalloc::PerThreadStorage::get):
2370         (bmalloc::PerThreadStorage::init):
2371         (bmalloc::PerThread<T>::getFastCase):
2372         (bmalloc::PerThread<T>::get):
2373         (bmalloc::PerThread<T>::destructor):
2374         (bmalloc::PerThread<T>::getSlowCase):
2375         * bmalloc/Range.h: Added.
2376         (bmalloc::Range::Range):
2377         (bmalloc::Range::begin):
2378         (bmalloc::Range::end):
2379         (bmalloc::Range::size):
2380         (bmalloc::Range::operator!):
2381         (bmalloc::Range::operator<):
2382         * bmalloc/SegregatedFreeList.cpp: Added.
2383         (bmalloc::SegregatedFreeList::SegregatedFreeList):
2384         (bmalloc::SegregatedFreeList::insert):
2385         (bmalloc::SegregatedFreeList::takeGreedy):
2386         (bmalloc::SegregatedFreeList::take):
2387         * bmalloc/SegregatedFreeList.h: Added.
2388         * bmalloc/Sizes.h: Added.
2389         * bmalloc/SmallAllocator.h: Added.
2390         (bmalloc::SmallAllocator::isNull):
2391         (bmalloc::SmallAllocator::canAllocate):
2392         (bmalloc::SmallAllocator::SmallAllocator):
2393         (bmalloc::SmallAllocator::line):
2394         (bmalloc::SmallAllocator::allocate):
2395         (bmalloc::SmallAllocator::objectCount):
2396         (bmalloc::SmallAllocator::derefCount):
2397         (bmalloc::SmallAllocator::refill):
2398         * bmalloc/SmallChunk.h: Added.
2399         * bmalloc/SmallLine.h: Added.
2400         * bmalloc/SmallPage.h: Added.
2401         * bmalloc/SmallTraits.h: Added.
2402         * bmalloc/Syscall.h: Added.
2403         * bmalloc/VMAllocate.h: Added.
2404         (bmalloc::vmSize):
2405         (bmalloc::vmValidate):
2406         (bmalloc::vmAllocate):
2407         (bmalloc::vmDeallocate):
2408         (bmalloc::vmDeallocatePhysicalPages):
2409         (bmalloc::vmAllocatePhysicalPages):
2410         (bmalloc::vmDeallocatePhysicalPagesSloppy):
2411         (bmalloc::vmAllocatePhysicalPagesSloppy):
2412         * bmalloc/VMHeap.cpp: Added.
2413         (bmalloc::VMHeap::VMHeap):
2414         (bmalloc::VMHeap::allocateSmallChunk):
2415         (bmalloc::VMHeap::allocateMediumChunk):
2416         (bmalloc::VMHeap::allocateLargeChunk):
2417         * bmalloc/VMHeap.h: Added.
2418         (bmalloc::VMHeap::allocateSmallPage):
2419         (bmalloc::VMHeap::allocateMediumPage):
2420         (bmalloc::VMHeap::allocateLargeRange):
2421         (bmalloc::VMHeap::deallocateSmallPage):
2422         (bmalloc::VMHeap::deallocateMediumPage):
2423         (bmalloc::VMHeap::deallocateLargeRange):
2424         * bmalloc/Vector.h: Added.
2425         (bmalloc::Vector::begin):
2426         (bmalloc::Vector::end):
2427         (bmalloc::Vector::size):
2428         (bmalloc::Vector::capacity):
2429         (bmalloc::Vector::last):
2430         (bmalloc::Vector::pop):
2431         (bmalloc::Vector<T>::Vector):
2432         (bmalloc::Vector<T>::~Vector):
2433         (bmalloc::Vector<T>::operator):
2434         (bmalloc::Vector<T>::push):
2435         (bmalloc::Vector<T>::pop):
2436         (bmalloc::Vector<T>::shrink):
2437         (bmalloc::Vector<T>::reallocateBuffer):
2438         (bmalloc::Vector<T>::shrinkCapacity):
2439         (bmalloc::Vector<T>::growCapacity):
2440         * bmalloc/XLargeChunk.h: Added.
2441         (bmalloc::XLargeChunk::get):
2442         (bmalloc::XLargeChunk::begin):
2443         (bmalloc::XLargeChunk::XLargeChunk):
2444         (bmalloc::XLargeChunk::create):
2445         (bmalloc::XLargeChunk::destroy):
2446         (bmalloc::XLargeChunk::range):
2447         (bmalloc::XLargeChunk::size):
2448         * bmalloc/bmalloc.h: Added.
2449         (bmalloc::api::malloc):
2450         (bmalloc::api::free):
2451         (bmalloc::api::realloc):
2452         * bmalloc/mbmalloc.cpp: Added.
2453