4a5be195e6690c5e267a5c5cfb9e4ab64fcba035
[WebKit-https.git] / Source / bmalloc / ChangeLog
1 2015-02-26  Geoffrey Garen  <ggaren@apple.com>
2
3         bmalloc: Refactored SegregatedFreeList and BoundaryTag::init
4         https://bugs.webkit.org/show_bug.cgi?id=142049
5
6         Reviewed by Anders Carlsson.
7
8         Split out a FreeList class from SegregatedFreeList. This will make it
9         easier to add behaviors on free list insertion and removal -- and it's
10         probably how I should have designed things at the start.
11
12         Moved BoundaryTag::init into LargeObject, since all the related logic
13         lives in LargeObject now too, and this allows us to remove BoundaryTagInlines.h.
14
15         * bmalloc.xcodeproj/project.pbxproj:
16         * bmalloc/BoundaryTagInlines.h: Removed.
17         * bmalloc/FreeList.cpp: Copied from Source/bmalloc/bmalloc/SegregatedFreeList.cpp.
18         (bmalloc::FreeList::takeGreedy):
19         (bmalloc::FreeList::take):
20         (bmalloc::SegregatedFreeList::SegregatedFreeList): Deleted.
21         (bmalloc::SegregatedFreeList::insert): Deleted.
22         (bmalloc::SegregatedFreeList::takeGreedy): Deleted.
23         (bmalloc::SegregatedFreeList::take): Deleted.
24         * bmalloc/FreeList.h: Copied from Source/bmalloc/bmalloc/SegregatedFreeList.h.
25         (bmalloc::FreeList::push):
26         * bmalloc/LargeObject.h:
27         (bmalloc::LargeObject::init):
28         * bmalloc/SegregatedFreeList.cpp:
29         (bmalloc::SegregatedFreeList::SegregatedFreeList):
30         (bmalloc::SegregatedFreeList::insert):
31         (bmalloc::SegregatedFreeList::takeGreedy):
32         (bmalloc::SegregatedFreeList::take):
33         * bmalloc/SegregatedFreeList.h:
34         * bmalloc/Sizes.h:
35         * bmalloc/VMHeap.cpp:
36         (bmalloc::VMHeap::grow):
37
38 2015-02-26  Geoffrey Garen  <ggaren@apple.com>
39
40         bmalloc: free up a bit in BoundaryTag
41         https://bugs.webkit.org/show_bug.cgi?id=142048
42
43         Reviewed by Brady Eidson.
44
45         We were wasting a bit by accident, and I need one now.
46
47         * bmalloc/Algorithm.h:
48         (bmalloc::rightShift): Deleted. Not needed, now that I've simplified
49         the math.
50
51         * bmalloc/BoundaryTag.h: Since each boundary tag bucket is 1024 bytes
52         long, the maximum offset into a bucket is 1023.
53
54         You need 5 bits to count up to 1024, but only 4 to count up to 1023.
55         
56         Math is hard.
57
58         (bmalloc::BoundaryTag::compactBegin): Switched to division because it
59         is simpler, and easier to match up with our ASSERT. The compiler will
60         turn division by constant power of two into a shift for us.
61
62         (bmalloc::BoundaryTag::setRange): Added an ASSERT for compactBegin
63         because we do encode it, so we should ASSERT that encoding did not
64         lose information.
65
66         * bmalloc/Sizes.h: Shifting is no longer used since we use division
67         instead.
68
69 2015-02-24  Stephanie Lewis  <slewis@apple.com>
70
71         Rolling out http://trac.webkit.org/changeset/180430 as it causes the PLT to crash.
72         <rdar://problem/19948015>
73
74         Unreviewed.
75
76         * bmalloc/VMHeap.cpp:
77         (bmalloc::VMHeap::grow):
78         * bmalloc/VMHeap.h:
79         * bmalloc/Zone.cpp:
80         (bmalloc::Zone::Zone):
81         (bmalloc::Zone::size): Deleted.
82         * bmalloc/Zone.h:
83
84 2015-02-24  Geoffrey Garen  <ggaren@apple.com>
85
86         bmalloc: Added a little more abstraction for large objects
87         https://bugs.webkit.org/show_bug.cgi?id=141978
88
89         Reviewed by Sam Weinig.
90
91         Previously, each client needed to manage the boundary tags of
92         a large object using free functions. This patch introduces a LargeObject
93         class that does things a little more automatically.
94
95         * bmalloc.xcodeproj/project.pbxproj:
96
97         * bmalloc/Allocator.cpp:
98         (bmalloc::Allocator::reallocate): Use the new LargeObject class.
99
100         * bmalloc/BeginTag.h:
101         (bmalloc::BeginTag::isInFreeList): Deleted. Moved this logic into the
102         LargeObject class.
103
104         * bmalloc/BoundaryTag.h:
105         (bmalloc::BoundaryTag::isSentinel):
106         (bmalloc::BoundaryTag::compactBegin):
107         (bmalloc::BoundaryTag::setRange):
108         (bmalloc::BoundaryTag::initSentinel): Added an explicit API for sentinels,
109         which we used to create and test for implicitly.
110
111         * bmalloc/BoundaryTagInlines.h:
112         (bmalloc::BoundaryTag::init):
113         (bmalloc::validate): Deleted.
114         (bmalloc::validatePrev): Deleted.
115         (bmalloc::validateNext): Deleted.
116         (bmalloc::BoundaryTag::mergeLeft): Deleted.
117         (bmalloc::BoundaryTag::mergeRight): Deleted.
118         (bmalloc::BoundaryTag::merge): Deleted.
119         (bmalloc::BoundaryTag::deallocate): Deleted.
120         (bmalloc::BoundaryTag::split): Deleted.
121         (bmalloc::BoundaryTag::allocate): Deleted. Moved this logic into the
122         LargeObject class.
123
124         * bmalloc/EndTag.h:
125         (bmalloc::EndTag::init):
126         (bmalloc::EndTag::operator=): Deleted. Re-reading this code, I found
127         special behavior in the assignment operator to be a surprising API.
128         So, I replaced the assignment operation with an explicit initializing
129         function.
130
131         * bmalloc/Heap.cpp:
132         (bmalloc::Heap::scavengeLargeRanges):
133         (bmalloc::Heap::allocateXLarge):
134         (bmalloc::Heap::findXLarge):
135         (bmalloc::Heap::deallocateXLarge):
136         (bmalloc::Heap::allocateLarge):
137         (bmalloc::Heap::deallocateLarge):
138         * bmalloc/Heap.h: No behavior changes here -- just adopting the
139         LargeObject interface.
140
141         * bmalloc/LargeObject.h: Added.
142         (bmalloc::LargeObject::operator!):
143         (bmalloc::LargeObject::begin):
144         (bmalloc::LargeObject::size):
145         (bmalloc::LargeObject::range):
146         (bmalloc::LargeObject::LargeObject):
147         (bmalloc::LargeObject::setFree):
148         (bmalloc::LargeObject::isFree):
149         (bmalloc::LargeObject::hasPhysicalPages):
150         (bmalloc::LargeObject::setHasPhysicalPages):
151         (bmalloc::LargeObject::isValidAndFree):
152         (bmalloc::LargeObject::merge):
153         (bmalloc::LargeObject::split):
154         (bmalloc::LargeObject::validateSelf):
155         (bmalloc::LargeObject::validate): Moved this code into a class, out of
156         BoundaryTag free functions.
157
158         New to the class are these features:
159
160             (1) Every reference to an object is validated upon creation and use.
161
162             (2) There's an explicit API for "This is a reference to an object
163             that might be stale (the DoNotValidate API)".
164
165             (3) The begin and end tags are kept in sync automatically.
166
167         * bmalloc/SegregatedFreeList.cpp:
168         (bmalloc::SegregatedFreeList::insert):
169         (bmalloc::SegregatedFreeList::takeGreedy):
170         (bmalloc::SegregatedFreeList::take):
171         * bmalloc/SegregatedFreeList.h: Adopt the LargeObject interface.
172
173         * bmalloc/VMHeap.cpp:
174         (bmalloc::VMHeap::grow):
175         * bmalloc/VMHeap.h:
176         (bmalloc::VMHeap::allocateLargeRange):
177         (bmalloc::VMHeap::deallocateLargeRange): Adopt the LargeObject interface.
178
179 2015-02-20  Geoffrey Garen  <ggaren@apple.com>
180
181         bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
182         https://bugs.webkit.org/show_bug.cgi?id=141802
183
184         Reviewed by Andreas Kling.
185
186         Rolling back in with a fix for a crash seen while using GuardMalloc.
187
188         * bmalloc/VMHeap.cpp:
189         (bmalloc::VMHeap::grow):
190         * bmalloc/VMHeap.h:
191         * bmalloc/Zone.cpp: Re-land the old patch.
192
193         (bmalloc::Zone::size): Be sure to implement the size() function since
194         it's accessible indirectly via the malloc_zone_from_ptr public API --
195         and GuardMalloc calls it all the time.
196
197         (bmalloc::Zone::Zone):
198         * bmalloc/Zone.h: Re-land the old patch.
199
200 2015-02-19  Commit Queue  <commit-queue@webkit.org>
201
202         Unreviewed, rolling out r180363.
203         https://bugs.webkit.org/show_bug.cgi?id=141814
204
205         Caused >50 crashes when running LayoutTests in GuardMalloc or
206         ASAN modes. (Requested by jernoble on #webkit).
207
208         Reverted changeset:
209
210         "bmalloc should implement malloc introspection (to stop false-
211         positive leaks when MallocStackLogging is off)"
212         https://bugs.webkit.org/show_bug.cgi?id=141802
213         http://trac.webkit.org/changeset/180363
214
215 2015-02-19  Geoffrey Garen  <ggaren@apple.com>
216
217         bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
218         https://bugs.webkit.org/show_bug.cgi?id=141802
219
220         Reviewed by Andreas Kling.
221
222         Fixed a last-minute type.
223
224         The macro is OS, not PLATFORM.
225
226         * bmalloc/VMHeap.cpp:
227         (bmalloc::VMHeap::grow):
228         * bmalloc/VMHeap.h:
229         * bmalloc/Zone.h:
230
231 2015-02-19  Geoffrey Garen  <ggaren@apple.com>
232
233         bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
234         https://bugs.webkit.org/show_bug.cgi?id=141802
235
236         Reviewed by Andreas Kling.
237
238         This patch does the bare minimum to stop false positive leaks from
239         being reported by the Darwin leaks tool. We register each super chunk
240         as a single object, and then request that the leaks tool scan it.
241
242         * bmalloc.xcodeproj/project.pbxproj: Added an abstraction for the malloc
243         zone introspection API.
244
245         * bmalloc/Algorithm.h: Missing #include.
246
247         * bmalloc/VMHeap.cpp:
248         (bmalloc::VMHeap::grow):
249         * bmalloc/VMHeap.h: Adopt the new abstraction.
250
251         * bmalloc/Zone.cpp: Added.
252         (bmalloc::remoteRead): Helper for reading an object out of another process.
253         (bmalloc::Zone::enumerator):
254         (bmalloc::Zone::Zone): Register a malloc zone so that we will participate
255         in introspection.
256
257         * bmalloc/Zone.h: Added.
258         (bmalloc::Zone::superChunks):
259         (bmalloc::Zone::addSuperChunk): Use a non-dynamically-allocated vector
260         since our dynamic allocations will not be scanned by leaks since they
261         will have the malloc VM tag.
262
263 2015-02-18  Geoffrey Garen  <ggaren@apple.com>
264
265         bmalloc: VMHeap should keep a record of all of its VM ranges (for malloc introspection)
266         https://bugs.webkit.org/show_bug.cgi?id=141759
267
268         Reviewed by Andreas Kling.
269
270         * bmalloc.xcodeproj/project.pbxproj:
271         * bmalloc/SuperChunk.h: Added.
272         (bmalloc::SuperChunk::create):
273         (bmalloc::SuperChunk::SuperChunk):
274         (bmalloc::SuperChunk::smallChunk):
275         (bmalloc::SuperChunk::mediumChunk):
276         (bmalloc::SuperChunk::largeChunk): Factored out super chunk creation
277         into a separate class, for clarity and type safety.
278
279         * bmalloc/VMHeap.cpp:
280         (bmalloc::VMHeap::grow):
281         (bmalloc::VMHeap::allocateSuperChunk): Renamed "allocateSuperChunk" to
282         "grow" because Andreas found "allocateSuperChunk" to be unclear.
283
284         * bmalloc/VMHeap.h: Track all our VM ranges. We will use this information
285         for malloc introspection.
286
287         (bmalloc::VMHeap::allocateSmallPage):
288         (bmalloc::VMHeap::allocateMediumPage):
289         (bmalloc::VMHeap::allocateLargeRange): Updated for renames.
290
291 2015-02-18  Zan Dobersek  <zdobersek@igalia.com>
292
293         Build bmalloc through CMake as a static library. It's then linked either
294         into the WTF library (if built as a shared library) or into the JSC and
295         WebKit2 libraries. There's no need to build it as a standalone shared library.
296
297         Rubber-stamped by Carlos Garcia Campos.
298
299         * CMakeLists.txt:
300
301 2015-02-13  Gyuyoung Kim  <gyuyoung.kim@samsung.com>
302
303         [BMalloc] Add a FIXME comment for memory alignas
304         https://bugs.webkit.org/show_bug.cgi?id=141556
305
306         Reviewed by Csaba Osztrogonác.
307
308         * bmalloc/Chunk.h: Add a FIXME comment.
309         * bmalloc/LargeChunk.h: ditto.
310
311 2015-02-11  Csaba Osztrogonác  <ossy@webkit.org>
312
313         bmalloc buildfix on 32 bit Linux (x86/ARM)
314         https://bugs.webkit.org/show_bug.cgi?id=141472
315
316         Reviewed by Gyuyoung Kim.
317
318         * bmalloc/Algorithm.h:
319         (bmalloc::roundUpToMultipleOf):
320         * bmalloc/FixedVector.h:
321         (bmalloc::FixedVector::clear):
322         * bmalloc/Sizes.h:
323         (bmalloc::Sizes::sizeClass):
324
325 2015-02-11  Gyuyoung Kim  <gyuyoung.kim@samsung.com>
326
327         [EFL][GTK] Use bmalloc instead of tcmalloc
328         https://bugs.webkit.org/show_bug.cgi?id=140162
329
330         Reviewed by Carlos Garcia Campos.
331
332         Support to use bmalloc on EFL and GTK ports.
333
334         * CMakeLists.txt: Added.
335         * bmalloc/Allocator.cpp:
336         (bmalloc::Allocator::allocate):
337             Fix unused return value caused by posix_memalign().
338         * bmalloc/AsyncTask.h:
339         * bmalloc/BoundaryTag.h:
340         (bmalloc::BoundaryTag::clear):
341         * bmalloc/Chunk.h:
342             Change Traits::Page with Traits::PageType in order to fix
343             -fpermitive build error on EFL and GTK port. 
344         * bmalloc/EndTag.h:
345         (bmalloc::EndTag::operator=):
346         * bmalloc/Line.h: ditto.
347         * bmalloc/MediumTraits.h:
348         * bmalloc/Page.h: ditto.
349         * bmalloc/PerThread.h:
350             EFL port doesn't support __has_include definition yet.
351             Define HAVE_PTHREAD_MACHDEP_H according to check if __has_include is supported.
352         * bmalloc/SmallTraits.h: ditto.
353         * bmalloc/VMAllocate.h:
354         (bmalloc::vmDeallocatePhysicalPages):
355         (bmalloc::vmAllocatePhysicalPages):
356         * bmalloc/Vector.h:
357         (bmalloc::Vector<T>::push):
358         (bmalloc::Vector<T>::reallocateBuffer):
359
360 2015-01-31  Sam Weinig  <sam@webkit.org>
361
362         Remove even more Mountain Lion support
363         https://bugs.webkit.org/show_bug.cgi?id=141124
364
365         Reviewed by Alexey Proskuryakov.
366
367         * Configurations/Base.xcconfig:
368         * Configurations/DebugRelease.xcconfig:
369
370 2015-01-30  Geoffrey Garen  <ggaren@apple.com>
371
372         GC marking threads should clear malloc caches
373         https://bugs.webkit.org/show_bug.cgi?id=141097
374
375         Reviewed by Andreas Kling.
376
377         Split the scavenging API into per-thread vs global, so that you can
378         request to scavenge your own thread without scavenging the whole heap.
379
380         * bmalloc/Cache.cpp:
381         (bmalloc::Cache::scavenge):
382         * bmalloc/bmalloc.h:
383         (bmalloc::api::scavengeThisThread):
384         (bmalloc::api::scavenge):
385
386 2015-01-28  Dana Burkart  <dburkart@apple.com>
387
388         Move ASan flag settings from DebugRelease.xcconfig to Base.xcconfig
389         https://bugs.webkit.org/show_bug.cgi?id=136765
390
391         Reviewed by Alexey Proskuryakov.
392
393         * Configurations/Base.xcconfig:
394         * Configurations/DebugRelease.xcconfig:
395
396 2015-01-21  Geoffrey Garen  <ggaren@apple.com>
397
398         bmalloc: support aligned allocation
399         https://bugs.webkit.org/show_bug.cgi?id=140732
400
401         Reviewed by Andreas Kling.
402
403         * bmalloc/Allocator.cpp:
404         (bmalloc::Allocator::allocate): New function for aligned allocation.
405
406         Small and medium requests just allocate and free until they find an
407         aligned pointer. This is slightly inefficient in the worst case, but
408         still constant-time with little-to-no space overhead.
409
410         Large requests use a new API that requires the client to specify both
411         its ideal size and alignment, and the worst-case size you would have to
412         allocate in order to produce some interior pointer of the requested size
413         and alignment. We put the burden of this calculation on the client
414         because it simplifies things if we guarantee that allocation won't fail.
415
416         XLarge requests are easy: we just forward them to vmAllocate, which
417         already supported aligned requests.
418
419         * bmalloc/BoundaryTag.h:
420         * bmalloc/BoundaryTagInlines.h:
421         (bmalloc::BoundaryTag::mergeLeft):
422         (bmalloc::BoundaryTag::mergeRight):
423         (bmalloc::BoundaryTag::merge):
424         (bmalloc::BoundaryTag::deallocate):
425         (bmalloc::BoundaryTag::split):
426         (bmalloc::BoundaryTag::allocate): No behavior change here. I just
427         refactored the interface to remove some reference out parameters in
428         order to clarify what changes and what doesn't.
429
430         * bmalloc/Heap.cpp:
431         (bmalloc::Heap::allocateXLarge): Added an alignment API.
432
433         (bmalloc::Heap::allocateLarge):
434         * bmalloc/Heap.h: Added an alignment API. I split out allocateLarge into
435         a few variants, so aligned and unaligned allocation could share some code.
436
437         * bmalloc/SegregatedFreeList.cpp:
438         (bmalloc::SegregatedFreeList::take):
439         * bmalloc/SegregatedFreeList.h: Changed to use a separate, explicit API
440         for aligned allocation. It turns out that the aligned path is pretty
441         different, since it ends up searching for two potential ways to satisfy
442         an allocation: either large enough and aligned, or large enough to split
443         into something not aligned and something large enough and aligned.
444
445         * bmalloc/VMAllocate.h:
446         (bmalloc::vmAllocate): Switched alignment to come before size because
447         that's how the memalign API specifies it.
448
449         * bmalloc/VMHeap.h:
450         (bmalloc::VMHeap::allocateLargeRange): Added an alignment API.
451
452 2015-01-20  Geoffrey Garen  <ggaren@apple.com>
453
454         bmalloc: a little bit of cleanup
455         https://bugs.webkit.org/show_bug.cgi?id=140687
456
457         Reviewed by Anders Carlsson.
458
459         * bmalloc/Algorithm.h:
460         (bmalloc::isPowerOfTwo): Added a check for 0, since 0 would break a lot
461         of code.
462
463         * bmalloc/BoundaryTag.h:
464         * bmalloc/BoundaryTagInlines.h:
465         (bmalloc::BoundaryTag::mergeLeft):
466         (bmalloc::BoundaryTag::mergeRight):
467         (bmalloc::BoundaryTag::merge):
468         (bmalloc::BoundaryTag::deallocate):
469         (bmalloc::BoundaryTag::split):
470         (bmalloc::BoundaryTag::allocate):
471         (bmalloc::BoundaryTag::mergeLargeLeft): Deleted.
472         (bmalloc::BoundaryTag::mergeLargeRight): Deleted.
473         (bmalloc::BoundaryTag::mergeLarge): Deleted.
474         (bmalloc::BoundaryTag::splitLarge): Deleted. Removed the word "Large"
475         from all these functions, since boundary tags always pertain to large
476         objects, and putting the word "Large" everywhere wasn't helping to
477         explain that.
478
479         * bmalloc/Heap.cpp:
480         (bmalloc::Heap::allocateXLarge):
481         (bmalloc::Heap::findXLarge):
482         (bmalloc::Heap::deallocateXLarge):
483         * bmalloc/Heap.h:
484         * bmalloc/VMHeap.h:
485         (bmalloc::VMHeap::allocateXLarge): Deleted.
486         (bmalloc::VMHeap::findXLarge): Deleted.
487         (bmalloc::VMHeap::deallocateXLarge): Deleted. Moved XLarge allocation
488         from VMHeap to Heap. Since the purpose of the VMHeap is to cache VM
489         ranges, and the VMHeap never caches any XLarge ranges, it doesn't
490         really make sense for the VMHeap to be involved.
491
492 2015-01-16  Geoffrey Garen  <ggaren@apple.com>
493
494         bmalloc: refactored XLarge allocation for better alignment
495         https://bugs.webkit.org/show_bug.cgi?id=140582
496
497         Reviewed by Andreas Kling.
498
499         XLarge objects used to be Large objects with an extra bit of metadata
500         that said "actually, I'm not large -- I'm extra large".
501
502         The metadata header in an XLarge allocation made it impossible for the
503         XLarge object to honor a very large alignment request.
504
505         The solution is to stop using a metadata header for XLarge objects, and
506         instead to store explicit metadata on the side.
507
508         This is a bit less astonishing, which is also nice.
509
510         Finding XLarge metadata is now a linear search. That's probably OK, since
511         it was always so in TCMalloc, and the usual number of XLarge allocations
512         in a process is 0.
513
514         This design makes it possible for the heap to cache XLarge allocations
515         with and/or without physical pages. I haven't actually done that yet
516         because the tradeoffs are subtle, so I don't want to do anything without
517         a motivating test case.
518
519         * bmalloc.xcodeproj/project.pbxproj:
520         * bmalloc/Allocator.cpp:
521         (bmalloc::Allocator::reallocate): Removed the concept of an XLargeChunk,
522         since an XLarge allocation is now just a naked buffer without a header.
523
524         (bmalloc::Allocator::allocateXLarge): Added an explicit qualifier for
525         XLarge alignment, since XLargeChunk won't give this to us implicitly
526         anymore.
527
528         * bmalloc/BoundaryTag.h:
529         (bmalloc::BoundaryTag::setRange):
530         (bmalloc::BoundaryTag::isXLarge): Deleted.
531         (bmalloc::BoundaryTag::setXLarge): Deleted.
532         * bmalloc/BoundaryTagInlines.h:
533         (bmalloc::validate):
534         (bmalloc::BoundaryTag::deallocate): Removed the XLarge hacks from Large allocations.
535
536         * bmalloc/Deallocator.cpp:
537         (bmalloc::Deallocator::deallocateXLarge):
538         (bmalloc::Deallocator::deallocateSlowCase):
539         * bmalloc/Heap.cpp:
540         (bmalloc::Heap::findXLarge):
541         (bmalloc::Heap::allocateXLarge):
542         (bmalloc::Heap::deallocateXLarge):
543         * bmalloc/Heap.h: Updated for interface changes.
544
545         * bmalloc/ObjectType.cpp:
546         (bmalloc::objectType):
547         * bmalloc/ObjectType.h:
548         (bmalloc::isXLarge): We can now tell if a pointer is XLarge just by
549         examining its bit pattern -- just like we do for other kinds of
550         allocations -- which is nice.
551
552         * bmalloc/Sizes.h:
553         * bmalloc/VMHeap.h:
554         (bmalloc::VMHeap::allocateXLarge):
555         (bmalloc::VMHeap::findXLarge):
556         (bmalloc::VMHeap::deallocateXLarge): Keep an explicit vector of metadata
557         for XLarge allocations.
558
559         * bmalloc/XLargeChunk.h: Removed.
560
561 2015-01-16  Geoffrey Garen  <ggaren@apple.com>
562
563         bmalloc: added some infrastructure for aligned allocation
564         https://bugs.webkit.org/show_bug.cgi?id=140572
565
566         Reviewed by Andreas Kling.
567
568         * bmalloc/Algorithm.h:
569         (bmalloc::isPowerOfTwo):
570         (bmalloc::roundUpToMultipleOf):
571         (bmalloc::roundDownToMultipleOf): Refactored some duplicate code to use our
572         isPowerOfTwo helper function.
573
574         * bmalloc/Allocator.cpp:
575         (bmalloc::Allocator::allocate):
576         * bmalloc/Allocator.h: Stubbed out an implementation of aligned allocation.
577         Doesn't do anything yet, but does correctly forward to system malloc
578         when bmalloc is disabled.
579
580         * bmalloc/Cache.cpp:
581         (bmalloc::Cache::allocateSlowCaseNullCache):
582         * bmalloc/Cache.h:
583         (bmalloc::Cache::allocate):
584         * bmalloc/bmalloc.h:
585         (bmalloc::api::memalign):
586         * bmalloc/mbmalloc.cpp: Stubbed out an API for aligned allocation.
587
588 2015-01-13  Geoffrey Garen  <ggaren@apple.com>
589
590         Consider alignment when allocating from a SegregatedFreeList
591         https://bugs.webkit.org/show_bug.cgi?id=140408
592
593         Reviewed by Sam Weinig.
594
595         In preparation for supporting aligned allocation.
596
597         No performance change.
598
599         Since this is just one extra branch in an already expensive function,
600         I decided not to duplicate the function just to avoid the branch in
601         the un-aligned case.
602
603         * bmalloc/SegregatedFreeList.cpp:
604         (bmalloc::SegregatedFreeList::take):
605         * bmalloc/SegregatedFreeList.h:
606
607 2015-01-13  Geoffrey Garen  <ggaren@apple.com>
608
609         Renamed minimum to size in SegregatedFreeList
610         https://bugs.webkit.org/show_bug.cgi?id=140406
611
612         Reviewed by Sam Weinig.
613
614         In preparation for supporting aligned allocation.
615
616         * bmalloc/SegregatedFreeList.cpp:
617         (bmalloc::SegregatedFreeList::takeGreedy):
618         (bmalloc::SegregatedFreeList::take): Every size passed to malloc is
619         really just a minimum. Let's not imply that this value is special.
620
621 2015-01-11  Dan Bernstein  <mitz@apple.com>
622
623         Geoff is organized, but he is not an organization.
624
625         Rubber-stamped by Anders Carlsson.
626
627         * bmalloc.xcodeproj/project.pbxproj: Removed the ORGANIZATIONNAME project attribute.
628
629 2015-01-07  Geoffrey Garen  <ggaren@apple.com>
630
631         Make bmalloc work with ASan
632         https://bugs.webkit.org/show_bug.cgi?id=140194
633
634         Reviewed by Mark Lam.
635
636         * bmalloc/BPlatform.h: Added a way to detect Darwin OSes, since we need
637         an OS-specific API to test for loaded runtime libraries.
638
639         * bmalloc/Environment.cpp:
640         (bmalloc::isASanEnabled):
641         (bmalloc::Environment::computeIsBmallocEnabled): Disabled bmalloc if
642         ASan is enabled, since system malloc has the Asan hooks we need.
643
644         You could check for the ASan compile-time flag instead, but doing this
645         check at runtime prepares bmalloc for a world where it is a dynamic
646         library that might be loaded into projects it did not compile with.
647
648 2015-01-05  Geoffrey Garen  <ggaren@apple.com>
649
650         Fix up bmalloc's PerThread for use on Linux
651         https://bugs.webkit.org/show_bug.cgi?id=139804
652
653         Reviewed by Anders Carlsson.
654
655         The previous implementation was a bit slow.
656
657         * bmalloc/PerThread.h:
658         (bmalloc::PerThreadStorage<Cache>::get):
659         (bmalloc::PerThreadStorage::get):
660         (bmalloc::PerThreadStorage::init): Added a catch-all cross-platform Unix
661         way to do fast per-thread access without taking a lock every time. This
662         probably works on all the platforms we care about, and it matches other
663         techniques we use elsewhere in WebKit.
664
665         (bmalloc::PerThread<T>::getFastCase): Removed the conditional from
666         this class because PerThreadStorage now encapsulates everything that
667         needs to be conditional.
668
669         (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Deleted.
670
671 2014-12-26  Dan Bernstein  <mitz@apple.com>
672
673         <rdar://problem/19348208> REGRESSION (r177027): iOS builds use the wrong toolchain
674         https://bugs.webkit.org/show_bug.cgi?id=139950
675
676         Reviewed by David Kilzer.
677
678         * Configurations/Base.xcconfig: Only define TOOLCHAINS when building for OS X, doing so
679         in a manner that works with Xcode 5.1.1.
680
681 2014-12-15  Geoffrey Garen  <ggaren@apple.com>
682
683         Safari crashes when you set Malloc environment variables
684         https://bugs.webkit.org/show_bug.cgi?id=139656
685
686         Reviewed by Michael Saboff.
687
688         I forgot to cover the realloc() case. Whoops. (OoPS?)
689
690         This time around, I ran the full MallocBench test suite in Malloc=1
691         mode, and it passed.
692
693         * bmalloc/Allocator.cpp:
694         (bmalloc::Allocator::reallocate):
695         * bmalloc/Allocator.h: Pushed realloc() logic down into the allocator.
696         It needs to be down there so that we can do the short-circuiting check
697         for whether bmalloc is enabled first.
698
699         Also added the check.
700
701         * bmalloc/Cache.cpp:
702         (bmalloc::Cache::scavenge):
703         (bmalloc::Cache::Cache):
704         (bmalloc::Cache::reallocateSlowCaseNullCache):
705         * bmalloc/Cache.h:
706         (bmalloc::Cache::deallocator):
707         (bmalloc::Cache::reallocate): Ditto.
708
709         * bmalloc/bmalloc.h:
710         (bmalloc::api::free):
711         (bmalloc::api::realloc): Ditto.
712
713         (bmalloc::api::scavenge): Pushed this down into Cache to match the
714         surrounding functions.
715
716 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
717
718         bmalloc should support system memory analysis tools (part 2)
719         https://bugs.webkit.org/show_bug.cgi?id=139565
720
721         Reviewed by Mark Lam.
722
723         This patch actually queries the environment to see if memory analysis
724         tools have been enabled.
725
726         * bmalloc/Deallocator.cpp:
727         (bmalloc::Deallocator::scavenge): Don't process the object log if
728         we've disabled bmalloc because it will be full of invalid nullptrs.
729
730         * bmalloc/Environment.cpp:
731         (bmalloc::isMallocEnvironmentVariableSet): Test for the list of known
732         Malloc debugging flags. I also added a plain "Malloc" catch-all for
733         when you want to disable bmalloc without enabling any kind of funny
734         business.
735
736         It would be slightly nicer just to iterate the list of environment
737         variables and strstr them, but getenv is the more portable option,
738         and performance here doesn't really matter.
739
740         (bmalloc::isLibgmallocEnabled): Test for the libgmalloc insertion
741         environment variable.
742
743         (bmalloc::Environment::computeIsBmallocEnabled):
744
745 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
746
747         Try to fix the iOS simulator build.
748
749         #include the declaration of malloc / free.
750
751         * bmalloc/Allocator.cpp:
752         * bmalloc/Deallocator.cpp:
753
754 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
755
756         Try to fix the build.
757
758         * bmalloc.xcodeproj/project.pbxproj: Marked a header exported.
759
760 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
761
762         bmalloc should support system memory analysis tools (part 1)
763         https://bugs.webkit.org/show_bug.cgi?id=139559
764
765         Reviewed by Mark Lam.
766
767         This patch adds the hooks to disable bmalloc at runtime if certain
768         environment variables are set, but doesn't actually read from the
769         environment yet.
770
771         No performance change.
772
773         * bmalloc.xcodeproj/project.pbxproj: Added the Environment class, which
774         we'll use to read environment variables and see if memory analysis tools
775         have been enabled.
776
777         * bmalloc/Allocator.cpp:
778         (bmalloc::Allocator::Allocator):
779         (bmalloc::Allocator::allocateSlowCase): Added a hook to disable bmalloc
780         on the allocation path. We cache the setting to make the check fast.
781
782         * bmalloc/Allocator.h: Interface changes.
783
784         * bmalloc/Cache.cpp:
785         (bmalloc::Cache::Cache): Pass a heap pointer through to our allocator
786         and deallocator. This main purpose is to enable them to query the
787         environment for whether bmalloc is enabled; but this is also a slightly
788         cleaner way to guarantee to them that the Heap has been pre-initialized.
789
790         * bmalloc/Deallocator.cpp:
791         (bmalloc::Deallocator::Deallocator): If bmalloc is disable, artificially
792         fill the object log to force us to take the slow path on all deallocations.
793
794         (bmalloc::Deallocator::deallocateSlowCase): Do the disabled check.
795
796         * bmalloc/Deallocator.h: Interface changes.
797
798         * bmalloc/Environment.cpp: Added.
799         (bmalloc::Environment::Environment):
800         (bmalloc::Environment::computeIsBmallocEnabled):
801         * bmalloc/Environment.h: Added.
802         (bmalloc::Environment::isBmallocEnabled): This is the class that will
803         encapsulate looking for environment variables that turn on heap
804         analysis tools.
805
806         * bmalloc/Heap.h:
807         (bmalloc::Heap::environment):
808
809         * bmalloc/Mutex.h:
810         (bmalloc::Mutex::Mutex):
811         * bmalloc/StaticMutex.h: A little refactoring to clarify these comments,
812         since I got super confused about them while writing this patch.
813
814         * bmalloc/VMHeap.cpp: Fixed an #include.
815
816 2014-12-09  David Kilzer  <ddkilzer@apple.com>
817
818         Switch from using PLATFORM_NAME to SDK selectors in ANGLE, bmalloc, gtest, JavaScriptCore, WTF
819         <http://webkit.org/b/139212>
820
821         Reviewed by Joseph Pecoraro.
822
823         * Configurations/Base.xcconfig:
824         - Only set GCC_ENABLE_OBJC_GC, GCC_MODEL_TUNING and TOOLCHAINS
825           on OS X.
826         * Configurations/DebugRelease.xcconfig:
827         - Only set MACOSX_DEPLOYMENT_TARGET and SDKROOT on OS X.
828
829 2014-11-07  Geoffrey Garen  <ggaren@apple.com>
830
831         bmalloc uses 8X more virtual memory than necessary
832         https://bugs.webkit.org/show_bug.cgi?id=138495
833
834         Reviewed by Mark Lam.
835
836         iOS has a per-process virtual memory cap around 1GB, so there's some
837         value to not going totally ham with virtual memory.
838
839         We currently use about 8X the necessary amount:
840             - 2X to align our VM allocation
841             - 4X to reserve small / medium / (2) large chunk VM ranges per superchunk
842
843         We can cut that down:
844             - Return the unaligned portion of our VM allocation (-2X)
845             - Use all the chunks in a superchunk, instead of allocating one
846               chunk per superchunk (-4X)
847
848         * bmalloc/Algorithm.h:
849         (bmalloc::roundUpToMultipleOf): Added a non-constant version of this
850         function so we can call it with getpagesize() at runtime.
851
852         * bmalloc/Chunk.h:
853         * bmalloc/LargeChunk.h:
854         (bmalloc::LargeChunk::create): Deleted. Instead of each chunk allocating
855         its own VM, VMHeap allocates the superchunk and all the chunks in it at a time.
856
857         * bmalloc/VMAllocate.h:
858         (bmalloc::vmValidate):
859         (bmalloc::vmAllocate): ASSERT that mmap succeeds to make crashes clearer
860         if it does not succeed. Allocate precisely, and give back the extra.
861
862         * bmalloc/VMHeap.cpp:
863         (bmalloc::VMHeap::allocateSuperChunk):
864         (bmalloc::VMHeap::allocateSmallChunk): Deleted.
865         (bmalloc::VMHeap::allocateMediumChunk): Deleted.
866         (bmalloc::VMHeap::allocateLargeChunk): Deleted. Use all the chunks
867         in a superchunk, instead of just one.
868
869         * bmalloc/VMHeap.h:
870         (bmalloc::VMHeap::allocateSmallPage):
871         (bmalloc::VMHeap::allocateMediumPage):
872         (bmalloc::VMHeap::allocateLargeRange):
873         * bmalloc/XLargeChunk.h:
874         (bmalloc::XLargeChunk::create): Updated to match changes above.
875
876 2014-11-01  David Kilzer  <ddkilzer@apple.com>
877
878         JavaScriptCore is missing debug info for bmalloc because libbmalloc.a is stripped
879         <https://webkit.org/b/138286>
880         <rdar://problem/18847087>
881
882         Reviewed by Dan Bernstein.
883
884         * Configurations/bmalloc.xcconfig: Set STRIP_INSTALLED_PRODUCT
885         to NO for the target that produces libbmalloc.a so that the
886         debug symbols will be linked into JavaScriptCore and end up in
887         its dSYM file.
888
889 2014-10-30  Dana Burkart  <dburkart@apple.com>
890
891         <rdar://problem/18821260> Prepare for the mysterious future
892
893         Reviewed by Lucas Forschler.
894
895         * Configurations/Base.xcconfig:
896         * Configurations/DebugRelease.xcconfig:
897
898 2014-09-24  Geoffrey Garen  <ggaren@apple.com>
899
900         bmalloc: cleaned up fast path vs slow path
901         https://bugs.webkit.org/show_bug.cgi?id=137081
902
903         Reviewed by Sam Weinig.
904
905         Might be a 1% speedup on MallocBench. Also cleans up the code a bit.
906
907         * bmalloc/Allocator.cpp:
908         (bmalloc::Allocator::Allocator): Merged the small and medium range
909         caches, just like the small and medium allocators. Ranges are abstract
910         objects that don't really care whether they hold small or medium objects,
911         so they don't need to be segregated.
912
913         (bmalloc::Allocator::scavenge): Ditto.
914
915         (bmalloc::Allocator::allocateBumpRangeSlowCase):
916         (bmalloc::Allocator::allocateBumpRange): Same thing here, except that
917         we do care a tiny bit, because we need to specify small vs medium when
918         allocating new ranges from the heap, to ensure that the heap allocates
919         from the right segment of VM.
920
921         (bmalloc::Allocator::allocateLarge):
922         (bmalloc::Allocator::allocateXLarge): NO_INLINE because this was clouding
923         up the fast path. Large allocation performance is dominated by allocation
924         logic and initialization, so inlining it doesn't help.
925
926         (bmalloc::Allocator::allocateSlowCase): Slow path got a bit cleaner since
927         it doesn't need to distinguish small vs medium objects.
928
929         (bmalloc::Allocator::allocateSmallBumpRange): Deleted.
930         (bmalloc::Allocator::allocateMediumBumpRange): Deleted.
931
932         * bmalloc/Allocator.h:
933         * bmalloc/BumpRange.h:
934
935         * bmalloc/Cache.cpp:
936         (bmalloc::Cache::allocateSlowCase): Deleted.
937         (bmalloc::Cache::deallocateSlowCase): Deleted.
938         * bmalloc/Cache.h:
939         (bmalloc::Cache::allocate):
940         (bmalloc::Cache::deallocate):
941         (bmalloc::Cache::allocateFastCase): Deleted.
942         (bmalloc::Cache::deallocateFastCase): Deleted. Removed the Cache slow
943         paths. The downside to this change is that the fast path branches to two
944         distinct failure cases instead of one. The upside is that the slow path
945         doesn't need to re-read the segment register, which is not as cheap as a
946         normal register, and it doesn't need to do an extra level of function 
947         call. Seems to be worth it.
948
949         * bmalloc/Deallocator.h:
950         * bmalloc/Heap.cpp:
951         (bmalloc::Heap::refillSmallBumpRangeCache):
952         (bmalloc::Heap::refillMediumBumpRangeCache):
953         * bmalloc/Heap.h: Updated for interface changes.
954
955         * bmalloc/Sizes.h: The most ranges a cache will hold is the number of
956         small lines in a page / 2, since any other free lines will coalesce
957         with their neighbors.
958
959 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
960
961         Rolled out r173346.
962
963             bmalloc should honor the FastMalloc statistics API
964             https://bugs.webkit.org/show_bug.cgi?id=136592
965
966         This didn't really work. Because we allow ranges with and without
967         physical pages to merge, and we allow double-committing and
968         double-decommitting, we can't rely on commit actions to track memory
969         footprint.
970
971         * bmalloc/Heap.cpp:
972         (bmalloc::Heap::size): Deleted.
973         (bmalloc::Heap::capacity): Deleted.
974         * bmalloc/Heap.h:
975         * bmalloc/VMHeap.cpp:
976         (bmalloc::VMHeap::VMHeap):
977         (bmalloc::VMHeap::allocateSmallChunk):
978         (bmalloc::VMHeap::allocateMediumChunk):
979         (bmalloc::VMHeap::allocateLargeChunk):
980         * bmalloc/VMHeap.h:
981         (bmalloc::VMHeap::allocateSmallPage):
982         (bmalloc::VMHeap::allocateMediumPage):
983         (bmalloc::VMHeap::allocateLargeRange):
984         (bmalloc::VMHeap::deallocateSmallPage):
985         (bmalloc::VMHeap::deallocateMediumPage):
986         (bmalloc::VMHeap::deallocateLargeRange):
987         (bmalloc::VMHeap::size): Deleted.
988         (bmalloc::VMHeap::capacity): Deleted.
989         * bmalloc/bmalloc.h:
990         (bmalloc::api::heapSize): Deleted.
991         (bmalloc::api::heapCapacity): Deleted.
992
993 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
994
995         bmalloc: Allocation should be more precise
996         https://bugs.webkit.org/show_bug.cgi?id=136993
997
998         Reviewed by Gavin Barraclough.
999
1000         13% reduction in heap size on the MallocBench *_memory_warning benchmarks.
1001
1002         This patch teaches the allocator to merge adjacent free lines into a
1003         single allocatable range. This allows us to shrink the size of an
1004         individual line without increasing fragmentation or the rate of allocator
1005         slow paths.
1006
1007         We'll only take more slow paths when available memory is sparse, which
1008         is exactly when it's worth it. When available memory is dense, we'll
1009         take fewer slow paths.
1010
1011         * bmalloc.xcodeproj/project.pbxproj:
1012         * bmalloc/Algorithm.h:
1013         (bmalloc::divideRoundingUp):
1014
1015         * bmalloc/Allocator.cpp:
1016         (bmalloc::Allocator::Allocator): Updated for interface changes.
1017
1018         (bmalloc::Allocator::scavenge): Scavenge by object instead of by line.
1019         Now that we merge lines, it's not convenient to scavenge by line.
1020
1021         (bmalloc::Allocator::allocateSmallBumpRange):
1022         (bmalloc::Allocator::allocateMediumBumpRange): Allocate whole ranges
1023         instead of individual lines.
1024
1025         (bmalloc::Allocator::allocateSlowCase):
1026         (bmalloc::Allocator::allocateSmallLine): Deleted.
1027         (bmalloc::Allocator::allocateMediumLine): Deleted.
1028         (bmalloc::Allocator::allocateMedium): Deleted.
1029         * bmalloc/Allocator.h:
1030         (bmalloc::Allocator::allocateFastCase): Folded medium allocations
1031         into the standard fast path with small allocations. Since a BumpAllocator
1032         just allocates out of an arbitrary range, it doesn't need to distinguish
1033         between small and medium lines.
1034
1035         * bmalloc/BumpAllocator.h:
1036         (bmalloc::BumpAllocator::size):
1037         (bmalloc::BumpAllocator::BumpAllocator):
1038         (bmalloc::BumpAllocator::init):
1039         (bmalloc::BumpAllocator::refill):
1040         (bmalloc::BumpAllocator::line): Deleted. No need to track line information
1041         anymore: the heap just gives us a pointer and a pre-computed number of
1042         objects, and we allocate them.
1043
1044         * bmalloc/Deallocator.cpp:
1045         (bmalloc::Deallocator::processObjectLog): Updated for interface changes.
1046
1047         * bmalloc/Heap.cpp:
1048         (bmalloc::Heap::Heap):
1049         (bmalloc::Heap::initializeLineMetadata): Pre-compute precise metadata
1050         detailing where all objects will lie in memory. After we merge two lines,
1051         we might allocate an object that spans from one line to the next. This
1052         metadata details which bits of memory overlap in that way, and how they
1053         overlap.
1054
1055         (bmalloc::Heap::refillSmallBumpRangeCache):
1056         (bmalloc::Heap::refillMediumBumpRangeCache): Scan a whole page at a time,
1057         and merge adjacent free lines into BumpRanges.
1058
1059         (bmalloc::Heap::allocateSmallPage):
1060         (bmalloc::Heap::allocateMediumPage):
1061         (bmalloc::Heap::deallocateSmallLine):
1062         (bmalloc::Heap::deallocateMediumLine): Track pages rather than lines,
1063         since we scan for free memory a page at a time.
1064
1065         (bmalloc::Heap::allocateSmallLineSlowCase): Deleted.
1066         (bmalloc::Heap::allocateMediumLineSlowCase): Deleted. Folded into the
1067         fast path.
1068
1069         * bmalloc/Heap.h:
1070         (bmalloc::Heap::derefSmallLine):
1071         (bmalloc::Heap::derefMediumLine):
1072         (bmalloc::Heap::deallocateSmallLine): Deleted.
1073         (bmalloc::Heap::allocateSmallLine): Deleted.
1074         (bmalloc::Heap::deallocateMediumLine): Deleted.
1075         (bmalloc::Heap::allocateMediumLine): Deleted. Updated for interface changes.
1076
1077         * bmalloc/Line.h:
1078         (bmalloc::Line<Traits>::ref):
1079         (bmalloc::Line<Traits>::deref):
1080         (bmalloc::Line<Traits>::concurrentRef): Deleted. We don't pass a derefCount
1081         anymore, since we only ever deref by 1 now.
1082
1083         * bmalloc/MediumAllocator.h:
1084         (bmalloc::MediumAllocator::isNull): Deleted.
1085         (bmalloc::MediumAllocator::MediumAllocator): Deleted.
1086         (bmalloc::MediumAllocator::line): Deleted.
1087         (bmalloc::MediumAllocator::allocate): Deleted.
1088         (bmalloc::MediumAllocator::derefCount): Deleted.
1089         (bmalloc::MediumAllocator::refill): Deleted.
1090         (bmalloc::MediumAllocator::clear): Deleted. Deleted some code that's
1091         been dead for a while, since it doesn't build anymore with this patch.
1092
1093         * bmalloc/Page.h:
1094         (bmalloc::Page::sizeClass):
1095         (bmalloc::Page::setSizeClass):
1096         (bmalloc::Page::smallSizeClass): Deleted.
1097         (bmalloc::Page::setSmallSizeClass): Deleted. Renamed setSmallSizeClass
1098         to sizeClass, since we use it for medium sizes too.
1099
1100         * bmalloc/Sizes.h:
1101         (bmalloc::Sizes::sizeClass):
1102         (bmalloc::Sizes::objectSize): Shrank line sizes to save memory.
1103
1104         (bmalloc::Sizes::smallSizeClassFor): Deleted.
1105         (bmalloc::Sizes::mediumSizeClassFor): Deleted.
1106
1107         * bmalloc/bmalloc.h:
1108         (bmalloc::api::realloc): Now that we have precise objects sizes, realloc
1109         can be a bit more precise. It also has to be, since we can't guarantee
1110         that an object ends at the end of a line anymore.
1111
1112 2014-09-19  Daniel Bates  <dabates@apple.com>
1113
1114         Always assume internal SDK when building configuration Production
1115         https://bugs.webkit.org/show_bug.cgi?id=136925
1116         <rdar://problem/18362399>
1117
1118         Reviewed by Dan Bernstein.
1119
1120         * Configurations/Base.xcconfig:
1121
1122 2014-09-16  Geoffrey Garen  <ggaren@apple.com>
1123
1124         bmalloc: moved line caches from the deallocator to the allocator
1125         https://bugs.webkit.org/show_bug.cgi?id=136868
1126
1127         Reviewed by Gavin Barraclough.
1128
1129         I did this mostly as a simplification, to make it easier to change the
1130         allocation strategy.
1131
1132         No throughput change on MallocBench. Saves about 50kB.
1133
1134         Since the deallocator needs to lock the heap when freeing lines anyway,
1135         there isn't much benefit to giving the deallocator a local cache of
1136         deallocated lines.
1137
1138         We still give the allocator a local cache of lines because that does
1139         reduce the frequency at which it needs to lock the heap in order to
1140         acquire more lines.
1141
1142         * bmalloc/Allocator.cpp:
1143         (bmalloc::Allocator::scavenge):
1144         (bmalloc::Allocator::allocateSmallLine):
1145         (bmalloc::Allocator::allocateMediumLine):
1146         (bmalloc::Allocator::allocateMedium):
1147         (bmalloc::Allocator::allocateSlowCase):
1148         * bmalloc/Allocator.h:
1149         * bmalloc/Deallocator.cpp:
1150         (bmalloc::Deallocator::Deallocator):
1151         (bmalloc::Deallocator::scavenge):
1152         (bmalloc::Deallocator::processObjectLog):
1153         (bmalloc::Deallocator::deallocateSmallLine): Deleted.
1154         (bmalloc::Deallocator::allocateSmallLine): Deleted.
1155         (bmalloc::Deallocator::deallocateMediumLine): Deleted.
1156         (bmalloc::Deallocator::allocateMediumLine): Deleted.
1157         * bmalloc/Deallocator.h:
1158
1159         * bmalloc/Sizes.h:
1160         * bmalloc/VMAllocate.h: Took the opportunity to make the line cache size
1161         exactly one page in size. That's about what we were shooting for anyway,
1162         and it may make it easier to switch to per-page allocation in future.
1163
1164 2014-09-15  Geoffrey Garen  <ggaren@apple.com>
1165
1166         bmalloc: allocate small and medium objects using the same bump pointer class
1167         https://bugs.webkit.org/show_bug.cgi?id=136843
1168
1169         Reviewed by Gavin Barraclough.
1170
1171         4% speedup on MallocBench.
1172
1173         Now that medium-sized objects have dedicated per-size allocators, they
1174         don't need to use an arbitrary bump pointer allocator. This means that
1175         every allocator knows how many objects it will allocate from the start,
1176         and we don't need a post-processing step to adjust refcounts based on
1177         real allocation count.
1178
1179         * bmalloc.xcodeproj/project.pbxproj: Renamed SmallAllocator to BumpAllocator
1180         since it's used for small and medium objects now.
1181
1182         * bmalloc/Allocator.cpp:
1183         (bmalloc::Allocator::Allocator): Updated to use new interface.
1184         (bmalloc::Allocator::scavenge): To "retire" an allocator, we just need
1185         to make sure that we finish allocating all the objects in it.
1186
1187         (bmalloc::Allocator::allocateMedium):
1188         (bmalloc::Allocator::allocateSlowCase):
1189         (bmalloc::Allocator::retire): Deleted.
1190         (bmalloc::Allocator::processSmallAllocatorLog): Deleted.
1191         (bmalloc::Allocator::processMediumAllocatorLog): Deleted.
1192         * bmalloc/Allocator.h:
1193         (bmalloc::Allocator::allocateFastCase): Removed abstractions and data
1194         used to post-process an allocator based on how many objects it allocated.
1195
1196         * bmalloc/BumpAllocator.h: Copied from Source/bmalloc/bmalloc/SmallAllocator.h.
1197         (bmalloc::BumpAllocator::BumpAllocator):
1198         (bmalloc::BumpAllocator::init):
1199         (bmalloc::BumpAllocator::line):
1200         (bmalloc::BumpAllocator::validate):
1201         (bmalloc::BumpAllocator::allocate):
1202         (bmalloc::BumpAllocator::refill):
1203         (bmalloc::BumpAllocator::clear): Updated these functions to be agnostic
1204         about the kinds of lines they allocate into. In some cases, the line
1205         type must be provided as a template parameter by the caller.
1206
1207         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
1208         (bmalloc::SmallAllocator::line): Deleted.
1209         (bmalloc::SmallAllocator::allocate): Deleted.
1210         (bmalloc::SmallAllocator::objectCount): Deleted.
1211         (bmalloc::SmallAllocator::derefCount): Deleted.
1212         (bmalloc::SmallAllocator::refill): Deleted.
1213         (bmalloc::SmallAllocator::clear): Deleted.
1214
1215         * bmalloc/ObjectType.h:
1216         (bmalloc::isMedium):
1217
1218         * bmalloc/SmallAllocator.h:
1219         (bmalloc::SmallAllocator::isNull): Deleted.
1220         (bmalloc::SmallAllocator::canAllocate): Deleted.
1221         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
1222         (bmalloc::SmallAllocator::line): Deleted.
1223         (bmalloc::SmallAllocator::allocate): Deleted.
1224         (bmalloc::SmallAllocator::objectCount): Deleted.
1225         (bmalloc::SmallAllocator::derefCount): Deleted.
1226         (bmalloc::SmallAllocator::refill): Deleted.
1227         (bmalloc::SmallAllocator::clear): Deleted.
1228
1229 2014-09-12  Geoffrey Garen  <ggaren@apple.com>
1230
1231         Fixed a goof in bmalloc Vector sizing
1232         https://bugs.webkit.org/show_bug.cgi?id=136795
1233
1234         Reviewed by Gavin Barraclough and Sam Weinig.
1235
1236         We want our minimum vector to be page-sized since the OS will give us
1237         a page no matter what -- but we want that many bytes, and not enough
1238         bytes to store that many elements.
1239
1240         * bmalloc/Vector.h: Math is hard.
1241
1242 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
1243
1244         bmalloc should segregate medium-sized objects by line like it does for small-sized objects
1245         https://bugs.webkit.org/show_bug.cgi?id=136693
1246
1247         Reviewed by Gavin Barraclough.
1248
1249         4% reduction in heap size on the MallocBench *_memory_warning benchmarks.
1250
1251         No throughput change.
1252
1253         We keep an array of medium allocators, just like our array of small
1254         allocators.
1255
1256         In future, we can simplify the allocation fast path by merging the small
1257         and medium allocator arrays. For now, this is the simplest change that
1258         gets the win.
1259
1260         * bmalloc/Allocator.cpp:
1261         (bmalloc::Allocator::Allocator):
1262         (bmalloc::Allocator::scavenge):
1263         (bmalloc::Allocator::allocateMedium):
1264         * bmalloc/Allocator.h:
1265         * bmalloc/Sizes.h:
1266         (bmalloc::Sizes::mediumSizeClassFor):
1267
1268 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
1269
1270         Reviewed by Sam Weinig.
1271
1272         Renamed log => retire for clarity.
1273
1274         * bmalloc/Allocator.cpp:
1275         (bmalloc::Allocator::scavenge):
1276         (bmalloc::Allocator::retire):
1277         (bmalloc::Allocator::allocateMedium):
1278         (bmalloc::Allocator::allocateSlowCase):
1279         (bmalloc::Allocator::log): Deleted.
1280         * bmalloc/Allocator.h:
1281
1282 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
1283
1284         bmalloc: eager scavenge leaves behind a bogus allocator
1285         https://bugs.webkit.org/show_bug.cgi?id=136743
1286
1287         Reviewed by Sam Weinig.
1288
1289         Be sure to clear the allocator after logging it in the eager scavenge
1290         case, so that we don't later try to allocate out of the lines that we
1291         have thrown away.
1292
1293         We didn't need to do this previously because scavenge would only happen
1294         at thread exit time, after which no further allocation from the per-thread
1295         cache would take place.
1296
1297         * bmalloc/Allocator.cpp:
1298         (bmalloc::Allocator::scavenge):
1299         * bmalloc/MediumAllocator.h:
1300         (bmalloc::MediumAllocator::clear):
1301         * bmalloc/SmallAllocator.h:
1302         (bmalloc::SmallAllocator::clear):
1303
1304 2014-09-05  Geoffrey Garen  <ggaren@apple.com>
1305
1306         bmalloc should honor the FastMalloc statistics API
1307         https://bugs.webkit.org/show_bug.cgi?id=136592
1308
1309         Reviewed by Gavin Barraclough.
1310
1311         We do this by tracking "size" and "capacity" in the VM heap.
1312
1313         The VM heap's "capacity" is all the VM we ever allocated.
1314
1315         The VM heap's "size" the subset of VM currently held onto by the
1316         VM heap (and therefore not in use by the regular heap).
1317
1318         Somewhat ironically, reducing the process's memory footprint, increases
1319         the size of the VM heap, since the VM heap holds the pages that are
1320         purely virtual and not physical.
1321
1322         * bmalloc/Heap.cpp:
1323         (bmalloc::Heap::size):
1324         (bmalloc::Heap::capacity):
1325         * bmalloc/Heap.h:
1326         * bmalloc/VMHeap.cpp:
1327         (bmalloc::VMHeap::VMHeap):
1328         (bmalloc::VMHeap::allocateSmallChunk):
1329         (bmalloc::VMHeap::allocateMediumChunk):
1330         (bmalloc::VMHeap::allocateLargeChunk):
1331         * bmalloc/VMHeap.h:
1332         (bmalloc::VMHeap::size):
1333         (bmalloc::VMHeap::capacity):
1334         (bmalloc::VMHeap::allocateSmallPage):
1335         (bmalloc::VMHeap::allocateMediumPage):
1336         (bmalloc::VMHeap::allocateLargeRange):
1337         (bmalloc::VMHeap::deallocateSmallPage):
1338         (bmalloc::VMHeap::deallocateMediumPage):
1339         (bmalloc::VMHeap::deallocateLargeRange):
1340         * bmalloc/bmalloc.h:
1341         (bmalloc::api::heapSize):
1342         (bmalloc::api::heapCapacity):
1343
1344 2014-09-02  Geoffrey Garen  <ggaren@apple.com>
1345
1346         bmalloc crashes on the EWS bots (due to bad large object allocation)
1347         https://bugs.webkit.org/show_bug.cgi?id=136469
1348
1349         Reviewed by Andreas Kling.
1350
1351         It's possible to convince bmalloc to perform a bad large object allocation,
1352         through these steps:
1353
1354         (1) Insert object A into freelist F0.
1355
1356         (2) Split, merge and split again A's neighbors such that object B is
1357         inserted into freelist F0, with boundary tag and size equal to object A,
1358         but pointer not completely equal to object A. Put object B at the head of F0.
1359
1360         (3) Allocate some other object from F0, swapping its position in the
1361         freelist with object B, such that object A is now ahead of object B.
1362
1363         --> Now, the next allocation for size A/B will allocate object A, which
1364         has a slightly wrong idea about where the object actually begins.
1365         Immediately, you'll corrupt a little memory, and over time, you'll also
1366         corrupt boundary tag metadata.
1367
1368         The solution is to store the begin pointer in the boundary tag. Luckily,
1369         this doesn't make the tag any bigger, and it's not a noticeable slowdown
1370         on MallocBench.
1371
1372         * bmalloc/Algorithm.h:
1373         (bmalloc::rightShift):
1374         * bmalloc/BeginTag.h:
1375         (bmalloc::BeginTag::isInFreeList): This is the bug fix. Make sure to
1376         validate the start pointer when popping off the free list. Through a
1377         very uncommon set of steps, it is possible to have an item in the free
1378         list that is valid by all accounts except for its start pointer.
1379
1380         * bmalloc/BoundaryTag.h:
1381         (bmalloc::BoundaryTag::compactBegin):
1382         (bmalloc::BoundaryTag::setRange):
1383         (bmalloc::BoundaryTag::setSize): Deleted. Record a compact version of the
1384         start pointer. We don't need the whole pointer -- just the offset, in
1385         largeAlignment increments, into the relevant boundary tag bucket.
1386
1387         * bmalloc/BoundaryTagInlines.h:
1388         (bmalloc::validateNext):
1389         (bmalloc::BoundaryTag::init):
1390         (bmalloc::BoundaryTag::mergeLarge):
1391         (bmalloc::BoundaryTag::splitLarge):
1392         * bmalloc/SegregatedFreeList.cpp:
1393         (bmalloc::SegregatedFreeList::insert):
1394         (bmalloc::SegregatedFreeList::takeGreedy):
1395         (bmalloc::SegregatedFreeList::take): Provide the whole range instead of
1396         the size when establishing a boundary tag, as required by the new
1397         interface.
1398
1399         * bmalloc/Sizes.h:
1400
1401 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
1402
1403         Fixed a bmalloc crash seen on the EWS bot
1404         https://bugs.webkit.org/show_bug.cgi?id=135955
1405
1406         Reviewed by Andreas Kling.
1407
1408         * bmalloc/Syscall.h: Some CG APIs vm_copy their input buffers. If the
1409         input buffer is a malloc region, that region will get marked Copy-On-Write
1410         by the kernel. Calls to madvise() for COW regions fail and return EINVAL
1411         on older OS X's. In 10.10, they still fail, but they do not return
1412         EINVAL.
1413
1414         So, we can only ASSERT that our syscalls succeed starting with 10.10.
1415
1416 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
1417
1418         Fixed the bmalloc build
1419         https://bugs.webkit.org/show_bug.cgi?id=135953
1420
1421         Reviewed by Andreas Kling.
1422
1423         * bmalloc.xcodeproj/project.pbxproj: Marked a few headers as private.
1424         These headers are used, so they must be available outside the project.
1425
1426 2014-08-13  Daniel Bates  <dabates@apple.com>
1427
1428         Attempt to fix the build following <http://trac.webkit.org/changeset/172576>
1429         (https://bugs.webkit.org/show_bug.cgi?id=135895)
1430
1431         Substitute PerThreadStorage<T>::initSharedKeyIfNeeded() for initSharedKeyIfNeeded() in
1432         implementation of PerThread<T>::getFastCase().
1433
1434         * bmalloc/PerThread.h:
1435         (bmalloc::PerThread<T>::getFastCase):
1436
1437 2014-08-13  Daniel Bates  <dabates@apple.com>
1438
1439         Make bmalloc::PerThread work without C++ thread local storage
1440         https://bugs.webkit.org/show_bug.cgi?id=135895
1441
1442         Reviewed by Geoffrey Garen.
1443
1444         Implement support for building bmalloc without C++ thread local storage.
1445
1446         * bmalloc/BPlatform.h: Remove macro define BPLATFORM_IOS_SIMULATOR. Added macro function
1447         BCOMPILER_SUPPORTS() and macro define BCOMPILER_SUPPORTS_CXX_THREAD_LOCAL that can be used
1448         to determine whether the compiler supports C++ thread local storage.
1449         * bmalloc/PerThread.h:
1450         (bmalloc::PerThreadStorage::get): Modified to call pthread_getspecific() when building
1451         without C++ thread local storage.
1452         (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Added.
1453         (bmalloc::PerThreadStorage::init): Moved logic to initialize shared Pthread key from here to
1454         PerThreadStorage::initSharedKeyIfNeeded().
1455         (bmalloc::PerThread<T>::getFastCase): Modified to call PerThreadStorage::initSharedKeyIfNeeded()
1456         before querying PerThreadStorage::get() when building without C++ thread local storage so as to
1457         ensure that the shared key has been initialized.
1458         (_pthread_setspecific_direct): Deleted.
1459         (_pthread_getspecific_direct): Deleted.
1460
1461 2014-08-13  Daniel Bates  <dabates@apple.com>
1462
1463         [iOS] Make JavaScriptCore and bmalloc build with the public SDK
1464         https://bugs.webkit.org/show_bug.cgi?id=135848
1465
1466         Reviewed by Geoffrey Garen.
1467
1468         * bmalloc/BPlatform.h: Added macro BPLATFORM_IOS_SIMULATOR, which evaluates to true
1469         when building for the iOS Simulator.
1470         * bmalloc/PerThread.h: Use pthread_machdep.h code path when building for iOS Simulator
1471         using the public SDK.
1472         (_pthread_setspecific_direct): Added; only defined when building for the iOS Simulator
1473         using the public SDK.
1474         (_pthread_getspecific_direct): Added; only defined when building for the iOS Simulator
1475         using the public SDK.
1476
1477 2014-08-12  Daniel Bates  <dabates@apple.com>
1478
1479         BPLATFORM(IOS) always evaluates to false
1480         https://bugs.webkit.org/show_bug.cgi?id=135843
1481
1482         Reviewed by Geoffrey Garen.
1483
1484         Fix typo in definition of BPLATFORM() and include system header TargetConditionals.h
1485         (when building on an Apple platform) so that BPLATFORM(X) evaluates to true when
1486         building for platform X. In particular, so that BPLATFORM(IOS) evaluates to true when
1487         building for iOS.
1488
1489         As a side effect of this change, the change made in <http://trac.webkit.org/changeset/167289>
1490         will be honored and iOS will assume a VM page size of 16kB (again) instead of 4kB.
1491
1492         * bmalloc/BPlatform.h:
1493
1494 2014-08-11  Andy Estes  <aestes@apple.com>
1495
1496         [iOS] Get rid of iOS.xcconfig
1497         https://bugs.webkit.org/show_bug.cgi?id=135809
1498
1499         Reviewed by Joseph Pecoraro.
1500
1501         All iOS.xcconfig did was include AspenFamily.xcconfig, so there's no need for the indirection.
1502
1503         * Configurations/Base.xcconfig:
1504         * Configurations/iOS.xcconfig: Removed.
1505         * bmalloc.xcodeproj/project.pbxproj:
1506
1507 2014-05-01  Dan Bernstein  <mitz@apple.com>
1508
1509         Fixed production builds for the iOS Simulator.
1510         <rdar://problem/16792221>
1511
1512         * Configurations/bmalloc.xcconfig: Include INSTALL_PATH_PREFIX in
1513         PRIVATE_HEADERS_FOLDER_PATH when installing.
1514
1515 2014-04-20  Geoffrey Garen  <ggaren@apple.com>
1516
1517         bmalloc: Segregate pages by objects size
1518         https://bugs.webkit.org/show_bug.cgi?id=131909
1519
1520         Reviewed by Andreas Kling.
1521
1522         2% reduction in memory-at-end on the Membuster memory_warning benchmarks.
1523
1524         * bmalloc/Allocator.cpp:
1525         (bmalloc::Allocator::allocateSlowCase):
1526         * bmalloc/Allocator.h:
1527         (bmalloc::Allocator::allocateFastCase):
1528         (bmalloc::Allocator::smallAllocatorFor): Use the new shared helper
1529         function for size class calculation.
1530
1531         * bmalloc/Deallocator.cpp:
1532         (bmalloc::Deallocator::Deallocator):
1533         (bmalloc::Deallocator::scavenge):
1534         (bmalloc::Deallocator::deallocateSmallLine):
1535         (bmalloc::Deallocator::allocateSmallLine):
1536         * bmalloc/Deallocator.h: Keep a cache for every size class, since the
1537         cache can't be shared anymore.
1538
1539         * bmalloc/Heap.cpp:
1540         (bmalloc::Heap::allocateSmallLineSlowCase):
1541         * bmalloc/Heap.h:
1542         (bmalloc::Heap::deallocateSmallLine): Ditto.
1543
1544         (bmalloc::Heap::allocateSmallLine): Check size class in addition to
1545         page refcount when allocating a line because we might have deallocated
1546         the page and the recycled it for another size class.
1547
1548         (bmalloc::Heap::deallocateMediumLine):
1549         (bmalloc::Heap::allocateMediumLine):
1550         * bmalloc/Line.h:
1551         (bmalloc::Line::refCount):
1552         * bmalloc/Page.h:
1553         (bmalloc::Page::refCount):
1554         (bmalloc::Page::smallSizeClass):
1555         (bmalloc::Page::setSmallSizeClass):
1556         (bmalloc::Page<Traits>::refCount): Deleted.
1557         * bmalloc/Sizes.h:
1558         (bmalloc::Sizes::smallSizeClassFor): New shared API for computing
1559         an index into an array from a size.
1560
1561 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1562
1563         bmalloc: Improved alignment in LargeChunk
1564         https://bugs.webkit.org/show_bug.cgi?id=131895
1565
1566         Reviewed by Andreas Kling.
1567
1568         * bmalloc/Chunk.h:
1569         * bmalloc/LargeChunk.h: Align to vmPageSize just like Chunk does.
1570         Technically, the previous alignment was harmless, but I would prefer,
1571         dear reader, not to have to explain the interlocking set of
1572         circumstances that made it so.
1573
1574 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1575
1576         Rolled out r167502 because it caused a crash on the facebook benchmark.
1577
1578         Unreviewed.
1579
1580             bmalloc: Added an XSmall line size
1581             https://bugs.webkit.org/show_bug.cgi?id=131851
1582
1583             Reviewed by Sam Weinig.
1584
1585 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1586
1587         bmalloc: Mutex should be harder to use wrong
1588         https://bugs.webkit.org/show_bug.cgi?id=131879
1589
1590         Reviewed by Andreas Kling.
1591
1592         Mutex now has a proper constructor, so you can't deadlock by forgetting
1593         to initialize it.
1594
1595         * bmalloc.xcodeproj/project.pbxproj:
1596         * bmalloc/Allocator.cpp:
1597         (bmalloc::Allocator::processXSmallAllocatorLog):
1598         (bmalloc::Allocator::processSmallAllocatorLog):
1599         (bmalloc::Allocator::processMediumAllocatorLog):
1600         (bmalloc::Allocator::allocateLarge):
1601         (bmalloc::Allocator::allocateXLarge): Global replace Mutex => StaticMutex,
1602         since the Heap mutex is a static.
1603
1604         * bmalloc/AsyncTask.h:
1605         (bmalloc::Function>::AsyncTask): Use Mutex, since we're not static. No
1606         need for explicit initialization anymore.
1607
1608         * bmalloc/Deallocator.cpp:
1609         (bmalloc::Deallocator::scavenge):
1610         (bmalloc::Deallocator::deallocateLarge):
1611         (bmalloc::Deallocator::deallocateXLarge):
1612         (bmalloc::Deallocator::processObjectLog):
1613         (bmalloc::Deallocator::deallocateSmallLine):
1614         (bmalloc::Deallocator::deallocateXSmallLine):
1615         (bmalloc::Deallocator::allocateSmallLine):
1616         (bmalloc::Deallocator::allocateXSmallLine):
1617         (bmalloc::Deallocator::deallocateMediumLine):
1618         (bmalloc::Deallocator::allocateMediumLine):
1619         * bmalloc/Deallocator.h:
1620         * bmalloc/Heap.cpp:
1621         (bmalloc::sleep):
1622         (bmalloc::Heap::Heap):
1623         (bmalloc::Heap::concurrentScavenge):
1624         (bmalloc::Heap::scavenge):
1625         (bmalloc::Heap::scavengeSmallPages):
1626         (bmalloc::Heap::scavengeXSmallPages):
1627         (bmalloc::Heap::scavengeMediumPages):
1628         (bmalloc::Heap::scavengeLargeRanges):
1629         (bmalloc::Heap::allocateXSmallLineSlowCase):
1630         (bmalloc::Heap::allocateSmallLineSlowCase):
1631         (bmalloc::Heap::allocateMediumLineSlowCase):
1632         (bmalloc::Heap::allocateXLarge):
1633         (bmalloc::Heap::deallocateXLarge):
1634         (bmalloc::Heap::allocateLarge):
1635         (bmalloc::Heap::deallocateLarge):
1636         * bmalloc/Heap.h:
1637         (bmalloc::Heap::deallocateXSmallLine):
1638         (bmalloc::Heap::allocateXSmallLine):
1639         (bmalloc::Heap::deallocateSmallLine):
1640         (bmalloc::Heap::allocateSmallLine):
1641         (bmalloc::Heap::deallocateMediumLine):
1642         (bmalloc::Heap::allocateMediumLine):
1643         * bmalloc/Line.h:
1644         (bmalloc::Line<Traits>::deref):
1645         * bmalloc/Mutex.cpp: Removed.
1646         * bmalloc/Mutex.h:
1647         (bmalloc::Mutex::Mutex):
1648         (bmalloc::Mutex::init): Deleted.
1649         (bmalloc::Mutex::try_lock): Deleted.
1650         (bmalloc::Mutex::lock): Deleted.
1651         (bmalloc::Mutex::unlock): Deleted.
1652         * bmalloc/Page.h:
1653         (bmalloc::Page<Traits>::ref):
1654         (bmalloc::Page<Traits>::deref):
1655         (bmalloc::Page<Traits>::refCount):
1656         * bmalloc/PerProcess.h:
1657         (bmalloc::PerProcess::mutex):
1658         (bmalloc::PerProcess<T>::getSlowCase):
1659         * bmalloc/StaticMutex.cpp: Added.
1660         (bmalloc::StaticMutex::lockSlowCase):
1661         * bmalloc/StaticMutex.h: Added.
1662         (bmalloc::StaticMutex::init):
1663         (bmalloc::StaticMutex::try_lock):
1664         (bmalloc::StaticMutex::lock):
1665         (bmalloc::StaticMutex::unlock):
1666         * bmalloc/VMHeap.h:
1667         (bmalloc::VMHeap::deallocateXSmallPage):
1668         (bmalloc::VMHeap::deallocateSmallPage):
1669         (bmalloc::VMHeap::deallocateMediumPage):
1670         (bmalloc::VMHeap::deallocateLargeRange):
1671         * bmalloc/bmalloc.h:
1672         (bmalloc::api::scavenge): Global replace Mutex => StaticMutex,
1673         since the Heap mutex is a static.
1674
1675 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
1676
1677         bmalloc: AsyncTask should use Mutex instead of std::mutex
1678         https://bugs.webkit.org/show_bug.cgi?id=131865
1679
1680         Reviewed by Gavin Barraclough.
1681
1682         std::mutex is so slow that it makes parallelizing simple tasks through
1683         AsyncTask a net regression. Mutex fixes this.
1684
1685         * bmalloc/AsyncTask.h:
1686         (bmalloc::Function>::AsyncTask):
1687         (bmalloc::Function>::join):
1688         (bmalloc::Function>::runSlowCase):
1689         (bmalloc::Function>::entryPoint):
1690         * bmalloc/Mutex.h:
1691         (bmalloc::Mutex::init):
1692
1693 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
1694
1695         bmalloc: Added an XSmall line size
1696         https://bugs.webkit.org/show_bug.cgi?id=131851
1697
1698         Reviewed by Sam Weinig.
1699
1700         Reduces malloc footprint on Membuster recordings by 10%.
1701
1702         This is a throughput regression, but we're still way ahead of TCMalloc.
1703         I have some ideas for how to recover the regression -- but I wanted to
1704         get this win in first.
1705
1706         Full set of benchmark results:
1707
1708                 bmalloc> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks --measure-heap nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1709                                                                                                 
1710                                                                nopatch                      patch                                Δ
1711                 Peak Memory:
1712                     reddit_memory_warning                      7,896kB                    7,532kB                  ^ 1.05x smaller
1713                     flickr_memory_warning                     12,968kB                   12,324kB                  ^ 1.05x smaller
1714                     theverge_memory_warning                   16,672kB                   15,200kB                   ^ 1.1x smaller
1715
1716                     <geometric mean>                          11,952kB                   11,216kB                  ^ 1.07x smaller
1717                     <arithmetic mean>                         12,512kB                   11,685kB                  ^ 1.07x smaller
1718                     <harmonic mean>                           11,375kB                   10,726kB                  ^ 1.06x smaller
1719
1720                 Memory at End:
1721                     reddit_memory_warning                      7,320kB                    6,856kB                  ^ 1.07x smaller
1722                     flickr_memory_warning                     10,848kB                    9,692kB                  ^ 1.12x smaller
1723                     theverge_memory_warning                   16,380kB                   14,872kB                   ^ 1.1x smaller
1724
1725                     <geometric mean>                          10,916kB                    9,961kB                   ^ 1.1x smaller
1726                     <arithmetic mean>                         11,516kB                   10,473kB                   ^ 1.1x smaller
1727                     <harmonic mean>                           10,350kB                    9,485kB                  ^ 1.09x smaller
1728
1729                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1730                                                                                                 
1731                                                    nopatch                patch                         Δ
1732                 Execution Time:
1733                     churn                            127ms                151ms            ! 1.19x slower
1734                     list_allocate                    130ms                164ms            ! 1.26x slower
1735                     tree_allocate                    109ms                127ms            ! 1.17x slower
1736                     tree_churn                       115ms                120ms            ! 1.04x slower
1737                     facebook                         240ms                259ms            ! 1.08x slower
1738                     fragment                          91ms                131ms            ! 1.44x slower
1739                     fragment_iterate                 105ms                106ms            ! 1.01x slower
1740                     message_one                      260ms                259ms             ^ 1.0x faster
1741                     message_many                     149ms                154ms            ! 1.03x slower
1742                     medium                           194ms                248ms            ! 1.28x slower
1743                     big                              157ms                160ms            ! 1.02x slower
1744
1745                     <geometric mean>                 144ms                163ms            ! 1.13x slower
1746                     <arithmetic mean>                152ms                171ms            ! 1.12x slower
1747                     <harmonic mean>                  137ms                156ms            ! 1.14x slower
1748
1749                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1750                                                                                                 
1751                                                                        nopatch                          patch                                     Δ
1752                 Execution Time:
1753                     churn                                                126ms                          148ms                        ! 1.17x slower
1754                     churn --parallel                                      62ms                           76ms                        ! 1.23x slower
1755                     list_allocate                                        130ms                          164ms                        ! 1.26x slower
1756                     list_allocate --parallel                             120ms                          175ms                        ! 1.46x slower
1757                     tree_allocate                                        111ms                          127ms                        ! 1.14x slower
1758                     tree_allocate --parallel                              95ms                          135ms                        ! 1.42x slower
1759                     tree_churn                                           115ms                          124ms                        ! 1.08x slower
1760                     tree_churn --parallel                                107ms                          126ms                        ! 1.18x slower
1761                     facebook                                             240ms                          276ms                        ! 1.15x slower
1762                     facebook --parallel                                  802ms                        1,088ms                        ! 1.36x slower
1763                     fragment                                              92ms                          130ms                        ! 1.41x slower
1764                     fragment --parallel                                   66ms                          124ms                        ! 1.88x slower
1765                     fragment_iterate                                     109ms                          127ms                        ! 1.17x slower
1766                     fragment_iterate --parallel                           55ms                           64ms                        ! 1.16x slower
1767                     message_one                                          260ms                          260ms                                      
1768                     message_many                                         170ms                          238ms                         ! 1.4x slower
1769                     medium                                               185ms                          250ms                        ! 1.35x slower
1770                     medium --parallel                                    210ms                          334ms                        ! 1.59x slower
1771                     big                                                  150ms                          169ms                        ! 1.13x slower
1772                     big --parallel                                       138ms                          144ms                        ! 1.04x slower
1773
1774                     <geometric mean>                                     135ms                          170ms                        ! 1.26x slower
1775                     <arithmetic mean>                                    167ms                          214ms                        ! 1.28x slower
1776                     <harmonic mean>                                      117ms                          148ms                        ! 1.26x slower
1777
1778                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
1779
1780                                                                     TC                      patch                                Δ
1781                 Peak Memory:
1782                     reddit_memory_warning                     13,836kB                   13,436kB                  ^ 1.03x smaller
1783                     flickr_memory_warning                     24,868kB                   25,188kB                   ! 1.01x bigger
1784                     theverge_memory_warning                   24,504kB                   26,636kB                   ! 1.09x bigger
1785
1786                     <geometric mean>                          20,353kB                   20,812kB                   ! 1.02x bigger
1787                     <arithmetic mean>                         21,069kB                   21,753kB                   ! 1.03x bigger
1788                     <harmonic mean>                           19,570kB                   19,780kB                   ! 1.01x bigger
1789
1790                 Memory at End:
1791                     reddit_memory_warning                      8,656kB                   10,016kB                   ! 1.16x bigger
1792                     flickr_memory_warning                     11,844kB                   13,784kB                   ! 1.16x bigger
1793                     theverge_memory_warning                   18,516kB                   22,748kB                   ! 1.23x bigger
1794
1795                     <geometric mean>                          12,382kB                   14,644kB                   ! 1.18x bigger
1796                     <arithmetic mean>                         13,005kB                   15,516kB                   ! 1.19x bigger
1797                     <harmonic mean>                           11,813kB                   13,867kB                   ! 1.17x bigger
1798
1799                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
1800                                                                                                 
1801                                                         TC                patch                         Δ
1802                 Execution Time:
1803                     churn                            416ms                148ms            ^ 2.81x faster
1804                     list_allocate                    463ms                164ms            ^ 2.82x faster
1805                     tree_allocate                    292ms                127ms             ^ 2.3x faster
1806                     tree_churn                       157ms                120ms            ^ 1.31x faster
1807                     facebook                         327ms                276ms            ^ 1.18x faster
1808                     fragment                         335ms                129ms             ^ 2.6x faster
1809                     fragment_iterate                 344ms                108ms            ^ 3.19x faster
1810                     message_one                      386ms                258ms             ^ 1.5x faster
1811                     message_many                     410ms                154ms            ^ 2.66x faster
1812                     medium                           391ms                245ms             ^ 1.6x faster
1813                     big                              261ms                167ms            ^ 1.56x faster
1814
1815                     <geometric mean>                 332ms                164ms            ^ 2.02x faster
1816                     <arithmetic mean>                344ms                172ms            ^ 1.99x faster
1817                     <harmonic mean>                  317ms                157ms            ^ 2.02x faster
1818
1819         * bmalloc.xcodeproj/project.pbxproj:
1820         * bmalloc/Allocator.cpp:
1821         (bmalloc::Allocator::Allocator): Don't assume that each allocator's
1822         index corresponds with its size. Instead, use the size selection function
1823         explicitly. Now that we have XSmall, some small allocator entries are
1824         unused.
1825
1826         (bmalloc::Allocator::scavenge):
1827         (bmalloc::Allocator::log):
1828         (bmalloc::Allocator::processXSmallAllocatorLog):
1829         (bmalloc::Allocator::allocateSlowCase):
1830         * bmalloc/Allocator.h:
1831         (bmalloc::Allocator::xSmallAllocatorFor):
1832         (bmalloc::Allocator::allocateFastCase):
1833         * bmalloc/Chunk.h:
1834         * bmalloc/Deallocator.cpp:
1835         (bmalloc::Deallocator::scavenge):
1836         (bmalloc::Deallocator::processObjectLog):
1837         (bmalloc::Deallocator::deallocateSlowCase):
1838         (bmalloc::Deallocator::deallocateXSmallLine):
1839         (bmalloc::Deallocator::allocateXSmallLine):
1840         * bmalloc/Deallocator.h:
1841         (bmalloc::Deallocator::deallocateFastCase):
1842         * bmalloc/Heap.cpp:
1843         (bmalloc::Heap::scavenge):
1844         (bmalloc::Heap::scavengeXSmallPages):
1845         (bmalloc::Heap::allocateXSmallLineSlowCase):
1846         * bmalloc/Heap.h:
1847         (bmalloc::Heap::deallocateXSmallLine):
1848         (bmalloc::Heap::allocateXSmallLine):
1849         * bmalloc/LargeChunk.h:
1850         (bmalloc::LargeChunk::get):
1851         (bmalloc::LargeChunk::endTag):
1852         * bmalloc/Line.h:
1853         * bmalloc/MediumAllocator.h:
1854         (bmalloc::MediumAllocator::allocate):
1855         (bmalloc::MediumAllocator::refill):
1856         * bmalloc/ObjectType.cpp:
1857         (bmalloc::objectType):
1858         * bmalloc/ObjectType.h:
1859         (bmalloc::isXSmall):
1860         (bmalloc::isSmall):
1861         (bmalloc::isMedium):
1862         (bmalloc::isLarge):
1863         (bmalloc::isSmallOrMedium): Deleted.
1864         * bmalloc/SegregatedFreeList.h: I boiler-plate copied existing code for
1865         handling small objects. There's probably a reasonable way to share this
1866         code in the future -- I'll look into that once it's stopped changing.
1867
1868         * bmalloc/Sizes.h: Tweaked size classes to make Membuster happy. This
1869         is the main reason things got slower.
1870
1871         * bmalloc/SmallAllocator.h:
1872         (bmalloc::SmallAllocator::allocate):
1873         * bmalloc/SmallTraits.h:
1874         * bmalloc/VMHeap.cpp:
1875         (bmalloc::VMHeap::allocateXSmallChunk):
1876         * bmalloc/VMHeap.h:
1877         (bmalloc::VMHeap::allocateXSmallPage):
1878         (bmalloc::VMHeap::deallocateXSmallPage):
1879         * bmalloc/XSmallAllocator.h: Added.
1880         (bmalloc::XSmallAllocator::isNull):
1881         (bmalloc::XSmallAllocator::canAllocate):
1882         (bmalloc::XSmallAllocator::XSmallAllocator):
1883         (bmalloc::XSmallAllocator::line):
1884         (bmalloc::XSmallAllocator::allocate):
1885         (bmalloc::XSmallAllocator::objectCount):
1886         (bmalloc::XSmallAllocator::derefCount):
1887         (bmalloc::XSmallAllocator::refill):
1888         * bmalloc/XSmallChunk.h: Added.
1889         * bmalloc/XSmallLine.h: Added.
1890         * bmalloc/XSmallPage.h: Added.
1891         * bmalloc/XSmallTraits.h: Added.
1892         * bmalloc/bmalloc.h:
1893         (bmalloc::api::realloc): Boiler-plate copy, as above.
1894
1895 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
1896
1897         MallocBench should scavenge explicitly instead of waiting
1898         https://bugs.webkit.org/show_bug.cgi?id=131661
1899
1900         Reviewed by Andreas Kling.
1901
1902         Added explicit scavenge support to bmalloc. This isn't a memory win,
1903         since bmalloc's per-thread cache is so small. But it makes testing
1904         simpler.
1905
1906         * bmalloc/Allocator.cpp:
1907         (bmalloc::Allocator::~Allocator):
1908         (bmalloc::Allocator::scavenge):
1909         * bmalloc/Allocator.h:
1910         * bmalloc/Cache.cpp:
1911         (bmalloc::Cache::operator new):
1912         (bmalloc::Cache::operator delete):
1913         (bmalloc::Cache::Cache):
1914         (bmalloc::Cache::scavenge):
1915         * bmalloc/Cache.h:
1916         * bmalloc/Deallocator.cpp:
1917         (bmalloc::Deallocator::~Deallocator):
1918         (bmalloc::Deallocator::scavenge):
1919         * bmalloc/Deallocator.h: Factored existing scavenging code into helper
1920         functions, for reuse.
1921
1922         * bmalloc/Heap.cpp:
1923         (bmalloc::sleep):
1924         (bmalloc::Heap::concurrentScavenge):
1925         (bmalloc::Heap::scavenge):
1926         (bmalloc::Heap::scavengeSmallPages):
1927         (bmalloc::Heap::scavengeMediumPages):
1928         (bmalloc::Heap::scavengeLargeRanges):
1929         * bmalloc/Heap.h: Made scavenge sleep duration a parameter. Forced
1930         scavenging -- in response to a benchmark or a low memory warning --
1931         wants to complete as soon as possible, so its sleep duration is 0.
1932
1933         * bmalloc/bmalloc.h:
1934         (bmalloc::api::scavenge):
1935         * bmalloc/mbmalloc.cpp: Exported the scavenge API for MallocBench's use.
1936
1937 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
1938
1939         Use 4kB pages on Mac
1940         https://bugs.webkit.org/show_bug.cgi?id=131658
1941
1942         Reviewed by Sam Weinig.
1943
1944         This reduces memory use a lot on Membuster:
1945
1946                                                                   base                      patch                                Δ
1947                 Execution Time:
1948                     reddit_memory_warning                         18ms                       17ms                   ^ 1.06x faster
1949                     flickr_memory_warning                         34ms                       36ms                   ! 1.06x slower
1950                     theverge_memory_warning                       39ms                       41ms                   ! 1.05x slower
1951
1952                     <geometric mean>                              29ms                       29ms                   ! 1.02x slower
1953                     <arithmetic mean>                             30ms                       31ms                   ! 1.03x slower
1954                     <harmonic mean>                               27ms                       27ms                    ^ 1.0x faster
1955
1956                 Peak Memory:
1957                     reddit_memory_warning                     16,412kB                   16,436kB                    ! 1.0x bigger
1958                     flickr_memory_warning                     30,120kB                   30,184kB                    ! 1.0x bigger
1959                     theverge_memory_warning                   33,408kB                   33,420kB                    ! 1.0x bigger
1960
1961                     <geometric mean>                          25,466kB                   25,499kB                    ! 1.0x bigger
1962                     <arithmetic mean>                         26,647kB                   26,680kB                    ! 1.0x bigger
1963                     <harmonic mean>                           24,181kB                   24,214kB                    ! 1.0x bigger
1964
1965                 Memory at End:
1966                     reddit_memory_warning                      2,404kB                    1,920kB                  ^ 1.25x smaller
1967                     flickr_memory_warning                      3,764kB                    3,072kB                  ^ 1.23x smaller
1968                     theverge_memory_warning                    3,648kB                    3,132kB                  ^ 1.16x smaller
1969
1970                     <geometric mean>                           3,208kB                    2,644kB                  ^ 1.21x smaller
1971                     <arithmetic mean>                          3,272kB                    2,708kB                  ^ 1.21x smaller
1972                     <harmonic mean>                            3,139kB                    2,574kB                  ^ 1.22x smaller
1973
1974
1975         * bmalloc.xcodeproj/project.pbxproj:
1976         * bmalloc/BPlatform.h: Added.
1977         * bmalloc/VMAllocate.h: Only use 16kB pages on iOS because the page size
1978         is 4kB on Mac.
1979
1980 2014-04-14  Alexey Proskuryakov  <ap@apple.com>
1981
1982         Fixed svn:ignore on bmalloc.xcodeproj, it had erroneous leading spaces.
1983
1984         * bmalloc.xcodeproj: Modified property svn:ignore.
1985
1986 2014-04-13  Geoffrey Garen  <ggaren@apple.com>
1987
1988         Fixed some mbmalloc exports
1989         https://bugs.webkit.org/show_bug.cgi?id=131599
1990
1991         Reviewed by Ryosuke Niwa.
1992
1993         * bmalloc.xcodeproj/project.pbxproj: Made some headers a private part
1994         of the project, so we can call them from API.
1995
1996         * bmalloc/mbmalloc.cpp: Marked the mbmalloc functions with default
1997         visibility, so they show up as exported in the .dylib.
1998
1999 2014-04-09  Geoffrey Garen  <ggaren@apple.com>
2000
2001         Put bmalloc headers in the right place
2002         https://bugs.webkit.org/show_bug.cgi?id=131464
2003
2004         Reviewed by Mark Rowe.
2005
2006         * Configurations/bmalloc.xcconfig: Set PRIVATE_HEADERS_FOLDER_PATH to
2007         specify that we don't just want to dump all of our generically-named
2008         headers into /usr/local/include.
2009
2010 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
2011
2012         Made bmalloc more #include friendly
2013         https://bugs.webkit.org/show_bug.cgi?id=131386
2014
2015         Reviewed by Andreas Kling.
2016
2017         Marked a bunch of headers private so they can be used from client code
2018         that #includes bmalloc.h.
2019
2020         Renamed ASSERT macros to BASSERT. This matches their header, which already
2021         had to be renamed, and fixes conflicts with WTF's ASSERT macros.
2022
2023         * bmalloc.xcodeproj/project.pbxproj:
2024         * bmalloc/Allocator.cpp:
2025         (bmalloc::Allocator::allocateSlowCase):
2026         * bmalloc/AsyncTask.h:
2027         (bmalloc::Function>::runSlowCase):
2028         * bmalloc/BAssert.h:
2029         * bmalloc/BoundaryTag.h:
2030         (bmalloc::BoundaryTag::setSize):
2031         * bmalloc/BoundaryTagInlines.h:
2032         (bmalloc::validate):
2033         (bmalloc::BoundaryTag::init):
2034         (bmalloc::BoundaryTag::deallocate):
2035         (bmalloc::BoundaryTag::splitLarge):
2036         (bmalloc::BoundaryTag::allocate):
2037         * bmalloc/Chunk.h:
2038         * bmalloc/Deallocator.cpp:
2039         (bmalloc::Deallocator::processObjectLog):
2040         (bmalloc::Deallocator::deallocateSlowCase):
2041         * bmalloc/Deallocator.h:
2042         (bmalloc::Deallocator::deallocateFastCase):
2043         * bmalloc/FixedVector.h:
2044         (bmalloc::Capacity>::operator):
2045         (bmalloc::Capacity>::push):
2046         (bmalloc::Capacity>::pop):
2047         (bmalloc::Capacity>::shrink):
2048         * bmalloc/Heap.cpp:
2049         (bmalloc::Heap::allocateLarge):
2050         * bmalloc/LargeChunk.h:
2051         (bmalloc::LargeChunk::get):
2052         (bmalloc::LargeChunk::endTag):
2053         * bmalloc/Line.h:
2054         (bmalloc::Line<Traits>::concurrentRef):
2055         (bmalloc::Line<Traits>::deref):
2056         * bmalloc/MediumAllocator.h:
2057         (bmalloc::MediumAllocator::allocate):
2058         * bmalloc/ObjectType.h:
2059         (bmalloc::isSmall):
2060         * bmalloc/Page.h:
2061         (bmalloc::Page<Traits>::ref):
2062         (bmalloc::Page<Traits>::deref):
2063         * bmalloc/PerThread.h:
2064         (bmalloc::PerThread<T>::getSlowCase):
2065         * bmalloc/SegregatedFreeList.cpp:
2066         (bmalloc::SegregatedFreeList::SegregatedFreeList):
2067         (bmalloc::SegregatedFreeList::insert):
2068         * bmalloc/SmallAllocator.h:
2069         (bmalloc::SmallAllocator::allocate):
2070         (bmalloc::SmallAllocator::refill):
2071         * bmalloc/Syscall.h:
2072         * bmalloc/VMAllocate.h:
2073         (bmalloc::vmValidate):
2074         (bmalloc::vmAllocate):
2075         (bmalloc::vmDeallocatePhysicalPagesSloppy):
2076         * bmalloc/Vector.h:
2077         (bmalloc::Vector<T>::operator):
2078         (bmalloc::Vector<T>::pop):
2079         (bmalloc::Vector<T>::shrink):
2080         * bmalloc/XLargeChunk.h:
2081         (bmalloc::XLargeChunk::range):
2082         (bmalloc::XLargeChunk::size):
2083
2084 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
2085
2086         Removed an unused file.
2087
2088         Unreviewed.
2089
2090         * bmalloc/AsyncTask.cpp: Removed.
2091
2092 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
2093
2094         Build bmalloc on Mac
2095         https://bugs.webkit.org/show_bug.cgi?id=131333
2096
2097         Reviewed by Mark Rowe.
2098
2099         * Makefile: Added. For make clients.
2100
2101         These files are required for building any project in WebKit. I copied
2102         them from WTF:
2103         * Configurations: Added.
2104         * Configurations/Base.xcconfig: Added.
2105         * Configurations/DebugRelease.xcconfig: Added.
2106         * Configurations/bmalloc.xcconfig: Added.
2107         * Configurations/iOS.xcconfig: Added.
2108         * Configurations/mbmalloc.xcconfig: Added.
2109
2110         * bmalloc.xcodeproj/project.pbxproj: I removed per-project-file stuff
2111         from here because everything is in .xcconfig files now.
2112
2113         I had to fix a bunch of minor warnings, since they're enabled in our
2114         .xcconfig files:
2115
2116         * bmalloc/AsyncTask.h:
2117         (bmalloc::Function>::AsyncTask):
2118         * bmalloc/BAssert.h:
2119         * bmalloc/BoundaryTagInlines.h:
2120         (bmalloc::validate):
2121         * bmalloc/Heap.cpp:
2122         (bmalloc::Heap::Heap):
2123         (bmalloc::Heap::allocateLarge):
2124         (bmalloc::Heap::deallocateLarge):
2125         * bmalloc/Mutex.h:
2126         (bmalloc::Mutex::Mutex): Deleted.
2127         * bmalloc/VMAllocate.h:
2128         (bmalloc::vmValidate):
2129         * bmalloc/mbmalloc.cpp:
2130
2131 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
2132
2133         bmalloc: Fixed a leak in the per-thread cache
2134         https://bugs.webkit.org/show_bug.cgi?id=131330
2135
2136         Reviewed by Andreas Kling.
2137
2138         Remember to deallocate our line caches upon thread exit.
2139
2140         * bmalloc/Deallocator.cpp:
2141         (bmalloc::Deallocator::~Deallocator):
2142
2143 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
2144
2145         bmalloc: rolled out the tryLock experiment
2146         https://bugs.webkit.org/show_bug.cgi?id=131328
2147
2148         Reviewed by Andreas Kling.
2149
2150         It wasn't a speedup.
2151
2152         * bmalloc.xcodeproj/project.pbxproj:
2153         * bmalloc/Allocator.cpp:
2154         (bmalloc::Allocator::processSmallAllocatorLog):
2155         (bmalloc::Allocator::processMediumAllocatorLog):
2156         * bmalloc/Deallocator.cpp:
2157         (bmalloc::Deallocator::processObjectLog):
2158         (bmalloc::Deallocator::deallocateSlowCase):
2159         (bmalloc::Deallocator::deallocateSmallLine):
2160         (bmalloc::Deallocator::deallocateMediumLine):
2161         * bmalloc/Deallocator.h:
2162         (bmalloc::Deallocator::deallocateFastCase):
2163         * bmalloc/Heap.h:
2164         (bmalloc::Heap::deallocateSmallLine):
2165         (bmalloc::Heap::deallocateMediumLine):
2166         * bmalloc/Line.h:
2167         (bmalloc::Line<Traits>::deref):
2168         * bmalloc/Page.h:
2169         (bmalloc::Page<Traits>::deref):
2170
2171 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
2172
2173         bmalloc
2174         https://bugs.webkit.org/show_bug.cgi?id=131170
2175
2176         Reviewed by Andreas Kling.
2177
2178         Initial commit.
2179
2180         * bmalloc: Added.
2181         * bmalloc.xcodeproj: Added.
2182         * bmalloc.xcodeproj/project.pbxproj: Added.
2183         * bmalloc/Algorithm.h: Added.
2184         (bmalloc::max):
2185         (bmalloc::min):
2186         (bmalloc::mask):
2187         (bmalloc::test):
2188         (bmalloc::roundUpToMultipleOf):
2189         (bmalloc::roundDownToMultipleOf):
2190         (bmalloc::sizeOf):
2191         (bmalloc::bitCount):
2192         (bmalloc::isPowerOfTwo):
2193         * bmalloc/Allocator.cpp: Added.
2194         (bmalloc::Allocator::Allocator):
2195         (bmalloc::Allocator::~Allocator):
2196         (bmalloc::Allocator::log):
2197         (bmalloc::Allocator::processSmallAllocatorLog):
2198         (bmalloc::Allocator::processMediumAllocatorLog):
2199         (bmalloc::Allocator::allocateLarge):
2200         (bmalloc::Allocator::allocateXLarge):
2201         (bmalloc::Allocator::allocateMedium):
2202         (bmalloc::Allocator::allocateSlowCase):
2203         * bmalloc/Allocator.h: Added.
2204         (bmalloc::Allocator::smallAllocatorFor):
2205         (bmalloc::Allocator::allocateFastCase):
2206         (bmalloc::Allocator::allocate):
2207         * bmalloc/AsyncTask.cpp: Added.
2208         (bmalloc::AsyncTask<Function>::runSlowCase):
2209         (bmalloc::AsyncTask<Function>::pthreadEntryPoint):
2210         (bmalloc::AsyncTask<Function>::entryPoint):
2211         * bmalloc/AsyncTask.h: Added.
2212         (bmalloc::Function>::AsyncTask):
2213         (bmalloc::Function>::join):
2214         (bmalloc::Function>::run):
2215         (bmalloc::Function>::runSlowCase):
2216         (bmalloc::Function>::pthreadEntryPoint):
2217         (bmalloc::Function>::entryPoint):
2218         * bmalloc/BAssert.h: Added.
2219         * bmalloc/BeginTag.h: Added.
2220         (bmalloc::BeginTag::isInFreeList):
2221         * bmalloc/BoundaryTag.h: Added.
2222         (bmalloc::BoundaryTag::isXLarge):
2223         (bmalloc::BoundaryTag::setXLarge):
2224         (bmalloc::BoundaryTag::isFree):
2225         (bmalloc::BoundaryTag::setFree):
2226         (bmalloc::BoundaryTag::isEnd):
2227         (bmalloc::BoundaryTag::setEnd):
2228         (bmalloc::BoundaryTag::hasPhysicalPages):
2229         (bmalloc::BoundaryTag::setHasPhysicalPages):
2230         (bmalloc::BoundaryTag::isNull):
2231         (bmalloc::BoundaryTag::clear):
2232         (bmalloc::BoundaryTag::size):
2233         (bmalloc::BoundaryTag::setSize):
2234         (bmalloc::BoundaryTag::prev):
2235         (bmalloc::BoundaryTag::next):
2236         * bmalloc/BoundaryTagInlines.h: Added.
2237         (bmalloc::validate):
2238         (bmalloc::validatePrev):
2239         (bmalloc::validateNext):
2240         (bmalloc::BoundaryTag::init):
2241         (bmalloc::BoundaryTag::mergeLargeLeft):
2242         (bmalloc::BoundaryTag::mergeLargeRight):
2243         (bmalloc::BoundaryTag::mergeLarge):
2244         (bmalloc::BoundaryTag::deallocate):
2245         (bmalloc::BoundaryTag::splitLarge):
2246         (bmalloc::BoundaryTag::allocate):
2247         * bmalloc/Cache.cpp: Added.
2248         (bmalloc::Cache::operator new):
2249         (bmalloc::Cache::operator delete):
2250         (bmalloc::Cache::Cache):
2251         (bmalloc::Cache::allocateSlowCase):
2252         (bmalloc::Cache::allocateSlowCaseNullCache):
2253         (bmalloc::Cache::deallocateSlowCase):
2254         (bmalloc::Cache::deallocateSlowCaseNullCache):
2255         * bmalloc/Cache.h: Added.
2256         (bmalloc::Cache::allocator):
2257         (bmalloc::Cache::deallocator):
2258         (bmalloc::Cache::allocateFastCase):
2259         (bmalloc::Cache::deallocateFastCase):
2260         (bmalloc::Cache::allocate):
2261         (bmalloc::Cache::deallocate):
2262         * bmalloc/Chunk.h: Added.
2263         (bmalloc::Chunk::begin):
2264         (bmalloc::Chunk::end):
2265         (bmalloc::Chunk::lines):
2266         (bmalloc::Chunk::pages):
2267         * bmalloc/Deallocator.cpp: Added.
2268         (bmalloc::Deallocator::Deallocator):
2269         (bmalloc::Deallocator::~Deallocator):
2270         (bmalloc::Deallocator::deallocateLarge):
2271         (bmalloc::Deallocator::deallocateXLarge):
2272         (bmalloc::Deallocator::processObjectLog):
2273         (bmalloc::Deallocator::deallocateSlowCase):
2274         (bmalloc::Deallocator::deallocateSmallLine):
2275         (bmalloc::Deallocator::allocateSmallLine):
2276         (bmalloc::Deallocator::deallocateMediumLine):
2277         (bmalloc::Deallocator::allocateMediumLine):
2278         * bmalloc/Deallocator.h: Added.
2279         (bmalloc::Deallocator::deallocateFastCase):
2280         (bmalloc::Deallocator::deallocate):
2281         * bmalloc/EndTag.h: Added.
2282         (bmalloc::EndTag::operator=):
2283         * bmalloc/FixedVector.h: Added.
2284         (bmalloc::FixedVector::begin):
2285         (bmalloc::FixedVector::end):
2286         (bmalloc::FixedVector::size):
2287         (bmalloc::FixedVector::capacity):
2288         (bmalloc::FixedVector::clear):
2289         (bmalloc::FixedVector::isEmpty):
2290         (bmalloc::Capacity>::FixedVector):
2291         (bmalloc::Capacity>::operator):
2292         (bmalloc::Capacity>::push):
2293         (bmalloc::Capacity>::pop):
2294         (bmalloc::Capacity>::shrink):
2295         * bmalloc/Heap.cpp: Added.
2296         (bmalloc::sleep):
2297         (bmalloc::Heap::Heap):
2298         (bmalloc::Heap::concurrentScavenge):
2299         (bmalloc::Heap::scavengeSmallPages):
2300         (bmalloc::Heap::scavengeMediumPages):
2301         (bmalloc::Heap::scavengeLargeRanges):
2302         (bmalloc::Heap::allocateSmallLineSlowCase):
2303         (bmalloc::Heap::allocateMediumLineSlowCase):
2304         (bmalloc::Heap::allocateXLarge):
2305         (bmalloc::Heap::deallocateXLarge):
2306         (bmalloc::Heap::allocateLarge):
2307         (bmalloc::Heap::deallocateLarge):
2308         * bmalloc/Heap.h: Added.
2309         (bmalloc::Heap::deallocateSmallLine):
2310         (bmalloc::Heap::allocateSmallLine):
2311         (bmalloc::Heap::deallocateMediumLine):
2312         (bmalloc::Heap::allocateMediumLine):
2313         * bmalloc/Inline.h: Added.
2314         * bmalloc/LargeChunk.h: Added.
2315         (bmalloc::LargeChunk::begin):
2316         (bmalloc::LargeChunk::end):
2317         (bmalloc::LargeChunk::create):
2318         (bmalloc::LargeChunk::get):
2319         (bmalloc::LargeChunk::beginTag):
2320         (bmalloc::LargeChunk::endTag):
2321         * bmalloc/Line.h: Added.
2322         (bmalloc::Line<Traits>::begin):
2323         (bmalloc::Line<Traits>::end):
2324         (bmalloc::Line<Traits>::concurrentRef):
2325         (bmalloc::Line<Traits>::deref):
2326         * bmalloc/MediumAllocator.h: Added.
2327         (bmalloc::MediumAllocator::isNull):
2328         (bmalloc::MediumAllocator::MediumAllocator):
2329         (bmalloc::MediumAllocator::line):
2330         (bmalloc::MediumAllocator::allocate):
2331         (bmalloc::MediumAllocator::derefCount):
2332         (bmalloc::MediumAllocator::refill):
2333         * bmalloc/MediumChunk.h: Added.
2334         * bmalloc/MediumLine.h: Added.
2335         * bmalloc/MediumPage.h: Added.
2336         * bmalloc/MediumTraits.h: Added.
2337         * bmalloc/Mutex.cpp: Added.
2338         (bmalloc::Mutex::lockSlowCase):
2339         * bmalloc/Mutex.h: Added.
2340         (bmalloc::Mutex::Mutex):
2341         (bmalloc::Mutex::try_lock):
2342         (bmalloc::Mutex::lock):
2343         (bmalloc::Mutex::unlock):
2344         * bmalloc/ObjectType.cpp: Added.
2345         (bmalloc::objectType):
2346         * bmalloc/ObjectType.h: Added.
2347         (bmalloc::isSmallOrMedium):
2348         (bmalloc::isSmall):
2349         * bmalloc/Page.h: Added.
2350         (bmalloc::Page<Traits>::ref):
2351         (bmalloc::Page<Traits>::deref):
2352         (bmalloc::Page<Traits>::refCount):
2353         * bmalloc/PerProcess.h: Added.
2354         (bmalloc::PerProcess::mutex):
2355         (bmalloc::PerProcess<T>::getFastCase):
2356         (bmalloc::PerProcess<T>::get):
2357         (bmalloc::PerProcess<T>::getSlowCase):
2358         * bmalloc/PerThread.h: Added.
2359         (bmalloc::PerThreadStorage<Cache>::get):
2360         (bmalloc::PerThreadStorage<Cache>::init):
2361         (bmalloc::PerThreadStorage::get):
2362         (bmalloc::PerThreadStorage::init):
2363         (bmalloc::PerThread<T>::getFastCase):
2364         (bmalloc::PerThread<T>::get):
2365         (bmalloc::PerThread<T>::destructor):
2366         (bmalloc::PerThread<T>::getSlowCase):
2367         * bmalloc/Range.h: Added.
2368         (bmalloc::Range::Range):
2369         (bmalloc::Range::begin):
2370         (bmalloc::Range::end):
2371         (bmalloc::Range::size):
2372         (bmalloc::Range::operator!):
2373         (bmalloc::Range::operator<):
2374         * bmalloc/SegregatedFreeList.cpp: Added.
2375         (bmalloc::SegregatedFreeList::SegregatedFreeList):
2376         (bmalloc::SegregatedFreeList::insert):
2377         (bmalloc::SegregatedFreeList::takeGreedy):
2378         (bmalloc::SegregatedFreeList::take):
2379         * bmalloc/SegregatedFreeList.h: Added.
2380         * bmalloc/Sizes.h: Added.
2381         * bmalloc/SmallAllocator.h: Added.
2382         (bmalloc::SmallAllocator::isNull):
2383         (bmalloc::SmallAllocator::canAllocate):
2384         (bmalloc::SmallAllocator::SmallAllocator):
2385         (bmalloc::SmallAllocator::line):
2386         (bmalloc::SmallAllocator::allocate):
2387         (bmalloc::SmallAllocator::objectCount):
2388         (bmalloc::SmallAllocator::derefCount):
2389         (bmalloc::SmallAllocator::refill):
2390         * bmalloc/SmallChunk.h: Added.
2391         * bmalloc/SmallLine.h: Added.
2392         * bmalloc/SmallPage.h: Added.
2393         * bmalloc/SmallTraits.h: Added.
2394         * bmalloc/Syscall.h: Added.
2395         * bmalloc/VMAllocate.h: Added.
2396         (bmalloc::vmSize):
2397         (bmalloc::vmValidate):
2398         (bmalloc::vmAllocate):
2399         (bmalloc::vmDeallocate):
2400         (bmalloc::vmDeallocatePhysicalPages):
2401         (bmalloc::vmAllocatePhysicalPages):
2402         (bmalloc::vmDeallocatePhysicalPagesSloppy):
2403         (bmalloc::vmAllocatePhysicalPagesSloppy):
2404         * bmalloc/VMHeap.cpp: Added.
2405         (bmalloc::VMHeap::VMHeap):
2406         (bmalloc::VMHeap::allocateSmallChunk):
2407         (bmalloc::VMHeap::allocateMediumChunk):
2408         (bmalloc::VMHeap::allocateLargeChunk):
2409         * bmalloc/VMHeap.h: Added.
2410         (bmalloc::VMHeap::allocateSmallPage):
2411         (bmalloc::VMHeap::allocateMediumPage):
2412         (bmalloc::VMHeap::allocateLargeRange):
2413         (bmalloc::VMHeap::deallocateSmallPage):
2414         (bmalloc::VMHeap::deallocateMediumPage):
2415         (bmalloc::VMHeap::deallocateLargeRange):
2416         * bmalloc/Vector.h: Added.
2417         (bmalloc::Vector::begin):
2418         (bmalloc::Vector::end):
2419         (bmalloc::Vector::size):
2420         (bmalloc::Vector::capacity):
2421         (bmalloc::Vector::last):
2422         (bmalloc::Vector::pop):
2423         (bmalloc::Vector<T>::Vector):
2424         (bmalloc::Vector<T>::~Vector):
2425         (bmalloc::Vector<T>::operator):
2426         (bmalloc::Vector<T>::push):
2427         (bmalloc::Vector<T>::pop):
2428         (bmalloc::Vector<T>::shrink):
2429         (bmalloc::Vector<T>::reallocateBuffer):
2430         (bmalloc::Vector<T>::shrinkCapacity):
2431         (bmalloc::Vector<T>::growCapacity):
2432         * bmalloc/XLargeChunk.h: Added.
2433         (bmalloc::XLargeChunk::get):
2434         (bmalloc::XLargeChunk::begin):
2435         (bmalloc::XLargeChunk::XLargeChunk):
2436         (bmalloc::XLargeChunk::create):
2437         (bmalloc::XLargeChunk::destroy):
2438         (bmalloc::XLargeChunk::range):
2439         (bmalloc::XLargeChunk::size):
2440         * bmalloc/bmalloc.h: Added.
2441         (bmalloc::api::malloc):
2442         (bmalloc::api::free):
2443         (bmalloc::api::realloc):
2444         * bmalloc/mbmalloc.cpp: Added.
2445