7aafbfc93375b586f13a17e91b35968cdd5e27c2
[WebKit-https.git] / Source / bmalloc / ChangeLog
1 2014-09-24  Geoffrey Garen  <ggaren@apple.com>
2
3         bmalloc: cleaned up fast path vs slow path
4         https://bugs.webkit.org/show_bug.cgi?id=137081
5
6         Reviewed by Sam Weinig.
7
8         Might be a 1% speedup on MallocBench. Also cleans up the code a bit.
9
10         * bmalloc/Allocator.cpp:
11         (bmalloc::Allocator::Allocator): Merged the small and medium range
12         caches, just like the small and medium allocators. Ranges are abstract
13         objects that don't really care whether they hold small or medium objects,
14         so they don't need to be segregated.
15
16         (bmalloc::Allocator::scavenge): Ditto.
17
18         (bmalloc::Allocator::allocateBumpRangeSlowCase):
19         (bmalloc::Allocator::allocateBumpRange): Same thing here, except that
20         we do care a tiny bit, because we need to specify small vs medium when
21         allocating new ranges from the heap, to ensure that the heap allocates
22         from the right segment of VM.
23
24         (bmalloc::Allocator::allocateLarge):
25         (bmalloc::Allocator::allocateXLarge): NO_INLINE because this was clouding
26         up the fast path. Large allocation performance is dominated by allocation
27         logic and initialization, so inlining it doesn't help.
28
29         (bmalloc::Allocator::allocateSlowCase): Slow path got a bit cleaner since
30         it doesn't need to distinguish small vs medium objects.
31
32         (bmalloc::Allocator::allocateSmallBumpRange): Deleted.
33         (bmalloc::Allocator::allocateMediumBumpRange): Deleted.
34
35         * bmalloc/Allocator.h:
36         * bmalloc/BumpRange.h:
37
38         * bmalloc/Cache.cpp:
39         (bmalloc::Cache::allocateSlowCase): Deleted.
40         (bmalloc::Cache::deallocateSlowCase): Deleted.
41         * bmalloc/Cache.h:
42         (bmalloc::Cache::allocate):
43         (bmalloc::Cache::deallocate):
44         (bmalloc::Cache::allocateFastCase): Deleted.
45         (bmalloc::Cache::deallocateFastCase): Deleted. Removed the Cache slow
46         paths. The downside to this change is that the fast path branches to two
47         distinct failure cases instead of one. The upside is that the slow path
48         doesn't need to re-read the segment register, which is not as cheap as a
49         normal register, and it doesn't need to do an extra level of function 
50         call. Seems to be worth it.
51
52         * bmalloc/Deallocator.h:
53         * bmalloc/Heap.cpp:
54         (bmalloc::Heap::refillSmallBumpRangeCache):
55         (bmalloc::Heap::refillMediumBumpRangeCache):
56         * bmalloc/Heap.h: Updated for interface changes.
57
58         * bmalloc/Sizes.h: The most ranges a cache will hold is the number of
59         small lines in a page / 2, since any other free lines will coalesce
60         with their neighbors.
61
62 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
63
64         Rolled out r173346.
65
66             bmalloc should honor the FastMalloc statistics API
67             https://bugs.webkit.org/show_bug.cgi?id=136592
68
69         This didn't really work. Because we allow ranges with and without
70         physical pages to merge, and we allow double-committing and
71         double-decommitting, we can't rely on commit actions to track memory
72         footprint.
73
74         * bmalloc/Heap.cpp:
75         (bmalloc::Heap::size): Deleted.
76         (bmalloc::Heap::capacity): Deleted.
77         * bmalloc/Heap.h:
78         * bmalloc/VMHeap.cpp:
79         (bmalloc::VMHeap::VMHeap):
80         (bmalloc::VMHeap::allocateSmallChunk):
81         (bmalloc::VMHeap::allocateMediumChunk):
82         (bmalloc::VMHeap::allocateLargeChunk):
83         * bmalloc/VMHeap.h:
84         (bmalloc::VMHeap::allocateSmallPage):
85         (bmalloc::VMHeap::allocateMediumPage):
86         (bmalloc::VMHeap::allocateLargeRange):
87         (bmalloc::VMHeap::deallocateSmallPage):
88         (bmalloc::VMHeap::deallocateMediumPage):
89         (bmalloc::VMHeap::deallocateLargeRange):
90         (bmalloc::VMHeap::size): Deleted.
91         (bmalloc::VMHeap::capacity): Deleted.
92         * bmalloc/bmalloc.h:
93         (bmalloc::api::heapSize): Deleted.
94         (bmalloc::api::heapCapacity): Deleted.
95
96 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
97
98         bmalloc: Allocation should be more precise
99         https://bugs.webkit.org/show_bug.cgi?id=136993
100
101         Reviewed by Gavin Barraclough.
102
103         13% reduction in heap size on the MallocBench *_memory_warning benchmarks.
104
105         This patch teaches the allocator to merge adjacent free lines into a
106         single allocatable range. This allows us to shrink the size of an
107         individual line without increasing fragmentation or the rate of allocator
108         slow paths.
109
110         We'll only take more slow paths when available memory is sparse, which
111         is exactly when it's worth it. When available memory is dense, we'll
112         take fewer slow paths.
113
114         * bmalloc.xcodeproj/project.pbxproj:
115         * bmalloc/Algorithm.h:
116         (bmalloc::divideRoundingUp):
117
118         * bmalloc/Allocator.cpp:
119         (bmalloc::Allocator::Allocator): Updated for interface changes.
120
121         (bmalloc::Allocator::scavenge): Scavenge by object instead of by line.
122         Now that we merge lines, it's not convenient to scavenge by line.
123
124         (bmalloc::Allocator::allocateSmallBumpRange):
125         (bmalloc::Allocator::allocateMediumBumpRange): Allocate whole ranges
126         instead of individual lines.
127
128         (bmalloc::Allocator::allocateSlowCase):
129         (bmalloc::Allocator::allocateSmallLine): Deleted.
130         (bmalloc::Allocator::allocateMediumLine): Deleted.
131         (bmalloc::Allocator::allocateMedium): Deleted.
132         * bmalloc/Allocator.h:
133         (bmalloc::Allocator::allocateFastCase): Folded medium allocations
134         into the standard fast path with small allocations. Since a BumpAllocator
135         just allocates out of an arbitrary range, it doesn't need to distinguish
136         between small and medium lines.
137
138         * bmalloc/BumpAllocator.h:
139         (bmalloc::BumpAllocator::size):
140         (bmalloc::BumpAllocator::BumpAllocator):
141         (bmalloc::BumpAllocator::init):
142         (bmalloc::BumpAllocator::refill):
143         (bmalloc::BumpAllocator::line): Deleted. No need to track line information
144         anymore: the heap just gives us a pointer and a pre-computed number of
145         objects, and we allocate them.
146
147         * bmalloc/Deallocator.cpp:
148         (bmalloc::Deallocator::processObjectLog): Updated for interface changes.
149
150         * bmalloc/Heap.cpp:
151         (bmalloc::Heap::Heap):
152         (bmalloc::Heap::initializeLineMetadata): Pre-compute precise metadata
153         detailing where all objects will lie in memory. After we merge two lines,
154         we might allocate an object that spans from one line to the next. This
155         metadata details which bits of memory overlap in that way, and how they
156         overlap.
157
158         (bmalloc::Heap::refillSmallBumpRangeCache):
159         (bmalloc::Heap::refillMediumBumpRangeCache): Scan a whole page at a time,
160         and merge adjacent free lines into BumpRanges.
161
162         (bmalloc::Heap::allocateSmallPage):
163         (bmalloc::Heap::allocateMediumPage):
164         (bmalloc::Heap::deallocateSmallLine):
165         (bmalloc::Heap::deallocateMediumLine): Track pages rather than lines,
166         since we scan for free memory a page at a time.
167
168         (bmalloc::Heap::allocateSmallLineSlowCase): Deleted.
169         (bmalloc::Heap::allocateMediumLineSlowCase): Deleted. Folded into the
170         fast path.
171
172         * bmalloc/Heap.h:
173         (bmalloc::Heap::derefSmallLine):
174         (bmalloc::Heap::derefMediumLine):
175         (bmalloc::Heap::deallocateSmallLine): Deleted.
176         (bmalloc::Heap::allocateSmallLine): Deleted.
177         (bmalloc::Heap::deallocateMediumLine): Deleted.
178         (bmalloc::Heap::allocateMediumLine): Deleted. Updated for interface changes.
179
180         * bmalloc/Line.h:
181         (bmalloc::Line<Traits>::ref):
182         (bmalloc::Line<Traits>::deref):
183         (bmalloc::Line<Traits>::concurrentRef): Deleted. We don't pass a derefCount
184         anymore, since we only ever deref by 1 now.
185
186         * bmalloc/MediumAllocator.h:
187         (bmalloc::MediumAllocator::isNull): Deleted.
188         (bmalloc::MediumAllocator::MediumAllocator): Deleted.
189         (bmalloc::MediumAllocator::line): Deleted.
190         (bmalloc::MediumAllocator::allocate): Deleted.
191         (bmalloc::MediumAllocator::derefCount): Deleted.
192         (bmalloc::MediumAllocator::refill): Deleted.
193         (bmalloc::MediumAllocator::clear): Deleted. Deleted some code that's
194         been dead for a while, since it doesn't build anymore with this patch.
195
196         * bmalloc/Page.h:
197         (bmalloc::Page::sizeClass):
198         (bmalloc::Page::setSizeClass):
199         (bmalloc::Page::smallSizeClass): Deleted.
200         (bmalloc::Page::setSmallSizeClass): Deleted. Renamed setSmallSizeClass
201         to sizeClass, since we use it for medium sizes too.
202
203         * bmalloc/Sizes.h:
204         (bmalloc::Sizes::sizeClass):
205         (bmalloc::Sizes::objectSize): Shrank line sizes to save memory.
206
207         (bmalloc::Sizes::smallSizeClassFor): Deleted.
208         (bmalloc::Sizes::mediumSizeClassFor): Deleted.
209
210         * bmalloc/bmalloc.h:
211         (bmalloc::api::realloc): Now that we have precise objects sizes, realloc
212         can be a bit more precise. It also has to be, since we can't guarantee
213         that an object ends at the end of a line anymore.
214
215 2014-09-19  Daniel Bates  <dabates@apple.com>
216
217         Always assume internal SDK when building configuration Production
218         https://bugs.webkit.org/show_bug.cgi?id=136925
219         <rdar://problem/18362399>
220
221         Reviewed by Dan Bernstein.
222
223         * Configurations/Base.xcconfig:
224
225 2014-09-16  Geoffrey Garen  <ggaren@apple.com>
226
227         bmalloc: moved line caches from the deallocator to the allocator
228         https://bugs.webkit.org/show_bug.cgi?id=136868
229
230         Reviewed by Gavin Barraclough.
231
232         I did this mostly as a simplification, to make it easier to change the
233         allocation strategy.
234
235         No throughput change on MallocBench. Saves about 50kB.
236
237         Since the deallocator needs to lock the heap when freeing lines anyway,
238         there isn't much benefit to giving the deallocator a local cache of
239         deallocated lines.
240
241         We still give the allocator a local cache of lines because that does
242         reduce the frequency at which it needs to lock the heap in order to
243         acquire more lines.
244
245         * bmalloc/Allocator.cpp:
246         (bmalloc::Allocator::scavenge):
247         (bmalloc::Allocator::allocateSmallLine):
248         (bmalloc::Allocator::allocateMediumLine):
249         (bmalloc::Allocator::allocateMedium):
250         (bmalloc::Allocator::allocateSlowCase):
251         * bmalloc/Allocator.h:
252         * bmalloc/Deallocator.cpp:
253         (bmalloc::Deallocator::Deallocator):
254         (bmalloc::Deallocator::scavenge):
255         (bmalloc::Deallocator::processObjectLog):
256         (bmalloc::Deallocator::deallocateSmallLine): Deleted.
257         (bmalloc::Deallocator::allocateSmallLine): Deleted.
258         (bmalloc::Deallocator::deallocateMediumLine): Deleted.
259         (bmalloc::Deallocator::allocateMediumLine): Deleted.
260         * bmalloc/Deallocator.h:
261
262         * bmalloc/Sizes.h:
263         * bmalloc/VMAllocate.h: Took the opportunity to make the line cache size
264         exactly one page in size. That's about what we were shooting for anyway,
265         and it may make it easier to switch to per-page allocation in future.
266
267 2014-09-15  Geoffrey Garen  <ggaren@apple.com>
268
269         bmalloc: allocate small and medium objects using the same bump pointer class
270         https://bugs.webkit.org/show_bug.cgi?id=136843
271
272         Reviewed by Gavin Barraclough.
273
274         4% speedup on MallocBench.
275
276         Now that medium-sized objects have dedicated per-size allocators, they
277         don't need to use an arbitrary bump pointer allocator. This means that
278         every allocator knows how many objects it will allocate from the start,
279         and we don't need a post-processing step to adjust refcounts based on
280         real allocation count.
281
282         * bmalloc.xcodeproj/project.pbxproj: Renamed SmallAllocator to BumpAllocator
283         since it's used for small and medium objects now.
284
285         * bmalloc/Allocator.cpp:
286         (bmalloc::Allocator::Allocator): Updated to use new interface.
287         (bmalloc::Allocator::scavenge): To "retire" an allocator, we just need
288         to make sure that we finish allocating all the objects in it.
289
290         (bmalloc::Allocator::allocateMedium):
291         (bmalloc::Allocator::allocateSlowCase):
292         (bmalloc::Allocator::retire): Deleted.
293         (bmalloc::Allocator::processSmallAllocatorLog): Deleted.
294         (bmalloc::Allocator::processMediumAllocatorLog): Deleted.
295         * bmalloc/Allocator.h:
296         (bmalloc::Allocator::allocateFastCase): Removed abstractions and data
297         used to post-process an allocator based on how many objects it allocated.
298
299         * bmalloc/BumpAllocator.h: Copied from Source/bmalloc/bmalloc/SmallAllocator.h.
300         (bmalloc::BumpAllocator::BumpAllocator):
301         (bmalloc::BumpAllocator::init):
302         (bmalloc::BumpAllocator::line):
303         (bmalloc::BumpAllocator::validate):
304         (bmalloc::BumpAllocator::allocate):
305         (bmalloc::BumpAllocator::refill):
306         (bmalloc::BumpAllocator::clear): Updated these functions to be agnostic
307         about the kinds of lines they allocate into. In some cases, the line
308         type must be provided as a template parameter by the caller.
309
310         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
311         (bmalloc::SmallAllocator::line): Deleted.
312         (bmalloc::SmallAllocator::allocate): Deleted.
313         (bmalloc::SmallAllocator::objectCount): Deleted.
314         (bmalloc::SmallAllocator::derefCount): Deleted.
315         (bmalloc::SmallAllocator::refill): Deleted.
316         (bmalloc::SmallAllocator::clear): Deleted.
317
318         * bmalloc/ObjectType.h:
319         (bmalloc::isMedium):
320
321         * bmalloc/SmallAllocator.h:
322         (bmalloc::SmallAllocator::isNull): Deleted.
323         (bmalloc::SmallAllocator::canAllocate): Deleted.
324         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
325         (bmalloc::SmallAllocator::line): Deleted.
326         (bmalloc::SmallAllocator::allocate): Deleted.
327         (bmalloc::SmallAllocator::objectCount): Deleted.
328         (bmalloc::SmallAllocator::derefCount): Deleted.
329         (bmalloc::SmallAllocator::refill): Deleted.
330         (bmalloc::SmallAllocator::clear): Deleted.
331
332 2014-09-12  Geoffrey Garen  <ggaren@apple.com>
333
334         Fixed a goof in bmalloc Vector sizing
335         https://bugs.webkit.org/show_bug.cgi?id=136795
336
337         Reviewed by Gavin Barraclough and Sam Weinig.
338
339         We want our minimum vector to be page-sized since the OS will give us
340         a page no matter what -- but we want that many bytes, and not enough
341         bytes to store that many elements.
342
343         * bmalloc/Vector.h: Math is hard.
344
345 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
346
347         bmalloc should segregate medium-sized objects by line like it does for small-sized objects
348         https://bugs.webkit.org/show_bug.cgi?id=136693
349
350         Reviewed by Gavin Barraclough.
351
352         4% reduction in heap size on the MallocBench *_memory_warning benchmarks.
353
354         No throughput change.
355
356         We keep an array of medium allocators, just like our array of small
357         allocators.
358
359         In future, we can simplify the allocation fast path by merging the small
360         and medium allocator arrays. For now, this is the simplest change that
361         gets the win.
362
363         * bmalloc/Allocator.cpp:
364         (bmalloc::Allocator::Allocator):
365         (bmalloc::Allocator::scavenge):
366         (bmalloc::Allocator::allocateMedium):
367         * bmalloc/Allocator.h:
368         * bmalloc/Sizes.h:
369         (bmalloc::Sizes::mediumSizeClassFor):
370
371 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
372
373         Reviewed by Sam Weinig.
374
375         Renamed log => retire for clarity.
376
377         * bmalloc/Allocator.cpp:
378         (bmalloc::Allocator::scavenge):
379         (bmalloc::Allocator::retire):
380         (bmalloc::Allocator::allocateMedium):
381         (bmalloc::Allocator::allocateSlowCase):
382         (bmalloc::Allocator::log): Deleted.
383         * bmalloc/Allocator.h:
384
385 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
386
387         bmalloc: eager scavenge leaves behind a bogus allocator
388         https://bugs.webkit.org/show_bug.cgi?id=136743
389
390         Reviewed by Sam Weinig.
391
392         Be sure to clear the allocator after logging it in the eager scavenge
393         case, so that we don't later try to allocate out of the lines that we
394         have thrown away.
395
396         We didn't need to do this previously because scavenge would only happen
397         at thread exit time, after which no further allocation from the per-thread
398         cache would take place.
399
400         * bmalloc/Allocator.cpp:
401         (bmalloc::Allocator::scavenge):
402         * bmalloc/MediumAllocator.h:
403         (bmalloc::MediumAllocator::clear):
404         * bmalloc/SmallAllocator.h:
405         (bmalloc::SmallAllocator::clear):
406
407 2014-09-05  Geoffrey Garen  <ggaren@apple.com>
408
409         bmalloc should honor the FastMalloc statistics API
410         https://bugs.webkit.org/show_bug.cgi?id=136592
411
412         Reviewed by Gavin Barraclough.
413
414         We do this by tracking "size" and "capacity" in the VM heap.
415
416         The VM heap's "capacity" is all the VM we ever allocated.
417
418         The VM heap's "size" the subset of VM currently held onto by the
419         VM heap (and therefore not in use by the regular heap).
420
421         Somewhat ironically, reducing the process's memory footprint, increases
422         the size of the VM heap, since the VM heap holds the pages that are
423         purely virtual and not physical.
424
425         * bmalloc/Heap.cpp:
426         (bmalloc::Heap::size):
427         (bmalloc::Heap::capacity):
428         * bmalloc/Heap.h:
429         * bmalloc/VMHeap.cpp:
430         (bmalloc::VMHeap::VMHeap):
431         (bmalloc::VMHeap::allocateSmallChunk):
432         (bmalloc::VMHeap::allocateMediumChunk):
433         (bmalloc::VMHeap::allocateLargeChunk):
434         * bmalloc/VMHeap.h:
435         (bmalloc::VMHeap::size):
436         (bmalloc::VMHeap::capacity):
437         (bmalloc::VMHeap::allocateSmallPage):
438         (bmalloc::VMHeap::allocateMediumPage):
439         (bmalloc::VMHeap::allocateLargeRange):
440         (bmalloc::VMHeap::deallocateSmallPage):
441         (bmalloc::VMHeap::deallocateMediumPage):
442         (bmalloc::VMHeap::deallocateLargeRange):
443         * bmalloc/bmalloc.h:
444         (bmalloc::api::heapSize):
445         (bmalloc::api::heapCapacity):
446
447 2014-09-02  Geoffrey Garen  <ggaren@apple.com>
448
449         bmalloc crashes on the EWS bots (due to bad large object allocation)
450         https://bugs.webkit.org/show_bug.cgi?id=136469
451
452         Reviewed by Andreas Kling.
453
454         It's possible to convince bmalloc to perform a bad large object allocation,
455         through these steps:
456
457         (1) Insert object A into freelist F0.
458
459         (2) Split, merge and split again A's neighbors such that object B is
460         inserted into freelist F0, with boundary tag and size equal to object A,
461         but pointer not completely equal to object A. Put object B at the head of F0.
462
463         (3) Allocate some other object from F0, swapping its position in the
464         freelist with object B, such that object A is now ahead of object B.
465
466         --> Now, the next allocation for size A/B will allocate object A, which
467         has a slightly wrong idea about where the object actually begins.
468         Immediately, you'll corrupt a little memory, and over time, you'll also
469         corrupt boundary tag metadata.
470
471         The solution is to store the begin pointer in the boundary tag. Luckily,
472         this doesn't make the tag any bigger, and it's not a noticeable slowdown
473         on MallocBench.
474
475         * bmalloc/Algorithm.h:
476         (bmalloc::rightShift):
477         * bmalloc/BeginTag.h:
478         (bmalloc::BeginTag::isInFreeList): This is the bug fix. Make sure to
479         validate the start pointer when popping off the free list. Through a
480         very uncommon set of steps, it is possible to have an item in the free
481         list that is valid by all accounts except for its start pointer.
482
483         * bmalloc/BoundaryTag.h:
484         (bmalloc::BoundaryTag::compactBegin):
485         (bmalloc::BoundaryTag::setRange):
486         (bmalloc::BoundaryTag::setSize): Deleted. Record a compact version of the
487         start pointer. We don't need the whole pointer -- just the offset, in
488         largeAlignment increments, into the relevant boundary tag bucket.
489
490         * bmalloc/BoundaryTagInlines.h:
491         (bmalloc::validateNext):
492         (bmalloc::BoundaryTag::init):
493         (bmalloc::BoundaryTag::mergeLarge):
494         (bmalloc::BoundaryTag::splitLarge):
495         * bmalloc/SegregatedFreeList.cpp:
496         (bmalloc::SegregatedFreeList::insert):
497         (bmalloc::SegregatedFreeList::takeGreedy):
498         (bmalloc::SegregatedFreeList::take): Provide the whole range instead of
499         the size when establishing a boundary tag, as required by the new
500         interface.
501
502         * bmalloc/Sizes.h:
503
504 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
505
506         Fixed a bmalloc crash seen on the EWS bot
507         https://bugs.webkit.org/show_bug.cgi?id=135955
508
509         Reviewed by Andreas Kling.
510
511         * bmalloc/Syscall.h: Some CG APIs vm_copy their input buffers. If the
512         input buffer is a malloc region, that region will get marked Copy-On-Write
513         by the kernel. Calls to madvise() for COW regions fail and return EINVAL
514         on older OS X's. In 10.10, they still fail, but they do not return
515         EINVAL.
516
517         So, we can only ASSERT that our syscalls succeed starting with 10.10.
518
519 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
520
521         Fixed the bmalloc build
522         https://bugs.webkit.org/show_bug.cgi?id=135953
523
524         Reviewed by Andreas Kling.
525
526         * bmalloc.xcodeproj/project.pbxproj: Marked a few headers as private.
527         These headers are used, so they must be available outside the project.
528
529 2014-08-13  Daniel Bates  <dabates@apple.com>
530
531         Attempt to fix the build following <http://trac.webkit.org/changeset/172576>
532         (https://bugs.webkit.org/show_bug.cgi?id=135895)
533
534         Substitute PerThreadStorage<T>::initSharedKeyIfNeeded() for initSharedKeyIfNeeded() in
535         implementation of PerThread<T>::getFastCase().
536
537         * bmalloc/PerThread.h:
538         (bmalloc::PerThread<T>::getFastCase):
539
540 2014-08-13  Daniel Bates  <dabates@apple.com>
541
542         Make bmalloc::PerThread work without C++ thread local storage
543         https://bugs.webkit.org/show_bug.cgi?id=135895
544
545         Reviewed by Geoffrey Garen.
546
547         Implement support for building bmalloc without C++ thread local storage.
548
549         * bmalloc/BPlatform.h: Remove macro define BPLATFORM_IOS_SIMULATOR. Added macro function
550         BCOMPILER_SUPPORTS() and macro define BCOMPILER_SUPPORTS_CXX_THREAD_LOCAL that can be used
551         to determine whether the compiler supports C++ thread local storage.
552         * bmalloc/PerThread.h:
553         (bmalloc::PerThreadStorage::get): Modified to call pthread_getspecific() when building
554         without C++ thread local storage.
555         (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Added.
556         (bmalloc::PerThreadStorage::init): Moved logic to initialize shared Pthread key from here to
557         PerThreadStorage::initSharedKeyIfNeeded().
558         (bmalloc::PerThread<T>::getFastCase): Modified to call PerThreadStorage::initSharedKeyIfNeeded()
559         before querying PerThreadStorage::get() when building without C++ thread local storage so as to
560         ensure that the shared key has been initialized.
561         (_pthread_setspecific_direct): Deleted.
562         (_pthread_getspecific_direct): Deleted.
563
564 2014-08-13  Daniel Bates  <dabates@apple.com>
565
566         [iOS] Make JavaScriptCore and bmalloc build with the public SDK
567         https://bugs.webkit.org/show_bug.cgi?id=135848
568
569         Reviewed by Geoffrey Garen.
570
571         * bmalloc/BPlatform.h: Added macro BPLATFORM_IOS_SIMULATOR, which evaluates to true
572         when building for the iOS Simulator.
573         * bmalloc/PerThread.h: Use pthread_machdep.h code path when building for iOS Simulator
574         using the public SDK.
575         (_pthread_setspecific_direct): Added; only defined when building for the iOS Simulator
576         using the public SDK.
577         (_pthread_getspecific_direct): Added; only defined when building for the iOS Simulator
578         using the public SDK.
579
580 2014-08-12  Daniel Bates  <dabates@apple.com>
581
582         BPLATFORM(IOS) always evaluates to false
583         https://bugs.webkit.org/show_bug.cgi?id=135843
584
585         Reviewed by Geoffrey Garen.
586
587         Fix typo in definition of BPLATFORM() and include system header TargetConditionals.h
588         (when building on an Apple platform) so that BPLATFORM(X) evaluates to true when
589         building for platform X. In particular, so that BPLATFORM(IOS) evaluates to true when
590         building for iOS.
591
592         As a side effect of this change, the change made in <http://trac.webkit.org/changeset/167289>
593         will be honored and iOS will assume a VM page size of 16kB (again) instead of 4kB.
594
595         * bmalloc/BPlatform.h:
596
597 2014-08-11  Andy Estes  <aestes@apple.com>
598
599         [iOS] Get rid of iOS.xcconfig
600         https://bugs.webkit.org/show_bug.cgi?id=135809
601
602         Reviewed by Joseph Pecoraro.
603
604         All iOS.xcconfig did was include AspenFamily.xcconfig, so there's no need for the indirection.
605
606         * Configurations/Base.xcconfig:
607         * Configurations/iOS.xcconfig: Removed.
608         * bmalloc.xcodeproj/project.pbxproj:
609
610 2014-05-01  Dan Bernstein  <mitz@apple.com>
611
612         Fixed production builds for the iOS Simulator.
613         <rdar://problem/16792221>
614
615         * Configurations/bmalloc.xcconfig: Include INSTALL_PATH_PREFIX in
616         PRIVATE_HEADERS_FOLDER_PATH when installing.
617
618 2014-04-20  Geoffrey Garen  <ggaren@apple.com>
619
620         bmalloc: Segregate pages by objects size
621         https://bugs.webkit.org/show_bug.cgi?id=131909
622
623         Reviewed by Andreas Kling.
624
625         2% reduction in memory-at-end on the Membuster memory_warning benchmarks.
626
627         * bmalloc/Allocator.cpp:
628         (bmalloc::Allocator::allocateSlowCase):
629         * bmalloc/Allocator.h:
630         (bmalloc::Allocator::allocateFastCase):
631         (bmalloc::Allocator::smallAllocatorFor): Use the new shared helper
632         function for size class calculation.
633
634         * bmalloc/Deallocator.cpp:
635         (bmalloc::Deallocator::Deallocator):
636         (bmalloc::Deallocator::scavenge):
637         (bmalloc::Deallocator::deallocateSmallLine):
638         (bmalloc::Deallocator::allocateSmallLine):
639         * bmalloc/Deallocator.h: Keep a cache for every size class, since the
640         cache can't be shared anymore.
641
642         * bmalloc/Heap.cpp:
643         (bmalloc::Heap::allocateSmallLineSlowCase):
644         * bmalloc/Heap.h:
645         (bmalloc::Heap::deallocateSmallLine): Ditto.
646
647         (bmalloc::Heap::allocateSmallLine): Check size class in addition to
648         page refcount when allocating a line because we might have deallocated
649         the page and the recycled it for another size class.
650
651         (bmalloc::Heap::deallocateMediumLine):
652         (bmalloc::Heap::allocateMediumLine):
653         * bmalloc/Line.h:
654         (bmalloc::Line::refCount):
655         * bmalloc/Page.h:
656         (bmalloc::Page::refCount):
657         (bmalloc::Page::smallSizeClass):
658         (bmalloc::Page::setSmallSizeClass):
659         (bmalloc::Page<Traits>::refCount): Deleted.
660         * bmalloc/Sizes.h:
661         (bmalloc::Sizes::smallSizeClassFor): New shared API for computing
662         an index into an array from a size.
663
664 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
665
666         bmalloc: Improved alignment in LargeChunk
667         https://bugs.webkit.org/show_bug.cgi?id=131895
668
669         Reviewed by Andreas Kling.
670
671         * bmalloc/Chunk.h:
672         * bmalloc/LargeChunk.h: Align to vmPageSize just like Chunk does.
673         Technically, the previous alignment was harmless, but I would prefer,
674         dear reader, not to have to explain the interlocking set of
675         circumstances that made it so.
676
677 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
678
679         Rolled out r167502 because it caused a crash on the facebook benchmark.
680
681         Unreviewed.
682
683             bmalloc: Added an XSmall line size
684             https://bugs.webkit.org/show_bug.cgi?id=131851
685
686             Reviewed by Sam Weinig.
687
688 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
689
690         bmalloc: Mutex should be harder to use wrong
691         https://bugs.webkit.org/show_bug.cgi?id=131879
692
693         Reviewed by Andreas Kling.
694
695         Mutex now has a proper constructor, so you can't deadlock by forgetting
696         to initialize it.
697
698         * bmalloc.xcodeproj/project.pbxproj:
699         * bmalloc/Allocator.cpp:
700         (bmalloc::Allocator::processXSmallAllocatorLog):
701         (bmalloc::Allocator::processSmallAllocatorLog):
702         (bmalloc::Allocator::processMediumAllocatorLog):
703         (bmalloc::Allocator::allocateLarge):
704         (bmalloc::Allocator::allocateXLarge): Global replace Mutex => StaticMutex,
705         since the Heap mutex is a static.
706
707         * bmalloc/AsyncTask.h:
708         (bmalloc::Function>::AsyncTask): Use Mutex, since we're not static. No
709         need for explicit initialization anymore.
710
711         * bmalloc/Deallocator.cpp:
712         (bmalloc::Deallocator::scavenge):
713         (bmalloc::Deallocator::deallocateLarge):
714         (bmalloc::Deallocator::deallocateXLarge):
715         (bmalloc::Deallocator::processObjectLog):
716         (bmalloc::Deallocator::deallocateSmallLine):
717         (bmalloc::Deallocator::deallocateXSmallLine):
718         (bmalloc::Deallocator::allocateSmallLine):
719         (bmalloc::Deallocator::allocateXSmallLine):
720         (bmalloc::Deallocator::deallocateMediumLine):
721         (bmalloc::Deallocator::allocateMediumLine):
722         * bmalloc/Deallocator.h:
723         * bmalloc/Heap.cpp:
724         (bmalloc::sleep):
725         (bmalloc::Heap::Heap):
726         (bmalloc::Heap::concurrentScavenge):
727         (bmalloc::Heap::scavenge):
728         (bmalloc::Heap::scavengeSmallPages):
729         (bmalloc::Heap::scavengeXSmallPages):
730         (bmalloc::Heap::scavengeMediumPages):
731         (bmalloc::Heap::scavengeLargeRanges):
732         (bmalloc::Heap::allocateXSmallLineSlowCase):
733         (bmalloc::Heap::allocateSmallLineSlowCase):
734         (bmalloc::Heap::allocateMediumLineSlowCase):
735         (bmalloc::Heap::allocateXLarge):
736         (bmalloc::Heap::deallocateXLarge):
737         (bmalloc::Heap::allocateLarge):
738         (bmalloc::Heap::deallocateLarge):
739         * bmalloc/Heap.h:
740         (bmalloc::Heap::deallocateXSmallLine):
741         (bmalloc::Heap::allocateXSmallLine):
742         (bmalloc::Heap::deallocateSmallLine):
743         (bmalloc::Heap::allocateSmallLine):
744         (bmalloc::Heap::deallocateMediumLine):
745         (bmalloc::Heap::allocateMediumLine):
746         * bmalloc/Line.h:
747         (bmalloc::Line<Traits>::deref):
748         * bmalloc/Mutex.cpp: Removed.
749         * bmalloc/Mutex.h:
750         (bmalloc::Mutex::Mutex):
751         (bmalloc::Mutex::init): Deleted.
752         (bmalloc::Mutex::try_lock): Deleted.
753         (bmalloc::Mutex::lock): Deleted.
754         (bmalloc::Mutex::unlock): Deleted.
755         * bmalloc/Page.h:
756         (bmalloc::Page<Traits>::ref):
757         (bmalloc::Page<Traits>::deref):
758         (bmalloc::Page<Traits>::refCount):
759         * bmalloc/PerProcess.h:
760         (bmalloc::PerProcess::mutex):
761         (bmalloc::PerProcess<T>::getSlowCase):
762         * bmalloc/StaticMutex.cpp: Added.
763         (bmalloc::StaticMutex::lockSlowCase):
764         * bmalloc/StaticMutex.h: Added.
765         (bmalloc::StaticMutex::init):
766         (bmalloc::StaticMutex::try_lock):
767         (bmalloc::StaticMutex::lock):
768         (bmalloc::StaticMutex::unlock):
769         * bmalloc/VMHeap.h:
770         (bmalloc::VMHeap::deallocateXSmallPage):
771         (bmalloc::VMHeap::deallocateSmallPage):
772         (bmalloc::VMHeap::deallocateMediumPage):
773         (bmalloc::VMHeap::deallocateLargeRange):
774         * bmalloc/bmalloc.h:
775         (bmalloc::api::scavenge): Global replace Mutex => StaticMutex,
776         since the Heap mutex is a static.
777
778 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
779
780         bmalloc: AsyncTask should use Mutex instead of std::mutex
781         https://bugs.webkit.org/show_bug.cgi?id=131865
782
783         Reviewed by Gavin Barraclough.
784
785         std::mutex is so slow that it makes parallelizing simple tasks through
786         AsyncTask a net regression. Mutex fixes this.
787
788         * bmalloc/AsyncTask.h:
789         (bmalloc::Function>::AsyncTask):
790         (bmalloc::Function>::join):
791         (bmalloc::Function>::runSlowCase):
792         (bmalloc::Function>::entryPoint):
793         * bmalloc/Mutex.h:
794         (bmalloc::Mutex::init):
795
796 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
797
798         bmalloc: Added an XSmall line size
799         https://bugs.webkit.org/show_bug.cgi?id=131851
800
801         Reviewed by Sam Weinig.
802
803         Reduces malloc footprint on Membuster recordings by 10%.
804
805         This is a throughput regression, but we're still way ahead of TCMalloc.
806         I have some ideas for how to recover the regression -- but I wanted to
807         get this win in first.
808
809         Full set of benchmark results:
810
811                 bmalloc> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks --measure-heap nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
812                                                                                                 
813                                                                nopatch                      patch                                Δ
814                 Peak Memory:
815                     reddit_memory_warning                      7,896kB                    7,532kB                  ^ 1.05x smaller
816                     flickr_memory_warning                     12,968kB                   12,324kB                  ^ 1.05x smaller
817                     theverge_memory_warning                   16,672kB                   15,200kB                   ^ 1.1x smaller
818
819                     <geometric mean>                          11,952kB                   11,216kB                  ^ 1.07x smaller
820                     <arithmetic mean>                         12,512kB                   11,685kB                  ^ 1.07x smaller
821                     <harmonic mean>                           11,375kB                   10,726kB                  ^ 1.06x smaller
822
823                 Memory at End:
824                     reddit_memory_warning                      7,320kB                    6,856kB                  ^ 1.07x smaller
825                     flickr_memory_warning                     10,848kB                    9,692kB                  ^ 1.12x smaller
826                     theverge_memory_warning                   16,380kB                   14,872kB                   ^ 1.1x smaller
827
828                     <geometric mean>                          10,916kB                    9,961kB                   ^ 1.1x smaller
829                     <arithmetic mean>                         11,516kB                   10,473kB                   ^ 1.1x smaller
830                     <harmonic mean>                           10,350kB                    9,485kB                  ^ 1.09x smaller
831
832                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
833                                                                                                 
834                                                    nopatch                patch                         Δ
835                 Execution Time:
836                     churn                            127ms                151ms            ! 1.19x slower
837                     list_allocate                    130ms                164ms            ! 1.26x slower
838                     tree_allocate                    109ms                127ms            ! 1.17x slower
839                     tree_churn                       115ms                120ms            ! 1.04x slower
840                     facebook                         240ms                259ms            ! 1.08x slower
841                     fragment                          91ms                131ms            ! 1.44x slower
842                     fragment_iterate                 105ms                106ms            ! 1.01x slower
843                     message_one                      260ms                259ms             ^ 1.0x faster
844                     message_many                     149ms                154ms            ! 1.03x slower
845                     medium                           194ms                248ms            ! 1.28x slower
846                     big                              157ms                160ms            ! 1.02x slower
847
848                     <geometric mean>                 144ms                163ms            ! 1.13x slower
849                     <arithmetic mean>                152ms                171ms            ! 1.12x slower
850                     <harmonic mean>                  137ms                156ms            ! 1.14x slower
851
852                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
853                                                                                                 
854                                                                        nopatch                          patch                                     Δ
855                 Execution Time:
856                     churn                                                126ms                          148ms                        ! 1.17x slower
857                     churn --parallel                                      62ms                           76ms                        ! 1.23x slower
858                     list_allocate                                        130ms                          164ms                        ! 1.26x slower
859                     list_allocate --parallel                             120ms                          175ms                        ! 1.46x slower
860                     tree_allocate                                        111ms                          127ms                        ! 1.14x slower
861                     tree_allocate --parallel                              95ms                          135ms                        ! 1.42x slower
862                     tree_churn                                           115ms                          124ms                        ! 1.08x slower
863                     tree_churn --parallel                                107ms                          126ms                        ! 1.18x slower
864                     facebook                                             240ms                          276ms                        ! 1.15x slower
865                     facebook --parallel                                  802ms                        1,088ms                        ! 1.36x slower
866                     fragment                                              92ms                          130ms                        ! 1.41x slower
867                     fragment --parallel                                   66ms                          124ms                        ! 1.88x slower
868                     fragment_iterate                                     109ms                          127ms                        ! 1.17x slower
869                     fragment_iterate --parallel                           55ms                           64ms                        ! 1.16x slower
870                     message_one                                          260ms                          260ms                                      
871                     message_many                                         170ms                          238ms                         ! 1.4x slower
872                     medium                                               185ms                          250ms                        ! 1.35x slower
873                     medium --parallel                                    210ms                          334ms                        ! 1.59x slower
874                     big                                                  150ms                          169ms                        ! 1.13x slower
875                     big --parallel                                       138ms                          144ms                        ! 1.04x slower
876
877                     <geometric mean>                                     135ms                          170ms                        ! 1.26x slower
878                     <arithmetic mean>                                    167ms                          214ms                        ! 1.28x slower
879                     <harmonic mean>                                      117ms                          148ms                        ! 1.26x slower
880
881                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
882
883                                                                     TC                      patch                                Δ
884                 Peak Memory:
885                     reddit_memory_warning                     13,836kB                   13,436kB                  ^ 1.03x smaller
886                     flickr_memory_warning                     24,868kB                   25,188kB                   ! 1.01x bigger
887                     theverge_memory_warning                   24,504kB                   26,636kB                   ! 1.09x bigger
888
889                     <geometric mean>                          20,353kB                   20,812kB                   ! 1.02x bigger
890                     <arithmetic mean>                         21,069kB                   21,753kB                   ! 1.03x bigger
891                     <harmonic mean>                           19,570kB                   19,780kB                   ! 1.01x bigger
892
893                 Memory at End:
894                     reddit_memory_warning                      8,656kB                   10,016kB                   ! 1.16x bigger
895                     flickr_memory_warning                     11,844kB                   13,784kB                   ! 1.16x bigger
896                     theverge_memory_warning                   18,516kB                   22,748kB                   ! 1.23x bigger
897
898                     <geometric mean>                          12,382kB                   14,644kB                   ! 1.18x bigger
899                     <arithmetic mean>                         13,005kB                   15,516kB                   ! 1.19x bigger
900                     <harmonic mean>                           11,813kB                   13,867kB                   ! 1.17x bigger
901
902                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
903                                                                                                 
904                                                         TC                patch                         Δ
905                 Execution Time:
906                     churn                            416ms                148ms            ^ 2.81x faster
907                     list_allocate                    463ms                164ms            ^ 2.82x faster
908                     tree_allocate                    292ms                127ms             ^ 2.3x faster
909                     tree_churn                       157ms                120ms            ^ 1.31x faster
910                     facebook                         327ms                276ms            ^ 1.18x faster
911                     fragment                         335ms                129ms             ^ 2.6x faster
912                     fragment_iterate                 344ms                108ms            ^ 3.19x faster
913                     message_one                      386ms                258ms             ^ 1.5x faster
914                     message_many                     410ms                154ms            ^ 2.66x faster
915                     medium                           391ms                245ms             ^ 1.6x faster
916                     big                              261ms                167ms            ^ 1.56x faster
917
918                     <geometric mean>                 332ms                164ms            ^ 2.02x faster
919                     <arithmetic mean>                344ms                172ms            ^ 1.99x faster
920                     <harmonic mean>                  317ms                157ms            ^ 2.02x faster
921
922         * bmalloc.xcodeproj/project.pbxproj:
923         * bmalloc/Allocator.cpp:
924         (bmalloc::Allocator::Allocator): Don't assume that each allocator's
925         index corresponds with its size. Instead, use the size selection function
926         explicitly. Now that we have XSmall, some small allocator entries are
927         unused.
928
929         (bmalloc::Allocator::scavenge):
930         (bmalloc::Allocator::log):
931         (bmalloc::Allocator::processXSmallAllocatorLog):
932         (bmalloc::Allocator::allocateSlowCase):
933         * bmalloc/Allocator.h:
934         (bmalloc::Allocator::xSmallAllocatorFor):
935         (bmalloc::Allocator::allocateFastCase):
936         * bmalloc/Chunk.h:
937         * bmalloc/Deallocator.cpp:
938         (bmalloc::Deallocator::scavenge):
939         (bmalloc::Deallocator::processObjectLog):
940         (bmalloc::Deallocator::deallocateSlowCase):
941         (bmalloc::Deallocator::deallocateXSmallLine):
942         (bmalloc::Deallocator::allocateXSmallLine):
943         * bmalloc/Deallocator.h:
944         (bmalloc::Deallocator::deallocateFastCase):
945         * bmalloc/Heap.cpp:
946         (bmalloc::Heap::scavenge):
947         (bmalloc::Heap::scavengeXSmallPages):
948         (bmalloc::Heap::allocateXSmallLineSlowCase):
949         * bmalloc/Heap.h:
950         (bmalloc::Heap::deallocateXSmallLine):
951         (bmalloc::Heap::allocateXSmallLine):
952         * bmalloc/LargeChunk.h:
953         (bmalloc::LargeChunk::get):
954         (bmalloc::LargeChunk::endTag):
955         * bmalloc/Line.h:
956         * bmalloc/MediumAllocator.h:
957         (bmalloc::MediumAllocator::allocate):
958         (bmalloc::MediumAllocator::refill):
959         * bmalloc/ObjectType.cpp:
960         (bmalloc::objectType):
961         * bmalloc/ObjectType.h:
962         (bmalloc::isXSmall):
963         (bmalloc::isSmall):
964         (bmalloc::isMedium):
965         (bmalloc::isLarge):
966         (bmalloc::isSmallOrMedium): Deleted.
967         * bmalloc/SegregatedFreeList.h: I boiler-plate copied existing code for
968         handling small objects. There's probably a reasonable way to share this
969         code in the future -- I'll look into that once it's stopped changing.
970
971         * bmalloc/Sizes.h: Tweaked size classes to make Membuster happy. This
972         is the main reason things got slower.
973
974         * bmalloc/SmallAllocator.h:
975         (bmalloc::SmallAllocator::allocate):
976         * bmalloc/SmallTraits.h:
977         * bmalloc/VMHeap.cpp:
978         (bmalloc::VMHeap::allocateXSmallChunk):
979         * bmalloc/VMHeap.h:
980         (bmalloc::VMHeap::allocateXSmallPage):
981         (bmalloc::VMHeap::deallocateXSmallPage):
982         * bmalloc/XSmallAllocator.h: Added.
983         (bmalloc::XSmallAllocator::isNull):
984         (bmalloc::XSmallAllocator::canAllocate):
985         (bmalloc::XSmallAllocator::XSmallAllocator):
986         (bmalloc::XSmallAllocator::line):
987         (bmalloc::XSmallAllocator::allocate):
988         (bmalloc::XSmallAllocator::objectCount):
989         (bmalloc::XSmallAllocator::derefCount):
990         (bmalloc::XSmallAllocator::refill):
991         * bmalloc/XSmallChunk.h: Added.
992         * bmalloc/XSmallLine.h: Added.
993         * bmalloc/XSmallPage.h: Added.
994         * bmalloc/XSmallTraits.h: Added.
995         * bmalloc/bmalloc.h:
996         (bmalloc::api::realloc): Boiler-plate copy, as above.
997
998 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
999
1000         MallocBench should scavenge explicitly instead of waiting
1001         https://bugs.webkit.org/show_bug.cgi?id=131661
1002
1003         Reviewed by Andreas Kling.
1004
1005         Added explicit scavenge support to bmalloc. This isn't a memory win,
1006         since bmalloc's per-thread cache is so small. But it makes testing
1007         simpler.
1008
1009         * bmalloc/Allocator.cpp:
1010         (bmalloc::Allocator::~Allocator):
1011         (bmalloc::Allocator::scavenge):
1012         * bmalloc/Allocator.h:
1013         * bmalloc/Cache.cpp:
1014         (bmalloc::Cache::operator new):
1015         (bmalloc::Cache::operator delete):
1016         (bmalloc::Cache::Cache):
1017         (bmalloc::Cache::scavenge):
1018         * bmalloc/Cache.h:
1019         * bmalloc/Deallocator.cpp:
1020         (bmalloc::Deallocator::~Deallocator):
1021         (bmalloc::Deallocator::scavenge):
1022         * bmalloc/Deallocator.h: Factored existing scavenging code into helper
1023         functions, for reuse.
1024
1025         * bmalloc/Heap.cpp:
1026         (bmalloc::sleep):
1027         (bmalloc::Heap::concurrentScavenge):
1028         (bmalloc::Heap::scavenge):
1029         (bmalloc::Heap::scavengeSmallPages):
1030         (bmalloc::Heap::scavengeMediumPages):
1031         (bmalloc::Heap::scavengeLargeRanges):
1032         * bmalloc/Heap.h: Made scavenge sleep duration a parameter. Forced
1033         scavenging -- in response to a benchmark or a low memory warning --
1034         wants to complete as soon as possible, so its sleep duration is 0.
1035
1036         * bmalloc/bmalloc.h:
1037         (bmalloc::api::scavenge):
1038         * bmalloc/mbmalloc.cpp: Exported the scavenge API for MallocBench's use.
1039
1040 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
1041
1042         Use 4kB pages on Mac
1043         https://bugs.webkit.org/show_bug.cgi?id=131658
1044
1045         Reviewed by Sam Weinig.
1046
1047         This reduces memory use a lot on Membuster:
1048
1049                                                                   base                      patch                                Δ
1050                 Execution Time:
1051                     reddit_memory_warning                         18ms                       17ms                   ^ 1.06x faster
1052                     flickr_memory_warning                         34ms                       36ms                   ! 1.06x slower
1053                     theverge_memory_warning                       39ms                       41ms                   ! 1.05x slower
1054
1055                     <geometric mean>                              29ms                       29ms                   ! 1.02x slower
1056                     <arithmetic mean>                             30ms                       31ms                   ! 1.03x slower
1057                     <harmonic mean>                               27ms                       27ms                    ^ 1.0x faster
1058
1059                 Peak Memory:
1060                     reddit_memory_warning                     16,412kB                   16,436kB                    ! 1.0x bigger
1061                     flickr_memory_warning                     30,120kB                   30,184kB                    ! 1.0x bigger
1062                     theverge_memory_warning                   33,408kB                   33,420kB                    ! 1.0x bigger
1063
1064                     <geometric mean>                          25,466kB                   25,499kB                    ! 1.0x bigger
1065                     <arithmetic mean>                         26,647kB                   26,680kB                    ! 1.0x bigger
1066                     <harmonic mean>                           24,181kB                   24,214kB                    ! 1.0x bigger
1067
1068                 Memory at End:
1069                     reddit_memory_warning                      2,404kB                    1,920kB                  ^ 1.25x smaller
1070                     flickr_memory_warning                      3,764kB                    3,072kB                  ^ 1.23x smaller
1071                     theverge_memory_warning                    3,648kB                    3,132kB                  ^ 1.16x smaller
1072
1073                     <geometric mean>                           3,208kB                    2,644kB                  ^ 1.21x smaller
1074                     <arithmetic mean>                          3,272kB                    2,708kB                  ^ 1.21x smaller
1075                     <harmonic mean>                            3,139kB                    2,574kB                  ^ 1.22x smaller
1076
1077
1078         * bmalloc.xcodeproj/project.pbxproj:
1079         * bmalloc/BPlatform.h: Added.
1080         * bmalloc/VMAllocate.h: Only use 16kB pages on iOS because the page size
1081         is 4kB on Mac.
1082
1083 2014-04-14  Alexey Proskuryakov  <ap@apple.com>
1084
1085         Fixed svn:ignore on bmalloc.xcodeproj, it had erroneous leading spaces.
1086
1087         * bmalloc.xcodeproj: Modified property svn:ignore.
1088
1089 2014-04-13  Geoffrey Garen  <ggaren@apple.com>
1090
1091         Fixed some mbmalloc exports
1092         https://bugs.webkit.org/show_bug.cgi?id=131599
1093
1094         Reviewed by Ryosuke Niwa.
1095
1096         * bmalloc.xcodeproj/project.pbxproj: Made some headers a private part
1097         of the project, so we can call them from API.
1098
1099         * bmalloc/mbmalloc.cpp: Marked the mbmalloc functions with default
1100         visibility, so they show up as exported in the .dylib.
1101
1102 2014-04-09  Geoffrey Garen  <ggaren@apple.com>
1103
1104         Put bmalloc headers in the right place
1105         https://bugs.webkit.org/show_bug.cgi?id=131464
1106
1107         Reviewed by Mark Rowe.
1108
1109         * Configurations/bmalloc.xcconfig: Set PRIVATE_HEADERS_FOLDER_PATH to
1110         specify that we don't just want to dump all of our generically-named
1111         headers into /usr/local/include.
1112
1113 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
1114
1115         Made bmalloc more #include friendly
1116         https://bugs.webkit.org/show_bug.cgi?id=131386
1117
1118         Reviewed by Andreas Kling.
1119
1120         Marked a bunch of headers private so they can be used from client code
1121         that #includes bmalloc.h.
1122
1123         Renamed ASSERT macros to BASSERT. This matches their header, which already
1124         had to be renamed, and fixes conflicts with WTF's ASSERT macros.
1125
1126         * bmalloc.xcodeproj/project.pbxproj:
1127         * bmalloc/Allocator.cpp:
1128         (bmalloc::Allocator::allocateSlowCase):
1129         * bmalloc/AsyncTask.h:
1130         (bmalloc::Function>::runSlowCase):
1131         * bmalloc/BAssert.h:
1132         * bmalloc/BoundaryTag.h:
1133         (bmalloc::BoundaryTag::setSize):
1134         * bmalloc/BoundaryTagInlines.h:
1135         (bmalloc::validate):
1136         (bmalloc::BoundaryTag::init):
1137         (bmalloc::BoundaryTag::deallocate):
1138         (bmalloc::BoundaryTag::splitLarge):
1139         (bmalloc::BoundaryTag::allocate):
1140         * bmalloc/Chunk.h:
1141         * bmalloc/Deallocator.cpp:
1142         (bmalloc::Deallocator::processObjectLog):
1143         (bmalloc::Deallocator::deallocateSlowCase):
1144         * bmalloc/Deallocator.h:
1145         (bmalloc::Deallocator::deallocateFastCase):
1146         * bmalloc/FixedVector.h:
1147         (bmalloc::Capacity>::operator):
1148         (bmalloc::Capacity>::push):
1149         (bmalloc::Capacity>::pop):
1150         (bmalloc::Capacity>::shrink):
1151         * bmalloc/Heap.cpp:
1152         (bmalloc::Heap::allocateLarge):
1153         * bmalloc/LargeChunk.h:
1154         (bmalloc::LargeChunk::get):
1155         (bmalloc::LargeChunk::endTag):
1156         * bmalloc/Line.h:
1157         (bmalloc::Line<Traits>::concurrentRef):
1158         (bmalloc::Line<Traits>::deref):
1159         * bmalloc/MediumAllocator.h:
1160         (bmalloc::MediumAllocator::allocate):
1161         * bmalloc/ObjectType.h:
1162         (bmalloc::isSmall):
1163         * bmalloc/Page.h:
1164         (bmalloc::Page<Traits>::ref):
1165         (bmalloc::Page<Traits>::deref):
1166         * bmalloc/PerThread.h:
1167         (bmalloc::PerThread<T>::getSlowCase):
1168         * bmalloc/SegregatedFreeList.cpp:
1169         (bmalloc::SegregatedFreeList::SegregatedFreeList):
1170         (bmalloc::SegregatedFreeList::insert):
1171         * bmalloc/SmallAllocator.h:
1172         (bmalloc::SmallAllocator::allocate):
1173         (bmalloc::SmallAllocator::refill):
1174         * bmalloc/Syscall.h:
1175         * bmalloc/VMAllocate.h:
1176         (bmalloc::vmValidate):
1177         (bmalloc::vmAllocate):
1178         (bmalloc::vmDeallocatePhysicalPagesSloppy):
1179         * bmalloc/Vector.h:
1180         (bmalloc::Vector<T>::operator):
1181         (bmalloc::Vector<T>::pop):
1182         (bmalloc::Vector<T>::shrink):
1183         * bmalloc/XLargeChunk.h:
1184         (bmalloc::XLargeChunk::range):
1185         (bmalloc::XLargeChunk::size):
1186
1187 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
1188
1189         Removed an unused file.
1190
1191         Unreviewed.
1192
1193         * bmalloc/AsyncTask.cpp: Removed.
1194
1195 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1196
1197         Build bmalloc on Mac
1198         https://bugs.webkit.org/show_bug.cgi?id=131333
1199
1200         Reviewed by Mark Rowe.
1201
1202         * Makefile: Added. For make clients.
1203
1204         These files are required for building any project in WebKit. I copied
1205         them from WTF:
1206         * Configurations: Added.
1207         * Configurations/Base.xcconfig: Added.
1208         * Configurations/DebugRelease.xcconfig: Added.
1209         * Configurations/bmalloc.xcconfig: Added.
1210         * Configurations/iOS.xcconfig: Added.
1211         * Configurations/mbmalloc.xcconfig: Added.
1212
1213         * bmalloc.xcodeproj/project.pbxproj: I removed per-project-file stuff
1214         from here because everything is in .xcconfig files now.
1215
1216         I had to fix a bunch of minor warnings, since they're enabled in our
1217         .xcconfig files:
1218
1219         * bmalloc/AsyncTask.h:
1220         (bmalloc::Function>::AsyncTask):
1221         * bmalloc/BAssert.h:
1222         * bmalloc/BoundaryTagInlines.h:
1223         (bmalloc::validate):
1224         * bmalloc/Heap.cpp:
1225         (bmalloc::Heap::Heap):
1226         (bmalloc::Heap::allocateLarge):
1227         (bmalloc::Heap::deallocateLarge):
1228         * bmalloc/Mutex.h:
1229         (bmalloc::Mutex::Mutex): Deleted.
1230         * bmalloc/VMAllocate.h:
1231         (bmalloc::vmValidate):
1232         * bmalloc/mbmalloc.cpp:
1233
1234 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1235
1236         bmalloc: Fixed a leak in the per-thread cache
1237         https://bugs.webkit.org/show_bug.cgi?id=131330
1238
1239         Reviewed by Andreas Kling.
1240
1241         Remember to deallocate our line caches upon thread exit.
1242
1243         * bmalloc/Deallocator.cpp:
1244         (bmalloc::Deallocator::~Deallocator):
1245
1246 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1247
1248         bmalloc: rolled out the tryLock experiment
1249         https://bugs.webkit.org/show_bug.cgi?id=131328
1250
1251         Reviewed by Andreas Kling.
1252
1253         It wasn't a speedup.
1254
1255         * bmalloc.xcodeproj/project.pbxproj:
1256         * bmalloc/Allocator.cpp:
1257         (bmalloc::Allocator::processSmallAllocatorLog):
1258         (bmalloc::Allocator::processMediumAllocatorLog):
1259         * bmalloc/Deallocator.cpp:
1260         (bmalloc::Deallocator::processObjectLog):
1261         (bmalloc::Deallocator::deallocateSlowCase):
1262         (bmalloc::Deallocator::deallocateSmallLine):
1263         (bmalloc::Deallocator::deallocateMediumLine):
1264         * bmalloc/Deallocator.h:
1265         (bmalloc::Deallocator::deallocateFastCase):
1266         * bmalloc/Heap.h:
1267         (bmalloc::Heap::deallocateSmallLine):
1268         (bmalloc::Heap::deallocateMediumLine):
1269         * bmalloc/Line.h:
1270         (bmalloc::Line<Traits>::deref):
1271         * bmalloc/Page.h:
1272         (bmalloc::Page<Traits>::deref):
1273
1274 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1275
1276         bmalloc
1277         https://bugs.webkit.org/show_bug.cgi?id=131170
1278
1279         Reviewed by Andreas Kling.
1280
1281         Initial commit.
1282
1283         * bmalloc: Added.
1284         * bmalloc.xcodeproj: Added.
1285         * bmalloc.xcodeproj/project.pbxproj: Added.
1286         * bmalloc/Algorithm.h: Added.
1287         (bmalloc::max):
1288         (bmalloc::min):
1289         (bmalloc::mask):
1290         (bmalloc::test):
1291         (bmalloc::roundUpToMultipleOf):
1292         (bmalloc::roundDownToMultipleOf):
1293         (bmalloc::sizeOf):
1294         (bmalloc::bitCount):
1295         (bmalloc::isPowerOfTwo):
1296         * bmalloc/Allocator.cpp: Added.
1297         (bmalloc::Allocator::Allocator):
1298         (bmalloc::Allocator::~Allocator):
1299         (bmalloc::Allocator::log):
1300         (bmalloc::Allocator::processSmallAllocatorLog):
1301         (bmalloc::Allocator::processMediumAllocatorLog):
1302         (bmalloc::Allocator::allocateLarge):
1303         (bmalloc::Allocator::allocateXLarge):
1304         (bmalloc::Allocator::allocateMedium):
1305         (bmalloc::Allocator::allocateSlowCase):
1306         * bmalloc/Allocator.h: Added.
1307         (bmalloc::Allocator::smallAllocatorFor):
1308         (bmalloc::Allocator::allocateFastCase):
1309         (bmalloc::Allocator::allocate):
1310         * bmalloc/AsyncTask.cpp: Added.
1311         (bmalloc::AsyncTask<Function>::runSlowCase):
1312         (bmalloc::AsyncTask<Function>::pthreadEntryPoint):
1313         (bmalloc::AsyncTask<Function>::entryPoint):
1314         * bmalloc/AsyncTask.h: Added.
1315         (bmalloc::Function>::AsyncTask):
1316         (bmalloc::Function>::join):
1317         (bmalloc::Function>::run):
1318         (bmalloc::Function>::runSlowCase):
1319         (bmalloc::Function>::pthreadEntryPoint):
1320         (bmalloc::Function>::entryPoint):
1321         * bmalloc/BAssert.h: Added.
1322         * bmalloc/BeginTag.h: Added.
1323         (bmalloc::BeginTag::isInFreeList):
1324         * bmalloc/BoundaryTag.h: Added.
1325         (bmalloc::BoundaryTag::isXLarge):
1326         (bmalloc::BoundaryTag::setXLarge):
1327         (bmalloc::BoundaryTag::isFree):
1328         (bmalloc::BoundaryTag::setFree):
1329         (bmalloc::BoundaryTag::isEnd):
1330         (bmalloc::BoundaryTag::setEnd):
1331         (bmalloc::BoundaryTag::hasPhysicalPages):
1332         (bmalloc::BoundaryTag::setHasPhysicalPages):
1333         (bmalloc::BoundaryTag::isNull):
1334         (bmalloc::BoundaryTag::clear):
1335         (bmalloc::BoundaryTag::size):
1336         (bmalloc::BoundaryTag::setSize):
1337         (bmalloc::BoundaryTag::prev):
1338         (bmalloc::BoundaryTag::next):
1339         * bmalloc/BoundaryTagInlines.h: Added.
1340         (bmalloc::validate):
1341         (bmalloc::validatePrev):
1342         (bmalloc::validateNext):
1343         (bmalloc::BoundaryTag::init):
1344         (bmalloc::BoundaryTag::mergeLargeLeft):
1345         (bmalloc::BoundaryTag::mergeLargeRight):
1346         (bmalloc::BoundaryTag::mergeLarge):
1347         (bmalloc::BoundaryTag::deallocate):
1348         (bmalloc::BoundaryTag::splitLarge):
1349         (bmalloc::BoundaryTag::allocate):
1350         * bmalloc/Cache.cpp: Added.
1351         (bmalloc::Cache::operator new):
1352         (bmalloc::Cache::operator delete):
1353         (bmalloc::Cache::Cache):
1354         (bmalloc::Cache::allocateSlowCase):
1355         (bmalloc::Cache::allocateSlowCaseNullCache):
1356         (bmalloc::Cache::deallocateSlowCase):
1357         (bmalloc::Cache::deallocateSlowCaseNullCache):
1358         * bmalloc/Cache.h: Added.
1359         (bmalloc::Cache::allocator):
1360         (bmalloc::Cache::deallocator):
1361         (bmalloc::Cache::allocateFastCase):
1362         (bmalloc::Cache::deallocateFastCase):
1363         (bmalloc::Cache::allocate):
1364         (bmalloc::Cache::deallocate):
1365         * bmalloc/Chunk.h: Added.
1366         (bmalloc::Chunk::begin):
1367         (bmalloc::Chunk::end):
1368         (bmalloc::Chunk::lines):
1369         (bmalloc::Chunk::pages):
1370         * bmalloc/Deallocator.cpp: Added.
1371         (bmalloc::Deallocator::Deallocator):
1372         (bmalloc::Deallocator::~Deallocator):
1373         (bmalloc::Deallocator::deallocateLarge):
1374         (bmalloc::Deallocator::deallocateXLarge):
1375         (bmalloc::Deallocator::processObjectLog):
1376         (bmalloc::Deallocator::deallocateSlowCase):
1377         (bmalloc::Deallocator::deallocateSmallLine):
1378         (bmalloc::Deallocator::allocateSmallLine):
1379         (bmalloc::Deallocator::deallocateMediumLine):
1380         (bmalloc::Deallocator::allocateMediumLine):
1381         * bmalloc/Deallocator.h: Added.
1382         (bmalloc::Deallocator::deallocateFastCase):
1383         (bmalloc::Deallocator::deallocate):
1384         * bmalloc/EndTag.h: Added.
1385         (bmalloc::EndTag::operator=):
1386         * bmalloc/FixedVector.h: Added.
1387         (bmalloc::FixedVector::begin):
1388         (bmalloc::FixedVector::end):
1389         (bmalloc::FixedVector::size):
1390         (bmalloc::FixedVector::capacity):
1391         (bmalloc::FixedVector::clear):
1392         (bmalloc::FixedVector::isEmpty):
1393         (bmalloc::Capacity>::FixedVector):
1394         (bmalloc::Capacity>::operator):
1395         (bmalloc::Capacity>::push):
1396         (bmalloc::Capacity>::pop):
1397         (bmalloc::Capacity>::shrink):
1398         * bmalloc/Heap.cpp: Added.
1399         (bmalloc::sleep):
1400         (bmalloc::Heap::Heap):
1401         (bmalloc::Heap::concurrentScavenge):
1402         (bmalloc::Heap::scavengeSmallPages):
1403         (bmalloc::Heap::scavengeMediumPages):
1404         (bmalloc::Heap::scavengeLargeRanges):
1405         (bmalloc::Heap::allocateSmallLineSlowCase):
1406         (bmalloc::Heap::allocateMediumLineSlowCase):
1407         (bmalloc::Heap::allocateXLarge):
1408         (bmalloc::Heap::deallocateXLarge):
1409         (bmalloc::Heap::allocateLarge):
1410         (bmalloc::Heap::deallocateLarge):
1411         * bmalloc/Heap.h: Added.
1412         (bmalloc::Heap::deallocateSmallLine):
1413         (bmalloc::Heap::allocateSmallLine):
1414         (bmalloc::Heap::deallocateMediumLine):
1415         (bmalloc::Heap::allocateMediumLine):
1416         * bmalloc/Inline.h: Added.
1417         * bmalloc/LargeChunk.h: Added.
1418         (bmalloc::LargeChunk::begin):
1419         (bmalloc::LargeChunk::end):
1420         (bmalloc::LargeChunk::create):
1421         (bmalloc::LargeChunk::get):
1422         (bmalloc::LargeChunk::beginTag):
1423         (bmalloc::LargeChunk::endTag):
1424         * bmalloc/Line.h: Added.
1425         (bmalloc::Line<Traits>::begin):
1426         (bmalloc::Line<Traits>::end):
1427         (bmalloc::Line<Traits>::concurrentRef):
1428         (bmalloc::Line<Traits>::deref):
1429         * bmalloc/MediumAllocator.h: Added.
1430         (bmalloc::MediumAllocator::isNull):
1431         (bmalloc::MediumAllocator::MediumAllocator):
1432         (bmalloc::MediumAllocator::line):
1433         (bmalloc::MediumAllocator::allocate):
1434         (bmalloc::MediumAllocator::derefCount):
1435         (bmalloc::MediumAllocator::refill):
1436         * bmalloc/MediumChunk.h: Added.
1437         * bmalloc/MediumLine.h: Added.
1438         * bmalloc/MediumPage.h: Added.
1439         * bmalloc/MediumTraits.h: Added.
1440         * bmalloc/Mutex.cpp: Added.
1441         (bmalloc::Mutex::lockSlowCase):
1442         * bmalloc/Mutex.h: Added.
1443         (bmalloc::Mutex::Mutex):
1444         (bmalloc::Mutex::try_lock):
1445         (bmalloc::Mutex::lock):
1446         (bmalloc::Mutex::unlock):
1447         * bmalloc/ObjectType.cpp: Added.
1448         (bmalloc::objectType):
1449         * bmalloc/ObjectType.h: Added.
1450         (bmalloc::isSmallOrMedium):
1451         (bmalloc::isSmall):
1452         * bmalloc/Page.h: Added.
1453         (bmalloc::Page<Traits>::ref):
1454         (bmalloc::Page<Traits>::deref):
1455         (bmalloc::Page<Traits>::refCount):
1456         * bmalloc/PerProcess.h: Added.
1457         (bmalloc::PerProcess::mutex):
1458         (bmalloc::PerProcess<T>::getFastCase):
1459         (bmalloc::PerProcess<T>::get):
1460         (bmalloc::PerProcess<T>::getSlowCase):
1461         * bmalloc/PerThread.h: Added.
1462         (bmalloc::PerThreadStorage<Cache>::get):
1463         (bmalloc::PerThreadStorage<Cache>::init):
1464         (bmalloc::PerThreadStorage::get):
1465         (bmalloc::PerThreadStorage::init):
1466         (bmalloc::PerThread<T>::getFastCase):
1467         (bmalloc::PerThread<T>::get):
1468         (bmalloc::PerThread<T>::destructor):
1469         (bmalloc::PerThread<T>::getSlowCase):
1470         * bmalloc/Range.h: Added.
1471         (bmalloc::Range::Range):
1472         (bmalloc::Range::begin):
1473         (bmalloc::Range::end):
1474         (bmalloc::Range::size):
1475         (bmalloc::Range::operator!):
1476         (bmalloc::Range::operator<):
1477         * bmalloc/SegregatedFreeList.cpp: Added.
1478         (bmalloc::SegregatedFreeList::SegregatedFreeList):
1479         (bmalloc::SegregatedFreeList::insert):
1480         (bmalloc::SegregatedFreeList::takeGreedy):
1481         (bmalloc::SegregatedFreeList::take):
1482         * bmalloc/SegregatedFreeList.h: Added.
1483         * bmalloc/Sizes.h: Added.
1484         * bmalloc/SmallAllocator.h: Added.
1485         (bmalloc::SmallAllocator::isNull):
1486         (bmalloc::SmallAllocator::canAllocate):
1487         (bmalloc::SmallAllocator::SmallAllocator):
1488         (bmalloc::SmallAllocator::line):
1489         (bmalloc::SmallAllocator::allocate):
1490         (bmalloc::SmallAllocator::objectCount):
1491         (bmalloc::SmallAllocator::derefCount):
1492         (bmalloc::SmallAllocator::refill):
1493         * bmalloc/SmallChunk.h: Added.
1494         * bmalloc/SmallLine.h: Added.
1495         * bmalloc/SmallPage.h: Added.
1496         * bmalloc/SmallTraits.h: Added.
1497         * bmalloc/Syscall.h: Added.
1498         * bmalloc/VMAllocate.h: Added.
1499         (bmalloc::vmSize):
1500         (bmalloc::vmValidate):
1501         (bmalloc::vmAllocate):
1502         (bmalloc::vmDeallocate):
1503         (bmalloc::vmDeallocatePhysicalPages):
1504         (bmalloc::vmAllocatePhysicalPages):
1505         (bmalloc::vmDeallocatePhysicalPagesSloppy):
1506         (bmalloc::vmAllocatePhysicalPagesSloppy):
1507         * bmalloc/VMHeap.cpp: Added.
1508         (bmalloc::VMHeap::VMHeap):
1509         (bmalloc::VMHeap::allocateSmallChunk):
1510         (bmalloc::VMHeap::allocateMediumChunk):
1511         (bmalloc::VMHeap::allocateLargeChunk):
1512         * bmalloc/VMHeap.h: Added.
1513         (bmalloc::VMHeap::allocateSmallPage):
1514         (bmalloc::VMHeap::allocateMediumPage):
1515         (bmalloc::VMHeap::allocateLargeRange):
1516         (bmalloc::VMHeap::deallocateSmallPage):
1517         (bmalloc::VMHeap::deallocateMediumPage):
1518         (bmalloc::VMHeap::deallocateLargeRange):
1519         * bmalloc/Vector.h: Added.
1520         (bmalloc::Vector::begin):
1521         (bmalloc::Vector::end):
1522         (bmalloc::Vector::size):
1523         (bmalloc::Vector::capacity):
1524         (bmalloc::Vector::last):
1525         (bmalloc::Vector::pop):
1526         (bmalloc::Vector<T>::Vector):
1527         (bmalloc::Vector<T>::~Vector):
1528         (bmalloc::Vector<T>::operator):
1529         (bmalloc::Vector<T>::push):
1530         (bmalloc::Vector<T>::pop):
1531         (bmalloc::Vector<T>::shrink):
1532         (bmalloc::Vector<T>::reallocateBuffer):
1533         (bmalloc::Vector<T>::shrinkCapacity):
1534         (bmalloc::Vector<T>::growCapacity):
1535         * bmalloc/XLargeChunk.h: Added.
1536         (bmalloc::XLargeChunk::get):
1537         (bmalloc::XLargeChunk::begin):
1538         (bmalloc::XLargeChunk::XLargeChunk):
1539         (bmalloc::XLargeChunk::create):
1540         (bmalloc::XLargeChunk::destroy):
1541         (bmalloc::XLargeChunk::range):
1542         (bmalloc::XLargeChunk::size):
1543         * bmalloc/bmalloc.h: Added.
1544         (bmalloc::api::malloc):
1545         (bmalloc::api::free):
1546         (bmalloc::api::realloc):
1547         * bmalloc/mbmalloc.cpp: Added.
1548