570cf87e61d3d5a140507accc3b3f13d36bd1609
[WebKit-https.git] / Source / bmalloc / ChangeLog
1 2014-10-30  Dana Burkart  <dburkart@apple.com>
2
3         <rdar://problem/18821260> Prepare for the mysterious future
4
5         Reviewed by Lucas Forschler.
6
7         * Configurations/Base.xcconfig:
8         * Configurations/DebugRelease.xcconfig:
9
10 2014-09-24  Geoffrey Garen  <ggaren@apple.com>
11
12         bmalloc: cleaned up fast path vs slow path
13         https://bugs.webkit.org/show_bug.cgi?id=137081
14
15         Reviewed by Sam Weinig.
16
17         Might be a 1% speedup on MallocBench. Also cleans up the code a bit.
18
19         * bmalloc/Allocator.cpp:
20         (bmalloc::Allocator::Allocator): Merged the small and medium range
21         caches, just like the small and medium allocators. Ranges are abstract
22         objects that don't really care whether they hold small or medium objects,
23         so they don't need to be segregated.
24
25         (bmalloc::Allocator::scavenge): Ditto.
26
27         (bmalloc::Allocator::allocateBumpRangeSlowCase):
28         (bmalloc::Allocator::allocateBumpRange): Same thing here, except that
29         we do care a tiny bit, because we need to specify small vs medium when
30         allocating new ranges from the heap, to ensure that the heap allocates
31         from the right segment of VM.
32
33         (bmalloc::Allocator::allocateLarge):
34         (bmalloc::Allocator::allocateXLarge): NO_INLINE because this was clouding
35         up the fast path. Large allocation performance is dominated by allocation
36         logic and initialization, so inlining it doesn't help.
37
38         (bmalloc::Allocator::allocateSlowCase): Slow path got a bit cleaner since
39         it doesn't need to distinguish small vs medium objects.
40
41         (bmalloc::Allocator::allocateSmallBumpRange): Deleted.
42         (bmalloc::Allocator::allocateMediumBumpRange): Deleted.
43
44         * bmalloc/Allocator.h:
45         * bmalloc/BumpRange.h:
46
47         * bmalloc/Cache.cpp:
48         (bmalloc::Cache::allocateSlowCase): Deleted.
49         (bmalloc::Cache::deallocateSlowCase): Deleted.
50         * bmalloc/Cache.h:
51         (bmalloc::Cache::allocate):
52         (bmalloc::Cache::deallocate):
53         (bmalloc::Cache::allocateFastCase): Deleted.
54         (bmalloc::Cache::deallocateFastCase): Deleted. Removed the Cache slow
55         paths. The downside to this change is that the fast path branches to two
56         distinct failure cases instead of one. The upside is that the slow path
57         doesn't need to re-read the segment register, which is not as cheap as a
58         normal register, and it doesn't need to do an extra level of function 
59         call. Seems to be worth it.
60
61         * bmalloc/Deallocator.h:
62         * bmalloc/Heap.cpp:
63         (bmalloc::Heap::refillSmallBumpRangeCache):
64         (bmalloc::Heap::refillMediumBumpRangeCache):
65         * bmalloc/Heap.h: Updated for interface changes.
66
67         * bmalloc/Sizes.h: The most ranges a cache will hold is the number of
68         small lines in a page / 2, since any other free lines will coalesce
69         with their neighbors.
70
71 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
72
73         Rolled out r173346.
74
75             bmalloc should honor the FastMalloc statistics API
76             https://bugs.webkit.org/show_bug.cgi?id=136592
77
78         This didn't really work. Because we allow ranges with and without
79         physical pages to merge, and we allow double-committing and
80         double-decommitting, we can't rely on commit actions to track memory
81         footprint.
82
83         * bmalloc/Heap.cpp:
84         (bmalloc::Heap::size): Deleted.
85         (bmalloc::Heap::capacity): Deleted.
86         * bmalloc/Heap.h:
87         * bmalloc/VMHeap.cpp:
88         (bmalloc::VMHeap::VMHeap):
89         (bmalloc::VMHeap::allocateSmallChunk):
90         (bmalloc::VMHeap::allocateMediumChunk):
91         (bmalloc::VMHeap::allocateLargeChunk):
92         * bmalloc/VMHeap.h:
93         (bmalloc::VMHeap::allocateSmallPage):
94         (bmalloc::VMHeap::allocateMediumPage):
95         (bmalloc::VMHeap::allocateLargeRange):
96         (bmalloc::VMHeap::deallocateSmallPage):
97         (bmalloc::VMHeap::deallocateMediumPage):
98         (bmalloc::VMHeap::deallocateLargeRange):
99         (bmalloc::VMHeap::size): Deleted.
100         (bmalloc::VMHeap::capacity): Deleted.
101         * bmalloc/bmalloc.h:
102         (bmalloc::api::heapSize): Deleted.
103         (bmalloc::api::heapCapacity): Deleted.
104
105 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
106
107         bmalloc: Allocation should be more precise
108         https://bugs.webkit.org/show_bug.cgi?id=136993
109
110         Reviewed by Gavin Barraclough.
111
112         13% reduction in heap size on the MallocBench *_memory_warning benchmarks.
113
114         This patch teaches the allocator to merge adjacent free lines into a
115         single allocatable range. This allows us to shrink the size of an
116         individual line without increasing fragmentation or the rate of allocator
117         slow paths.
118
119         We'll only take more slow paths when available memory is sparse, which
120         is exactly when it's worth it. When available memory is dense, we'll
121         take fewer slow paths.
122
123         * bmalloc.xcodeproj/project.pbxproj:
124         * bmalloc/Algorithm.h:
125         (bmalloc::divideRoundingUp):
126
127         * bmalloc/Allocator.cpp:
128         (bmalloc::Allocator::Allocator): Updated for interface changes.
129
130         (bmalloc::Allocator::scavenge): Scavenge by object instead of by line.
131         Now that we merge lines, it's not convenient to scavenge by line.
132
133         (bmalloc::Allocator::allocateSmallBumpRange):
134         (bmalloc::Allocator::allocateMediumBumpRange): Allocate whole ranges
135         instead of individual lines.
136
137         (bmalloc::Allocator::allocateSlowCase):
138         (bmalloc::Allocator::allocateSmallLine): Deleted.
139         (bmalloc::Allocator::allocateMediumLine): Deleted.
140         (bmalloc::Allocator::allocateMedium): Deleted.
141         * bmalloc/Allocator.h:
142         (bmalloc::Allocator::allocateFastCase): Folded medium allocations
143         into the standard fast path with small allocations. Since a BumpAllocator
144         just allocates out of an arbitrary range, it doesn't need to distinguish
145         between small and medium lines.
146
147         * bmalloc/BumpAllocator.h:
148         (bmalloc::BumpAllocator::size):
149         (bmalloc::BumpAllocator::BumpAllocator):
150         (bmalloc::BumpAllocator::init):
151         (bmalloc::BumpAllocator::refill):
152         (bmalloc::BumpAllocator::line): Deleted. No need to track line information
153         anymore: the heap just gives us a pointer and a pre-computed number of
154         objects, and we allocate them.
155
156         * bmalloc/Deallocator.cpp:
157         (bmalloc::Deallocator::processObjectLog): Updated for interface changes.
158
159         * bmalloc/Heap.cpp:
160         (bmalloc::Heap::Heap):
161         (bmalloc::Heap::initializeLineMetadata): Pre-compute precise metadata
162         detailing where all objects will lie in memory. After we merge two lines,
163         we might allocate an object that spans from one line to the next. This
164         metadata details which bits of memory overlap in that way, and how they
165         overlap.
166
167         (bmalloc::Heap::refillSmallBumpRangeCache):
168         (bmalloc::Heap::refillMediumBumpRangeCache): Scan a whole page at a time,
169         and merge adjacent free lines into BumpRanges.
170
171         (bmalloc::Heap::allocateSmallPage):
172         (bmalloc::Heap::allocateMediumPage):
173         (bmalloc::Heap::deallocateSmallLine):
174         (bmalloc::Heap::deallocateMediumLine): Track pages rather than lines,
175         since we scan for free memory a page at a time.
176
177         (bmalloc::Heap::allocateSmallLineSlowCase): Deleted.
178         (bmalloc::Heap::allocateMediumLineSlowCase): Deleted. Folded into the
179         fast path.
180
181         * bmalloc/Heap.h:
182         (bmalloc::Heap::derefSmallLine):
183         (bmalloc::Heap::derefMediumLine):
184         (bmalloc::Heap::deallocateSmallLine): Deleted.
185         (bmalloc::Heap::allocateSmallLine): Deleted.
186         (bmalloc::Heap::deallocateMediumLine): Deleted.
187         (bmalloc::Heap::allocateMediumLine): Deleted. Updated for interface changes.
188
189         * bmalloc/Line.h:
190         (bmalloc::Line<Traits>::ref):
191         (bmalloc::Line<Traits>::deref):
192         (bmalloc::Line<Traits>::concurrentRef): Deleted. We don't pass a derefCount
193         anymore, since we only ever deref by 1 now.
194
195         * bmalloc/MediumAllocator.h:
196         (bmalloc::MediumAllocator::isNull): Deleted.
197         (bmalloc::MediumAllocator::MediumAllocator): Deleted.
198         (bmalloc::MediumAllocator::line): Deleted.
199         (bmalloc::MediumAllocator::allocate): Deleted.
200         (bmalloc::MediumAllocator::derefCount): Deleted.
201         (bmalloc::MediumAllocator::refill): Deleted.
202         (bmalloc::MediumAllocator::clear): Deleted. Deleted some code that's
203         been dead for a while, since it doesn't build anymore with this patch.
204
205         * bmalloc/Page.h:
206         (bmalloc::Page::sizeClass):
207         (bmalloc::Page::setSizeClass):
208         (bmalloc::Page::smallSizeClass): Deleted.
209         (bmalloc::Page::setSmallSizeClass): Deleted. Renamed setSmallSizeClass
210         to sizeClass, since we use it for medium sizes too.
211
212         * bmalloc/Sizes.h:
213         (bmalloc::Sizes::sizeClass):
214         (bmalloc::Sizes::objectSize): Shrank line sizes to save memory.
215
216         (bmalloc::Sizes::smallSizeClassFor): Deleted.
217         (bmalloc::Sizes::mediumSizeClassFor): Deleted.
218
219         * bmalloc/bmalloc.h:
220         (bmalloc::api::realloc): Now that we have precise objects sizes, realloc
221         can be a bit more precise. It also has to be, since we can't guarantee
222         that an object ends at the end of a line anymore.
223
224 2014-09-19  Daniel Bates  <dabates@apple.com>
225
226         Always assume internal SDK when building configuration Production
227         https://bugs.webkit.org/show_bug.cgi?id=136925
228         <rdar://problem/18362399>
229
230         Reviewed by Dan Bernstein.
231
232         * Configurations/Base.xcconfig:
233
234 2014-09-16  Geoffrey Garen  <ggaren@apple.com>
235
236         bmalloc: moved line caches from the deallocator to the allocator
237         https://bugs.webkit.org/show_bug.cgi?id=136868
238
239         Reviewed by Gavin Barraclough.
240
241         I did this mostly as a simplification, to make it easier to change the
242         allocation strategy.
243
244         No throughput change on MallocBench. Saves about 50kB.
245
246         Since the deallocator needs to lock the heap when freeing lines anyway,
247         there isn't much benefit to giving the deallocator a local cache of
248         deallocated lines.
249
250         We still give the allocator a local cache of lines because that does
251         reduce the frequency at which it needs to lock the heap in order to
252         acquire more lines.
253
254         * bmalloc/Allocator.cpp:
255         (bmalloc::Allocator::scavenge):
256         (bmalloc::Allocator::allocateSmallLine):
257         (bmalloc::Allocator::allocateMediumLine):
258         (bmalloc::Allocator::allocateMedium):
259         (bmalloc::Allocator::allocateSlowCase):
260         * bmalloc/Allocator.h:
261         * bmalloc/Deallocator.cpp:
262         (bmalloc::Deallocator::Deallocator):
263         (bmalloc::Deallocator::scavenge):
264         (bmalloc::Deallocator::processObjectLog):
265         (bmalloc::Deallocator::deallocateSmallLine): Deleted.
266         (bmalloc::Deallocator::allocateSmallLine): Deleted.
267         (bmalloc::Deallocator::deallocateMediumLine): Deleted.
268         (bmalloc::Deallocator::allocateMediumLine): Deleted.
269         * bmalloc/Deallocator.h:
270
271         * bmalloc/Sizes.h:
272         * bmalloc/VMAllocate.h: Took the opportunity to make the line cache size
273         exactly one page in size. That's about what we were shooting for anyway,
274         and it may make it easier to switch to per-page allocation in future.
275
276 2014-09-15  Geoffrey Garen  <ggaren@apple.com>
277
278         bmalloc: allocate small and medium objects using the same bump pointer class
279         https://bugs.webkit.org/show_bug.cgi?id=136843
280
281         Reviewed by Gavin Barraclough.
282
283         4% speedup on MallocBench.
284
285         Now that medium-sized objects have dedicated per-size allocators, they
286         don't need to use an arbitrary bump pointer allocator. This means that
287         every allocator knows how many objects it will allocate from the start,
288         and we don't need a post-processing step to adjust refcounts based on
289         real allocation count.
290
291         * bmalloc.xcodeproj/project.pbxproj: Renamed SmallAllocator to BumpAllocator
292         since it's used for small and medium objects now.
293
294         * bmalloc/Allocator.cpp:
295         (bmalloc::Allocator::Allocator): Updated to use new interface.
296         (bmalloc::Allocator::scavenge): To "retire" an allocator, we just need
297         to make sure that we finish allocating all the objects in it.
298
299         (bmalloc::Allocator::allocateMedium):
300         (bmalloc::Allocator::allocateSlowCase):
301         (bmalloc::Allocator::retire): Deleted.
302         (bmalloc::Allocator::processSmallAllocatorLog): Deleted.
303         (bmalloc::Allocator::processMediumAllocatorLog): Deleted.
304         * bmalloc/Allocator.h:
305         (bmalloc::Allocator::allocateFastCase): Removed abstractions and data
306         used to post-process an allocator based on how many objects it allocated.
307
308         * bmalloc/BumpAllocator.h: Copied from Source/bmalloc/bmalloc/SmallAllocator.h.
309         (bmalloc::BumpAllocator::BumpAllocator):
310         (bmalloc::BumpAllocator::init):
311         (bmalloc::BumpAllocator::line):
312         (bmalloc::BumpAllocator::validate):
313         (bmalloc::BumpAllocator::allocate):
314         (bmalloc::BumpAllocator::refill):
315         (bmalloc::BumpAllocator::clear): Updated these functions to be agnostic
316         about the kinds of lines they allocate into. In some cases, the line
317         type must be provided as a template parameter by the caller.
318
319         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
320         (bmalloc::SmallAllocator::line): Deleted.
321         (bmalloc::SmallAllocator::allocate): Deleted.
322         (bmalloc::SmallAllocator::objectCount): Deleted.
323         (bmalloc::SmallAllocator::derefCount): Deleted.
324         (bmalloc::SmallAllocator::refill): Deleted.
325         (bmalloc::SmallAllocator::clear): Deleted.
326
327         * bmalloc/ObjectType.h:
328         (bmalloc::isMedium):
329
330         * bmalloc/SmallAllocator.h:
331         (bmalloc::SmallAllocator::isNull): Deleted.
332         (bmalloc::SmallAllocator::canAllocate): Deleted.
333         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
334         (bmalloc::SmallAllocator::line): Deleted.
335         (bmalloc::SmallAllocator::allocate): Deleted.
336         (bmalloc::SmallAllocator::objectCount): Deleted.
337         (bmalloc::SmallAllocator::derefCount): Deleted.
338         (bmalloc::SmallAllocator::refill): Deleted.
339         (bmalloc::SmallAllocator::clear): Deleted.
340
341 2014-09-12  Geoffrey Garen  <ggaren@apple.com>
342
343         Fixed a goof in bmalloc Vector sizing
344         https://bugs.webkit.org/show_bug.cgi?id=136795
345
346         Reviewed by Gavin Barraclough and Sam Weinig.
347
348         We want our minimum vector to be page-sized since the OS will give us
349         a page no matter what -- but we want that many bytes, and not enough
350         bytes to store that many elements.
351
352         * bmalloc/Vector.h: Math is hard.
353
354 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
355
356         bmalloc should segregate medium-sized objects by line like it does for small-sized objects
357         https://bugs.webkit.org/show_bug.cgi?id=136693
358
359         Reviewed by Gavin Barraclough.
360
361         4% reduction in heap size on the MallocBench *_memory_warning benchmarks.
362
363         No throughput change.
364
365         We keep an array of medium allocators, just like our array of small
366         allocators.
367
368         In future, we can simplify the allocation fast path by merging the small
369         and medium allocator arrays. For now, this is the simplest change that
370         gets the win.
371
372         * bmalloc/Allocator.cpp:
373         (bmalloc::Allocator::Allocator):
374         (bmalloc::Allocator::scavenge):
375         (bmalloc::Allocator::allocateMedium):
376         * bmalloc/Allocator.h:
377         * bmalloc/Sizes.h:
378         (bmalloc::Sizes::mediumSizeClassFor):
379
380 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
381
382         Reviewed by Sam Weinig.
383
384         Renamed log => retire for clarity.
385
386         * bmalloc/Allocator.cpp:
387         (bmalloc::Allocator::scavenge):
388         (bmalloc::Allocator::retire):
389         (bmalloc::Allocator::allocateMedium):
390         (bmalloc::Allocator::allocateSlowCase):
391         (bmalloc::Allocator::log): Deleted.
392         * bmalloc/Allocator.h:
393
394 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
395
396         bmalloc: eager scavenge leaves behind a bogus allocator
397         https://bugs.webkit.org/show_bug.cgi?id=136743
398
399         Reviewed by Sam Weinig.
400
401         Be sure to clear the allocator after logging it in the eager scavenge
402         case, so that we don't later try to allocate out of the lines that we
403         have thrown away.
404
405         We didn't need to do this previously because scavenge would only happen
406         at thread exit time, after which no further allocation from the per-thread
407         cache would take place.
408
409         * bmalloc/Allocator.cpp:
410         (bmalloc::Allocator::scavenge):
411         * bmalloc/MediumAllocator.h:
412         (bmalloc::MediumAllocator::clear):
413         * bmalloc/SmallAllocator.h:
414         (bmalloc::SmallAllocator::clear):
415
416 2014-09-05  Geoffrey Garen  <ggaren@apple.com>
417
418         bmalloc should honor the FastMalloc statistics API
419         https://bugs.webkit.org/show_bug.cgi?id=136592
420
421         Reviewed by Gavin Barraclough.
422
423         We do this by tracking "size" and "capacity" in the VM heap.
424
425         The VM heap's "capacity" is all the VM we ever allocated.
426
427         The VM heap's "size" the subset of VM currently held onto by the
428         VM heap (and therefore not in use by the regular heap).
429
430         Somewhat ironically, reducing the process's memory footprint, increases
431         the size of the VM heap, since the VM heap holds the pages that are
432         purely virtual and not physical.
433
434         * bmalloc/Heap.cpp:
435         (bmalloc::Heap::size):
436         (bmalloc::Heap::capacity):
437         * bmalloc/Heap.h:
438         * bmalloc/VMHeap.cpp:
439         (bmalloc::VMHeap::VMHeap):
440         (bmalloc::VMHeap::allocateSmallChunk):
441         (bmalloc::VMHeap::allocateMediumChunk):
442         (bmalloc::VMHeap::allocateLargeChunk):
443         * bmalloc/VMHeap.h:
444         (bmalloc::VMHeap::size):
445         (bmalloc::VMHeap::capacity):
446         (bmalloc::VMHeap::allocateSmallPage):
447         (bmalloc::VMHeap::allocateMediumPage):
448         (bmalloc::VMHeap::allocateLargeRange):
449         (bmalloc::VMHeap::deallocateSmallPage):
450         (bmalloc::VMHeap::deallocateMediumPage):
451         (bmalloc::VMHeap::deallocateLargeRange):
452         * bmalloc/bmalloc.h:
453         (bmalloc::api::heapSize):
454         (bmalloc::api::heapCapacity):
455
456 2014-09-02  Geoffrey Garen  <ggaren@apple.com>
457
458         bmalloc crashes on the EWS bots (due to bad large object allocation)
459         https://bugs.webkit.org/show_bug.cgi?id=136469
460
461         Reviewed by Andreas Kling.
462
463         It's possible to convince bmalloc to perform a bad large object allocation,
464         through these steps:
465
466         (1) Insert object A into freelist F0.
467
468         (2) Split, merge and split again A's neighbors such that object B is
469         inserted into freelist F0, with boundary tag and size equal to object A,
470         but pointer not completely equal to object A. Put object B at the head of F0.
471
472         (3) Allocate some other object from F0, swapping its position in the
473         freelist with object B, such that object A is now ahead of object B.
474
475         --> Now, the next allocation for size A/B will allocate object A, which
476         has a slightly wrong idea about where the object actually begins.
477         Immediately, you'll corrupt a little memory, and over time, you'll also
478         corrupt boundary tag metadata.
479
480         The solution is to store the begin pointer in the boundary tag. Luckily,
481         this doesn't make the tag any bigger, and it's not a noticeable slowdown
482         on MallocBench.
483
484         * bmalloc/Algorithm.h:
485         (bmalloc::rightShift):
486         * bmalloc/BeginTag.h:
487         (bmalloc::BeginTag::isInFreeList): This is the bug fix. Make sure to
488         validate the start pointer when popping off the free list. Through a
489         very uncommon set of steps, it is possible to have an item in the free
490         list that is valid by all accounts except for its start pointer.
491
492         * bmalloc/BoundaryTag.h:
493         (bmalloc::BoundaryTag::compactBegin):
494         (bmalloc::BoundaryTag::setRange):
495         (bmalloc::BoundaryTag::setSize): Deleted. Record a compact version of the
496         start pointer. We don't need the whole pointer -- just the offset, in
497         largeAlignment increments, into the relevant boundary tag bucket.
498
499         * bmalloc/BoundaryTagInlines.h:
500         (bmalloc::validateNext):
501         (bmalloc::BoundaryTag::init):
502         (bmalloc::BoundaryTag::mergeLarge):
503         (bmalloc::BoundaryTag::splitLarge):
504         * bmalloc/SegregatedFreeList.cpp:
505         (bmalloc::SegregatedFreeList::insert):
506         (bmalloc::SegregatedFreeList::takeGreedy):
507         (bmalloc::SegregatedFreeList::take): Provide the whole range instead of
508         the size when establishing a boundary tag, as required by the new
509         interface.
510
511         * bmalloc/Sizes.h:
512
513 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
514
515         Fixed a bmalloc crash seen on the EWS bot
516         https://bugs.webkit.org/show_bug.cgi?id=135955
517
518         Reviewed by Andreas Kling.
519
520         * bmalloc/Syscall.h: Some CG APIs vm_copy their input buffers. If the
521         input buffer is a malloc region, that region will get marked Copy-On-Write
522         by the kernel. Calls to madvise() for COW regions fail and return EINVAL
523         on older OS X's. In 10.10, they still fail, but they do not return
524         EINVAL.
525
526         So, we can only ASSERT that our syscalls succeed starting with 10.10.
527
528 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
529
530         Fixed the bmalloc build
531         https://bugs.webkit.org/show_bug.cgi?id=135953
532
533         Reviewed by Andreas Kling.
534
535         * bmalloc.xcodeproj/project.pbxproj: Marked a few headers as private.
536         These headers are used, so they must be available outside the project.
537
538 2014-08-13  Daniel Bates  <dabates@apple.com>
539
540         Attempt to fix the build following <http://trac.webkit.org/changeset/172576>
541         (https://bugs.webkit.org/show_bug.cgi?id=135895)
542
543         Substitute PerThreadStorage<T>::initSharedKeyIfNeeded() for initSharedKeyIfNeeded() in
544         implementation of PerThread<T>::getFastCase().
545
546         * bmalloc/PerThread.h:
547         (bmalloc::PerThread<T>::getFastCase):
548
549 2014-08-13  Daniel Bates  <dabates@apple.com>
550
551         Make bmalloc::PerThread work without C++ thread local storage
552         https://bugs.webkit.org/show_bug.cgi?id=135895
553
554         Reviewed by Geoffrey Garen.
555
556         Implement support for building bmalloc without C++ thread local storage.
557
558         * bmalloc/BPlatform.h: Remove macro define BPLATFORM_IOS_SIMULATOR. Added macro function
559         BCOMPILER_SUPPORTS() and macro define BCOMPILER_SUPPORTS_CXX_THREAD_LOCAL that can be used
560         to determine whether the compiler supports C++ thread local storage.
561         * bmalloc/PerThread.h:
562         (bmalloc::PerThreadStorage::get): Modified to call pthread_getspecific() when building
563         without C++ thread local storage.
564         (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Added.
565         (bmalloc::PerThreadStorage::init): Moved logic to initialize shared Pthread key from here to
566         PerThreadStorage::initSharedKeyIfNeeded().
567         (bmalloc::PerThread<T>::getFastCase): Modified to call PerThreadStorage::initSharedKeyIfNeeded()
568         before querying PerThreadStorage::get() when building without C++ thread local storage so as to
569         ensure that the shared key has been initialized.
570         (_pthread_setspecific_direct): Deleted.
571         (_pthread_getspecific_direct): Deleted.
572
573 2014-08-13  Daniel Bates  <dabates@apple.com>
574
575         [iOS] Make JavaScriptCore and bmalloc build with the public SDK
576         https://bugs.webkit.org/show_bug.cgi?id=135848
577
578         Reviewed by Geoffrey Garen.
579
580         * bmalloc/BPlatform.h: Added macro BPLATFORM_IOS_SIMULATOR, which evaluates to true
581         when building for the iOS Simulator.
582         * bmalloc/PerThread.h: Use pthread_machdep.h code path when building for iOS Simulator
583         using the public SDK.
584         (_pthread_setspecific_direct): Added; only defined when building for the iOS Simulator
585         using the public SDK.
586         (_pthread_getspecific_direct): Added; only defined when building for the iOS Simulator
587         using the public SDK.
588
589 2014-08-12  Daniel Bates  <dabates@apple.com>
590
591         BPLATFORM(IOS) always evaluates to false
592         https://bugs.webkit.org/show_bug.cgi?id=135843
593
594         Reviewed by Geoffrey Garen.
595
596         Fix typo in definition of BPLATFORM() and include system header TargetConditionals.h
597         (when building on an Apple platform) so that BPLATFORM(X) evaluates to true when
598         building for platform X. In particular, so that BPLATFORM(IOS) evaluates to true when
599         building for iOS.
600
601         As a side effect of this change, the change made in <http://trac.webkit.org/changeset/167289>
602         will be honored and iOS will assume a VM page size of 16kB (again) instead of 4kB.
603
604         * bmalloc/BPlatform.h:
605
606 2014-08-11  Andy Estes  <aestes@apple.com>
607
608         [iOS] Get rid of iOS.xcconfig
609         https://bugs.webkit.org/show_bug.cgi?id=135809
610
611         Reviewed by Joseph Pecoraro.
612
613         All iOS.xcconfig did was include AspenFamily.xcconfig, so there's no need for the indirection.
614
615         * Configurations/Base.xcconfig:
616         * Configurations/iOS.xcconfig: Removed.
617         * bmalloc.xcodeproj/project.pbxproj:
618
619 2014-05-01  Dan Bernstein  <mitz@apple.com>
620
621         Fixed production builds for the iOS Simulator.
622         <rdar://problem/16792221>
623
624         * Configurations/bmalloc.xcconfig: Include INSTALL_PATH_PREFIX in
625         PRIVATE_HEADERS_FOLDER_PATH when installing.
626
627 2014-04-20  Geoffrey Garen  <ggaren@apple.com>
628
629         bmalloc: Segregate pages by objects size
630         https://bugs.webkit.org/show_bug.cgi?id=131909
631
632         Reviewed by Andreas Kling.
633
634         2% reduction in memory-at-end on the Membuster memory_warning benchmarks.
635
636         * bmalloc/Allocator.cpp:
637         (bmalloc::Allocator::allocateSlowCase):
638         * bmalloc/Allocator.h:
639         (bmalloc::Allocator::allocateFastCase):
640         (bmalloc::Allocator::smallAllocatorFor): Use the new shared helper
641         function for size class calculation.
642
643         * bmalloc/Deallocator.cpp:
644         (bmalloc::Deallocator::Deallocator):
645         (bmalloc::Deallocator::scavenge):
646         (bmalloc::Deallocator::deallocateSmallLine):
647         (bmalloc::Deallocator::allocateSmallLine):
648         * bmalloc/Deallocator.h: Keep a cache for every size class, since the
649         cache can't be shared anymore.
650
651         * bmalloc/Heap.cpp:
652         (bmalloc::Heap::allocateSmallLineSlowCase):
653         * bmalloc/Heap.h:
654         (bmalloc::Heap::deallocateSmallLine): Ditto.
655
656         (bmalloc::Heap::allocateSmallLine): Check size class in addition to
657         page refcount when allocating a line because we might have deallocated
658         the page and the recycled it for another size class.
659
660         (bmalloc::Heap::deallocateMediumLine):
661         (bmalloc::Heap::allocateMediumLine):
662         * bmalloc/Line.h:
663         (bmalloc::Line::refCount):
664         * bmalloc/Page.h:
665         (bmalloc::Page::refCount):
666         (bmalloc::Page::smallSizeClass):
667         (bmalloc::Page::setSmallSizeClass):
668         (bmalloc::Page<Traits>::refCount): Deleted.
669         * bmalloc/Sizes.h:
670         (bmalloc::Sizes::smallSizeClassFor): New shared API for computing
671         an index into an array from a size.
672
673 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
674
675         bmalloc: Improved alignment in LargeChunk
676         https://bugs.webkit.org/show_bug.cgi?id=131895
677
678         Reviewed by Andreas Kling.
679
680         * bmalloc/Chunk.h:
681         * bmalloc/LargeChunk.h: Align to vmPageSize just like Chunk does.
682         Technically, the previous alignment was harmless, but I would prefer,
683         dear reader, not to have to explain the interlocking set of
684         circumstances that made it so.
685
686 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
687
688         Rolled out r167502 because it caused a crash on the facebook benchmark.
689
690         Unreviewed.
691
692             bmalloc: Added an XSmall line size
693             https://bugs.webkit.org/show_bug.cgi?id=131851
694
695             Reviewed by Sam Weinig.
696
697 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
698
699         bmalloc: Mutex should be harder to use wrong
700         https://bugs.webkit.org/show_bug.cgi?id=131879
701
702         Reviewed by Andreas Kling.
703
704         Mutex now has a proper constructor, so you can't deadlock by forgetting
705         to initialize it.
706
707         * bmalloc.xcodeproj/project.pbxproj:
708         * bmalloc/Allocator.cpp:
709         (bmalloc::Allocator::processXSmallAllocatorLog):
710         (bmalloc::Allocator::processSmallAllocatorLog):
711         (bmalloc::Allocator::processMediumAllocatorLog):
712         (bmalloc::Allocator::allocateLarge):
713         (bmalloc::Allocator::allocateXLarge): Global replace Mutex => StaticMutex,
714         since the Heap mutex is a static.
715
716         * bmalloc/AsyncTask.h:
717         (bmalloc::Function>::AsyncTask): Use Mutex, since we're not static. No
718         need for explicit initialization anymore.
719
720         * bmalloc/Deallocator.cpp:
721         (bmalloc::Deallocator::scavenge):
722         (bmalloc::Deallocator::deallocateLarge):
723         (bmalloc::Deallocator::deallocateXLarge):
724         (bmalloc::Deallocator::processObjectLog):
725         (bmalloc::Deallocator::deallocateSmallLine):
726         (bmalloc::Deallocator::deallocateXSmallLine):
727         (bmalloc::Deallocator::allocateSmallLine):
728         (bmalloc::Deallocator::allocateXSmallLine):
729         (bmalloc::Deallocator::deallocateMediumLine):
730         (bmalloc::Deallocator::allocateMediumLine):
731         * bmalloc/Deallocator.h:
732         * bmalloc/Heap.cpp:
733         (bmalloc::sleep):
734         (bmalloc::Heap::Heap):
735         (bmalloc::Heap::concurrentScavenge):
736         (bmalloc::Heap::scavenge):
737         (bmalloc::Heap::scavengeSmallPages):
738         (bmalloc::Heap::scavengeXSmallPages):
739         (bmalloc::Heap::scavengeMediumPages):
740         (bmalloc::Heap::scavengeLargeRanges):
741         (bmalloc::Heap::allocateXSmallLineSlowCase):
742         (bmalloc::Heap::allocateSmallLineSlowCase):
743         (bmalloc::Heap::allocateMediumLineSlowCase):
744         (bmalloc::Heap::allocateXLarge):
745         (bmalloc::Heap::deallocateXLarge):
746         (bmalloc::Heap::allocateLarge):
747         (bmalloc::Heap::deallocateLarge):
748         * bmalloc/Heap.h:
749         (bmalloc::Heap::deallocateXSmallLine):
750         (bmalloc::Heap::allocateXSmallLine):
751         (bmalloc::Heap::deallocateSmallLine):
752         (bmalloc::Heap::allocateSmallLine):
753         (bmalloc::Heap::deallocateMediumLine):
754         (bmalloc::Heap::allocateMediumLine):
755         * bmalloc/Line.h:
756         (bmalloc::Line<Traits>::deref):
757         * bmalloc/Mutex.cpp: Removed.
758         * bmalloc/Mutex.h:
759         (bmalloc::Mutex::Mutex):
760         (bmalloc::Mutex::init): Deleted.
761         (bmalloc::Mutex::try_lock): Deleted.
762         (bmalloc::Mutex::lock): Deleted.
763         (bmalloc::Mutex::unlock): Deleted.
764         * bmalloc/Page.h:
765         (bmalloc::Page<Traits>::ref):
766         (bmalloc::Page<Traits>::deref):
767         (bmalloc::Page<Traits>::refCount):
768         * bmalloc/PerProcess.h:
769         (bmalloc::PerProcess::mutex):
770         (bmalloc::PerProcess<T>::getSlowCase):
771         * bmalloc/StaticMutex.cpp: Added.
772         (bmalloc::StaticMutex::lockSlowCase):
773         * bmalloc/StaticMutex.h: Added.
774         (bmalloc::StaticMutex::init):
775         (bmalloc::StaticMutex::try_lock):
776         (bmalloc::StaticMutex::lock):
777         (bmalloc::StaticMutex::unlock):
778         * bmalloc/VMHeap.h:
779         (bmalloc::VMHeap::deallocateXSmallPage):
780         (bmalloc::VMHeap::deallocateSmallPage):
781         (bmalloc::VMHeap::deallocateMediumPage):
782         (bmalloc::VMHeap::deallocateLargeRange):
783         * bmalloc/bmalloc.h:
784         (bmalloc::api::scavenge): Global replace Mutex => StaticMutex,
785         since the Heap mutex is a static.
786
787 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
788
789         bmalloc: AsyncTask should use Mutex instead of std::mutex
790         https://bugs.webkit.org/show_bug.cgi?id=131865
791
792         Reviewed by Gavin Barraclough.
793
794         std::mutex is so slow that it makes parallelizing simple tasks through
795         AsyncTask a net regression. Mutex fixes this.
796
797         * bmalloc/AsyncTask.h:
798         (bmalloc::Function>::AsyncTask):
799         (bmalloc::Function>::join):
800         (bmalloc::Function>::runSlowCase):
801         (bmalloc::Function>::entryPoint):
802         * bmalloc/Mutex.h:
803         (bmalloc::Mutex::init):
804
805 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
806
807         bmalloc: Added an XSmall line size
808         https://bugs.webkit.org/show_bug.cgi?id=131851
809
810         Reviewed by Sam Weinig.
811
812         Reduces malloc footprint on Membuster recordings by 10%.
813
814         This is a throughput regression, but we're still way ahead of TCMalloc.
815         I have some ideas for how to recover the regression -- but I wanted to
816         get this win in first.
817
818         Full set of benchmark results:
819
820                 bmalloc> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks --measure-heap nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
821                                                                                                 
822                                                                nopatch                      patch                                Δ
823                 Peak Memory:
824                     reddit_memory_warning                      7,896kB                    7,532kB                  ^ 1.05x smaller
825                     flickr_memory_warning                     12,968kB                   12,324kB                  ^ 1.05x smaller
826                     theverge_memory_warning                   16,672kB                   15,200kB                   ^ 1.1x smaller
827
828                     <geometric mean>                          11,952kB                   11,216kB                  ^ 1.07x smaller
829                     <arithmetic mean>                         12,512kB                   11,685kB                  ^ 1.07x smaller
830                     <harmonic mean>                           11,375kB                   10,726kB                  ^ 1.06x smaller
831
832                 Memory at End:
833                     reddit_memory_warning                      7,320kB                    6,856kB                  ^ 1.07x smaller
834                     flickr_memory_warning                     10,848kB                    9,692kB                  ^ 1.12x smaller
835                     theverge_memory_warning                   16,380kB                   14,872kB                   ^ 1.1x smaller
836
837                     <geometric mean>                          10,916kB                    9,961kB                   ^ 1.1x smaller
838                     <arithmetic mean>                         11,516kB                   10,473kB                   ^ 1.1x smaller
839                     <harmonic mean>                           10,350kB                    9,485kB                  ^ 1.09x smaller
840
841                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
842                                                                                                 
843                                                    nopatch                patch                         Δ
844                 Execution Time:
845                     churn                            127ms                151ms            ! 1.19x slower
846                     list_allocate                    130ms                164ms            ! 1.26x slower
847                     tree_allocate                    109ms                127ms            ! 1.17x slower
848                     tree_churn                       115ms                120ms            ! 1.04x slower
849                     facebook                         240ms                259ms            ! 1.08x slower
850                     fragment                          91ms                131ms            ! 1.44x slower
851                     fragment_iterate                 105ms                106ms            ! 1.01x slower
852                     message_one                      260ms                259ms             ^ 1.0x faster
853                     message_many                     149ms                154ms            ! 1.03x slower
854                     medium                           194ms                248ms            ! 1.28x slower
855                     big                              157ms                160ms            ! 1.02x slower
856
857                     <geometric mean>                 144ms                163ms            ! 1.13x slower
858                     <arithmetic mean>                152ms                171ms            ! 1.12x slower
859                     <harmonic mean>                  137ms                156ms            ! 1.14x slower
860
861                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
862                                                                                                 
863                                                                        nopatch                          patch                                     Δ
864                 Execution Time:
865                     churn                                                126ms                          148ms                        ! 1.17x slower
866                     churn --parallel                                      62ms                           76ms                        ! 1.23x slower
867                     list_allocate                                        130ms                          164ms                        ! 1.26x slower
868                     list_allocate --parallel                             120ms                          175ms                        ! 1.46x slower
869                     tree_allocate                                        111ms                          127ms                        ! 1.14x slower
870                     tree_allocate --parallel                              95ms                          135ms                        ! 1.42x slower
871                     tree_churn                                           115ms                          124ms                        ! 1.08x slower
872                     tree_churn --parallel                                107ms                          126ms                        ! 1.18x slower
873                     facebook                                             240ms                          276ms                        ! 1.15x slower
874                     facebook --parallel                                  802ms                        1,088ms                        ! 1.36x slower
875                     fragment                                              92ms                          130ms                        ! 1.41x slower
876                     fragment --parallel                                   66ms                          124ms                        ! 1.88x slower
877                     fragment_iterate                                     109ms                          127ms                        ! 1.17x slower
878                     fragment_iterate --parallel                           55ms                           64ms                        ! 1.16x slower
879                     message_one                                          260ms                          260ms                                      
880                     message_many                                         170ms                          238ms                         ! 1.4x slower
881                     medium                                               185ms                          250ms                        ! 1.35x slower
882                     medium --parallel                                    210ms                          334ms                        ! 1.59x slower
883                     big                                                  150ms                          169ms                        ! 1.13x slower
884                     big --parallel                                       138ms                          144ms                        ! 1.04x slower
885
886                     <geometric mean>                                     135ms                          170ms                        ! 1.26x slower
887                     <arithmetic mean>                                    167ms                          214ms                        ! 1.28x slower
888                     <harmonic mean>                                      117ms                          148ms                        ! 1.26x slower
889
890                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
891
892                                                                     TC                      patch                                Δ
893                 Peak Memory:
894                     reddit_memory_warning                     13,836kB                   13,436kB                  ^ 1.03x smaller
895                     flickr_memory_warning                     24,868kB                   25,188kB                   ! 1.01x bigger
896                     theverge_memory_warning                   24,504kB                   26,636kB                   ! 1.09x bigger
897
898                     <geometric mean>                          20,353kB                   20,812kB                   ! 1.02x bigger
899                     <arithmetic mean>                         21,069kB                   21,753kB                   ! 1.03x bigger
900                     <harmonic mean>                           19,570kB                   19,780kB                   ! 1.01x bigger
901
902                 Memory at End:
903                     reddit_memory_warning                      8,656kB                   10,016kB                   ! 1.16x bigger
904                     flickr_memory_warning                     11,844kB                   13,784kB                   ! 1.16x bigger
905                     theverge_memory_warning                   18,516kB                   22,748kB                   ! 1.23x bigger
906
907                     <geometric mean>                          12,382kB                   14,644kB                   ! 1.18x bigger
908                     <arithmetic mean>                         13,005kB                   15,516kB                   ! 1.19x bigger
909                     <harmonic mean>                           11,813kB                   13,867kB                   ! 1.17x bigger
910
911                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
912                                                                                                 
913                                                         TC                patch                         Δ
914                 Execution Time:
915                     churn                            416ms                148ms            ^ 2.81x faster
916                     list_allocate                    463ms                164ms            ^ 2.82x faster
917                     tree_allocate                    292ms                127ms             ^ 2.3x faster
918                     tree_churn                       157ms                120ms            ^ 1.31x faster
919                     facebook                         327ms                276ms            ^ 1.18x faster
920                     fragment                         335ms                129ms             ^ 2.6x faster
921                     fragment_iterate                 344ms                108ms            ^ 3.19x faster
922                     message_one                      386ms                258ms             ^ 1.5x faster
923                     message_many                     410ms                154ms            ^ 2.66x faster
924                     medium                           391ms                245ms             ^ 1.6x faster
925                     big                              261ms                167ms            ^ 1.56x faster
926
927                     <geometric mean>                 332ms                164ms            ^ 2.02x faster
928                     <arithmetic mean>                344ms                172ms            ^ 1.99x faster
929                     <harmonic mean>                  317ms                157ms            ^ 2.02x faster
930
931         * bmalloc.xcodeproj/project.pbxproj:
932         * bmalloc/Allocator.cpp:
933         (bmalloc::Allocator::Allocator): Don't assume that each allocator's
934         index corresponds with its size. Instead, use the size selection function
935         explicitly. Now that we have XSmall, some small allocator entries are
936         unused.
937
938         (bmalloc::Allocator::scavenge):
939         (bmalloc::Allocator::log):
940         (bmalloc::Allocator::processXSmallAllocatorLog):
941         (bmalloc::Allocator::allocateSlowCase):
942         * bmalloc/Allocator.h:
943         (bmalloc::Allocator::xSmallAllocatorFor):
944         (bmalloc::Allocator::allocateFastCase):
945         * bmalloc/Chunk.h:
946         * bmalloc/Deallocator.cpp:
947         (bmalloc::Deallocator::scavenge):
948         (bmalloc::Deallocator::processObjectLog):
949         (bmalloc::Deallocator::deallocateSlowCase):
950         (bmalloc::Deallocator::deallocateXSmallLine):
951         (bmalloc::Deallocator::allocateXSmallLine):
952         * bmalloc/Deallocator.h:
953         (bmalloc::Deallocator::deallocateFastCase):
954         * bmalloc/Heap.cpp:
955         (bmalloc::Heap::scavenge):
956         (bmalloc::Heap::scavengeXSmallPages):
957         (bmalloc::Heap::allocateXSmallLineSlowCase):
958         * bmalloc/Heap.h:
959         (bmalloc::Heap::deallocateXSmallLine):
960         (bmalloc::Heap::allocateXSmallLine):
961         * bmalloc/LargeChunk.h:
962         (bmalloc::LargeChunk::get):
963         (bmalloc::LargeChunk::endTag):
964         * bmalloc/Line.h:
965         * bmalloc/MediumAllocator.h:
966         (bmalloc::MediumAllocator::allocate):
967         (bmalloc::MediumAllocator::refill):
968         * bmalloc/ObjectType.cpp:
969         (bmalloc::objectType):
970         * bmalloc/ObjectType.h:
971         (bmalloc::isXSmall):
972         (bmalloc::isSmall):
973         (bmalloc::isMedium):
974         (bmalloc::isLarge):
975         (bmalloc::isSmallOrMedium): Deleted.
976         * bmalloc/SegregatedFreeList.h: I boiler-plate copied existing code for
977         handling small objects. There's probably a reasonable way to share this
978         code in the future -- I'll look into that once it's stopped changing.
979
980         * bmalloc/Sizes.h: Tweaked size classes to make Membuster happy. This
981         is the main reason things got slower.
982
983         * bmalloc/SmallAllocator.h:
984         (bmalloc::SmallAllocator::allocate):
985         * bmalloc/SmallTraits.h:
986         * bmalloc/VMHeap.cpp:
987         (bmalloc::VMHeap::allocateXSmallChunk):
988         * bmalloc/VMHeap.h:
989         (bmalloc::VMHeap::allocateXSmallPage):
990         (bmalloc::VMHeap::deallocateXSmallPage):
991         * bmalloc/XSmallAllocator.h: Added.
992         (bmalloc::XSmallAllocator::isNull):
993         (bmalloc::XSmallAllocator::canAllocate):
994         (bmalloc::XSmallAllocator::XSmallAllocator):
995         (bmalloc::XSmallAllocator::line):
996         (bmalloc::XSmallAllocator::allocate):
997         (bmalloc::XSmallAllocator::objectCount):
998         (bmalloc::XSmallAllocator::derefCount):
999         (bmalloc::XSmallAllocator::refill):
1000         * bmalloc/XSmallChunk.h: Added.
1001         * bmalloc/XSmallLine.h: Added.
1002         * bmalloc/XSmallPage.h: Added.
1003         * bmalloc/XSmallTraits.h: Added.
1004         * bmalloc/bmalloc.h:
1005         (bmalloc::api::realloc): Boiler-plate copy, as above.
1006
1007 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
1008
1009         MallocBench should scavenge explicitly instead of waiting
1010         https://bugs.webkit.org/show_bug.cgi?id=131661
1011
1012         Reviewed by Andreas Kling.
1013
1014         Added explicit scavenge support to bmalloc. This isn't a memory win,
1015         since bmalloc's per-thread cache is so small. But it makes testing
1016         simpler.
1017
1018         * bmalloc/Allocator.cpp:
1019         (bmalloc::Allocator::~Allocator):
1020         (bmalloc::Allocator::scavenge):
1021         * bmalloc/Allocator.h:
1022         * bmalloc/Cache.cpp:
1023         (bmalloc::Cache::operator new):
1024         (bmalloc::Cache::operator delete):
1025         (bmalloc::Cache::Cache):
1026         (bmalloc::Cache::scavenge):
1027         * bmalloc/Cache.h:
1028         * bmalloc/Deallocator.cpp:
1029         (bmalloc::Deallocator::~Deallocator):
1030         (bmalloc::Deallocator::scavenge):
1031         * bmalloc/Deallocator.h: Factored existing scavenging code into helper
1032         functions, for reuse.
1033
1034         * bmalloc/Heap.cpp:
1035         (bmalloc::sleep):
1036         (bmalloc::Heap::concurrentScavenge):
1037         (bmalloc::Heap::scavenge):
1038         (bmalloc::Heap::scavengeSmallPages):
1039         (bmalloc::Heap::scavengeMediumPages):
1040         (bmalloc::Heap::scavengeLargeRanges):
1041         * bmalloc/Heap.h: Made scavenge sleep duration a parameter. Forced
1042         scavenging -- in response to a benchmark or a low memory warning --
1043         wants to complete as soon as possible, so its sleep duration is 0.
1044
1045         * bmalloc/bmalloc.h:
1046         (bmalloc::api::scavenge):
1047         * bmalloc/mbmalloc.cpp: Exported the scavenge API for MallocBench's use.
1048
1049 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
1050
1051         Use 4kB pages on Mac
1052         https://bugs.webkit.org/show_bug.cgi?id=131658
1053
1054         Reviewed by Sam Weinig.
1055
1056         This reduces memory use a lot on Membuster:
1057
1058                                                                   base                      patch                                Δ
1059                 Execution Time:
1060                     reddit_memory_warning                         18ms                       17ms                   ^ 1.06x faster
1061                     flickr_memory_warning                         34ms                       36ms                   ! 1.06x slower
1062                     theverge_memory_warning                       39ms                       41ms                   ! 1.05x slower
1063
1064                     <geometric mean>                              29ms                       29ms                   ! 1.02x slower
1065                     <arithmetic mean>                             30ms                       31ms                   ! 1.03x slower
1066                     <harmonic mean>                               27ms                       27ms                    ^ 1.0x faster
1067
1068                 Peak Memory:
1069                     reddit_memory_warning                     16,412kB                   16,436kB                    ! 1.0x bigger
1070                     flickr_memory_warning                     30,120kB                   30,184kB                    ! 1.0x bigger
1071                     theverge_memory_warning                   33,408kB                   33,420kB                    ! 1.0x bigger
1072
1073                     <geometric mean>                          25,466kB                   25,499kB                    ! 1.0x bigger
1074                     <arithmetic mean>                         26,647kB                   26,680kB                    ! 1.0x bigger
1075                     <harmonic mean>                           24,181kB                   24,214kB                    ! 1.0x bigger
1076
1077                 Memory at End:
1078                     reddit_memory_warning                      2,404kB                    1,920kB                  ^ 1.25x smaller
1079                     flickr_memory_warning                      3,764kB                    3,072kB                  ^ 1.23x smaller
1080                     theverge_memory_warning                    3,648kB                    3,132kB                  ^ 1.16x smaller
1081
1082                     <geometric mean>                           3,208kB                    2,644kB                  ^ 1.21x smaller
1083                     <arithmetic mean>                          3,272kB                    2,708kB                  ^ 1.21x smaller
1084                     <harmonic mean>                            3,139kB                    2,574kB                  ^ 1.22x smaller
1085
1086
1087         * bmalloc.xcodeproj/project.pbxproj:
1088         * bmalloc/BPlatform.h: Added.
1089         * bmalloc/VMAllocate.h: Only use 16kB pages on iOS because the page size
1090         is 4kB on Mac.
1091
1092 2014-04-14  Alexey Proskuryakov  <ap@apple.com>
1093
1094         Fixed svn:ignore on bmalloc.xcodeproj, it had erroneous leading spaces.
1095
1096         * bmalloc.xcodeproj: Modified property svn:ignore.
1097
1098 2014-04-13  Geoffrey Garen  <ggaren@apple.com>
1099
1100         Fixed some mbmalloc exports
1101         https://bugs.webkit.org/show_bug.cgi?id=131599
1102
1103         Reviewed by Ryosuke Niwa.
1104
1105         * bmalloc.xcodeproj/project.pbxproj: Made some headers a private part
1106         of the project, so we can call them from API.
1107
1108         * bmalloc/mbmalloc.cpp: Marked the mbmalloc functions with default
1109         visibility, so they show up as exported in the .dylib.
1110
1111 2014-04-09  Geoffrey Garen  <ggaren@apple.com>
1112
1113         Put bmalloc headers in the right place
1114         https://bugs.webkit.org/show_bug.cgi?id=131464
1115
1116         Reviewed by Mark Rowe.
1117
1118         * Configurations/bmalloc.xcconfig: Set PRIVATE_HEADERS_FOLDER_PATH to
1119         specify that we don't just want to dump all of our generically-named
1120         headers into /usr/local/include.
1121
1122 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
1123
1124         Made bmalloc more #include friendly
1125         https://bugs.webkit.org/show_bug.cgi?id=131386
1126
1127         Reviewed by Andreas Kling.
1128
1129         Marked a bunch of headers private so they can be used from client code
1130         that #includes bmalloc.h.
1131
1132         Renamed ASSERT macros to BASSERT. This matches their header, which already
1133         had to be renamed, and fixes conflicts with WTF's ASSERT macros.
1134
1135         * bmalloc.xcodeproj/project.pbxproj:
1136         * bmalloc/Allocator.cpp:
1137         (bmalloc::Allocator::allocateSlowCase):
1138         * bmalloc/AsyncTask.h:
1139         (bmalloc::Function>::runSlowCase):
1140         * bmalloc/BAssert.h:
1141         * bmalloc/BoundaryTag.h:
1142         (bmalloc::BoundaryTag::setSize):
1143         * bmalloc/BoundaryTagInlines.h:
1144         (bmalloc::validate):
1145         (bmalloc::BoundaryTag::init):
1146         (bmalloc::BoundaryTag::deallocate):
1147         (bmalloc::BoundaryTag::splitLarge):
1148         (bmalloc::BoundaryTag::allocate):
1149         * bmalloc/Chunk.h:
1150         * bmalloc/Deallocator.cpp:
1151         (bmalloc::Deallocator::processObjectLog):
1152         (bmalloc::Deallocator::deallocateSlowCase):
1153         * bmalloc/Deallocator.h:
1154         (bmalloc::Deallocator::deallocateFastCase):
1155         * bmalloc/FixedVector.h:
1156         (bmalloc::Capacity>::operator):
1157         (bmalloc::Capacity>::push):
1158         (bmalloc::Capacity>::pop):
1159         (bmalloc::Capacity>::shrink):
1160         * bmalloc/Heap.cpp:
1161         (bmalloc::Heap::allocateLarge):
1162         * bmalloc/LargeChunk.h:
1163         (bmalloc::LargeChunk::get):
1164         (bmalloc::LargeChunk::endTag):
1165         * bmalloc/Line.h:
1166         (bmalloc::Line<Traits>::concurrentRef):
1167         (bmalloc::Line<Traits>::deref):
1168         * bmalloc/MediumAllocator.h:
1169         (bmalloc::MediumAllocator::allocate):
1170         * bmalloc/ObjectType.h:
1171         (bmalloc::isSmall):
1172         * bmalloc/Page.h:
1173         (bmalloc::Page<Traits>::ref):
1174         (bmalloc::Page<Traits>::deref):
1175         * bmalloc/PerThread.h:
1176         (bmalloc::PerThread<T>::getSlowCase):
1177         * bmalloc/SegregatedFreeList.cpp:
1178         (bmalloc::SegregatedFreeList::SegregatedFreeList):
1179         (bmalloc::SegregatedFreeList::insert):
1180         * bmalloc/SmallAllocator.h:
1181         (bmalloc::SmallAllocator::allocate):
1182         (bmalloc::SmallAllocator::refill):
1183         * bmalloc/Syscall.h:
1184         * bmalloc/VMAllocate.h:
1185         (bmalloc::vmValidate):
1186         (bmalloc::vmAllocate):
1187         (bmalloc::vmDeallocatePhysicalPagesSloppy):
1188         * bmalloc/Vector.h:
1189         (bmalloc::Vector<T>::operator):
1190         (bmalloc::Vector<T>::pop):
1191         (bmalloc::Vector<T>::shrink):
1192         * bmalloc/XLargeChunk.h:
1193         (bmalloc::XLargeChunk::range):
1194         (bmalloc::XLargeChunk::size):
1195
1196 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
1197
1198         Removed an unused file.
1199
1200         Unreviewed.
1201
1202         * bmalloc/AsyncTask.cpp: Removed.
1203
1204 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1205
1206         Build bmalloc on Mac
1207         https://bugs.webkit.org/show_bug.cgi?id=131333
1208
1209         Reviewed by Mark Rowe.
1210
1211         * Makefile: Added. For make clients.
1212
1213         These files are required for building any project in WebKit. I copied
1214         them from WTF:
1215         * Configurations: Added.
1216         * Configurations/Base.xcconfig: Added.
1217         * Configurations/DebugRelease.xcconfig: Added.
1218         * Configurations/bmalloc.xcconfig: Added.
1219         * Configurations/iOS.xcconfig: Added.
1220         * Configurations/mbmalloc.xcconfig: Added.
1221
1222         * bmalloc.xcodeproj/project.pbxproj: I removed per-project-file stuff
1223         from here because everything is in .xcconfig files now.
1224
1225         I had to fix a bunch of minor warnings, since they're enabled in our
1226         .xcconfig files:
1227
1228         * bmalloc/AsyncTask.h:
1229         (bmalloc::Function>::AsyncTask):
1230         * bmalloc/BAssert.h:
1231         * bmalloc/BoundaryTagInlines.h:
1232         (bmalloc::validate):
1233         * bmalloc/Heap.cpp:
1234         (bmalloc::Heap::Heap):
1235         (bmalloc::Heap::allocateLarge):
1236         (bmalloc::Heap::deallocateLarge):
1237         * bmalloc/Mutex.h:
1238         (bmalloc::Mutex::Mutex): Deleted.
1239         * bmalloc/VMAllocate.h:
1240         (bmalloc::vmValidate):
1241         * bmalloc/mbmalloc.cpp:
1242
1243 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1244
1245         bmalloc: Fixed a leak in the per-thread cache
1246         https://bugs.webkit.org/show_bug.cgi?id=131330
1247
1248         Reviewed by Andreas Kling.
1249
1250         Remember to deallocate our line caches upon thread exit.
1251
1252         * bmalloc/Deallocator.cpp:
1253         (bmalloc::Deallocator::~Deallocator):
1254
1255 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1256
1257         bmalloc: rolled out the tryLock experiment
1258         https://bugs.webkit.org/show_bug.cgi?id=131328
1259
1260         Reviewed by Andreas Kling.
1261
1262         It wasn't a speedup.
1263
1264         * bmalloc.xcodeproj/project.pbxproj:
1265         * bmalloc/Allocator.cpp:
1266         (bmalloc::Allocator::processSmallAllocatorLog):
1267         (bmalloc::Allocator::processMediumAllocatorLog):
1268         * bmalloc/Deallocator.cpp:
1269         (bmalloc::Deallocator::processObjectLog):
1270         (bmalloc::Deallocator::deallocateSlowCase):
1271         (bmalloc::Deallocator::deallocateSmallLine):
1272         (bmalloc::Deallocator::deallocateMediumLine):
1273         * bmalloc/Deallocator.h:
1274         (bmalloc::Deallocator::deallocateFastCase):
1275         * bmalloc/Heap.h:
1276         (bmalloc::Heap::deallocateSmallLine):
1277         (bmalloc::Heap::deallocateMediumLine):
1278         * bmalloc/Line.h:
1279         (bmalloc::Line<Traits>::deref):
1280         * bmalloc/Page.h:
1281         (bmalloc::Page<Traits>::deref):
1282
1283 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1284
1285         bmalloc
1286         https://bugs.webkit.org/show_bug.cgi?id=131170
1287
1288         Reviewed by Andreas Kling.
1289
1290         Initial commit.
1291
1292         * bmalloc: Added.
1293         * bmalloc.xcodeproj: Added.
1294         * bmalloc.xcodeproj/project.pbxproj: Added.
1295         * bmalloc/Algorithm.h: Added.
1296         (bmalloc::max):
1297         (bmalloc::min):
1298         (bmalloc::mask):
1299         (bmalloc::test):
1300         (bmalloc::roundUpToMultipleOf):
1301         (bmalloc::roundDownToMultipleOf):
1302         (bmalloc::sizeOf):
1303         (bmalloc::bitCount):
1304         (bmalloc::isPowerOfTwo):
1305         * bmalloc/Allocator.cpp: Added.
1306         (bmalloc::Allocator::Allocator):
1307         (bmalloc::Allocator::~Allocator):
1308         (bmalloc::Allocator::log):
1309         (bmalloc::Allocator::processSmallAllocatorLog):
1310         (bmalloc::Allocator::processMediumAllocatorLog):
1311         (bmalloc::Allocator::allocateLarge):
1312         (bmalloc::Allocator::allocateXLarge):
1313         (bmalloc::Allocator::allocateMedium):
1314         (bmalloc::Allocator::allocateSlowCase):
1315         * bmalloc/Allocator.h: Added.
1316         (bmalloc::Allocator::smallAllocatorFor):
1317         (bmalloc::Allocator::allocateFastCase):
1318         (bmalloc::Allocator::allocate):
1319         * bmalloc/AsyncTask.cpp: Added.
1320         (bmalloc::AsyncTask<Function>::runSlowCase):
1321         (bmalloc::AsyncTask<Function>::pthreadEntryPoint):
1322         (bmalloc::AsyncTask<Function>::entryPoint):
1323         * bmalloc/AsyncTask.h: Added.
1324         (bmalloc::Function>::AsyncTask):
1325         (bmalloc::Function>::join):
1326         (bmalloc::Function>::run):
1327         (bmalloc::Function>::runSlowCase):
1328         (bmalloc::Function>::pthreadEntryPoint):
1329         (bmalloc::Function>::entryPoint):
1330         * bmalloc/BAssert.h: Added.
1331         * bmalloc/BeginTag.h: Added.
1332         (bmalloc::BeginTag::isInFreeList):
1333         * bmalloc/BoundaryTag.h: Added.
1334         (bmalloc::BoundaryTag::isXLarge):
1335         (bmalloc::BoundaryTag::setXLarge):
1336         (bmalloc::BoundaryTag::isFree):
1337         (bmalloc::BoundaryTag::setFree):
1338         (bmalloc::BoundaryTag::isEnd):
1339         (bmalloc::BoundaryTag::setEnd):
1340         (bmalloc::BoundaryTag::hasPhysicalPages):
1341         (bmalloc::BoundaryTag::setHasPhysicalPages):
1342         (bmalloc::BoundaryTag::isNull):
1343         (bmalloc::BoundaryTag::clear):
1344         (bmalloc::BoundaryTag::size):
1345         (bmalloc::BoundaryTag::setSize):
1346         (bmalloc::BoundaryTag::prev):
1347         (bmalloc::BoundaryTag::next):
1348         * bmalloc/BoundaryTagInlines.h: Added.
1349         (bmalloc::validate):
1350         (bmalloc::validatePrev):
1351         (bmalloc::validateNext):
1352         (bmalloc::BoundaryTag::init):
1353         (bmalloc::BoundaryTag::mergeLargeLeft):
1354         (bmalloc::BoundaryTag::mergeLargeRight):
1355         (bmalloc::BoundaryTag::mergeLarge):
1356         (bmalloc::BoundaryTag::deallocate):
1357         (bmalloc::BoundaryTag::splitLarge):
1358         (bmalloc::BoundaryTag::allocate):
1359         * bmalloc/Cache.cpp: Added.
1360         (bmalloc::Cache::operator new):
1361         (bmalloc::Cache::operator delete):
1362         (bmalloc::Cache::Cache):
1363         (bmalloc::Cache::allocateSlowCase):
1364         (bmalloc::Cache::allocateSlowCaseNullCache):
1365         (bmalloc::Cache::deallocateSlowCase):
1366         (bmalloc::Cache::deallocateSlowCaseNullCache):
1367         * bmalloc/Cache.h: Added.
1368         (bmalloc::Cache::allocator):
1369         (bmalloc::Cache::deallocator):
1370         (bmalloc::Cache::allocateFastCase):
1371         (bmalloc::Cache::deallocateFastCase):
1372         (bmalloc::Cache::allocate):
1373         (bmalloc::Cache::deallocate):
1374         * bmalloc/Chunk.h: Added.
1375         (bmalloc::Chunk::begin):
1376         (bmalloc::Chunk::end):
1377         (bmalloc::Chunk::lines):
1378         (bmalloc::Chunk::pages):
1379         * bmalloc/Deallocator.cpp: Added.
1380         (bmalloc::Deallocator::Deallocator):
1381         (bmalloc::Deallocator::~Deallocator):
1382         (bmalloc::Deallocator::deallocateLarge):
1383         (bmalloc::Deallocator::deallocateXLarge):
1384         (bmalloc::Deallocator::processObjectLog):
1385         (bmalloc::Deallocator::deallocateSlowCase):
1386         (bmalloc::Deallocator::deallocateSmallLine):
1387         (bmalloc::Deallocator::allocateSmallLine):
1388         (bmalloc::Deallocator::deallocateMediumLine):
1389         (bmalloc::Deallocator::allocateMediumLine):
1390         * bmalloc/Deallocator.h: Added.
1391         (bmalloc::Deallocator::deallocateFastCase):
1392         (bmalloc::Deallocator::deallocate):
1393         * bmalloc/EndTag.h: Added.
1394         (bmalloc::EndTag::operator=):
1395         * bmalloc/FixedVector.h: Added.
1396         (bmalloc::FixedVector::begin):
1397         (bmalloc::FixedVector::end):
1398         (bmalloc::FixedVector::size):
1399         (bmalloc::FixedVector::capacity):
1400         (bmalloc::FixedVector::clear):
1401         (bmalloc::FixedVector::isEmpty):
1402         (bmalloc::Capacity>::FixedVector):
1403         (bmalloc::Capacity>::operator):
1404         (bmalloc::Capacity>::push):
1405         (bmalloc::Capacity>::pop):
1406         (bmalloc::Capacity>::shrink):
1407         * bmalloc/Heap.cpp: Added.
1408         (bmalloc::sleep):
1409         (bmalloc::Heap::Heap):
1410         (bmalloc::Heap::concurrentScavenge):
1411         (bmalloc::Heap::scavengeSmallPages):
1412         (bmalloc::Heap::scavengeMediumPages):
1413         (bmalloc::Heap::scavengeLargeRanges):
1414         (bmalloc::Heap::allocateSmallLineSlowCase):
1415         (bmalloc::Heap::allocateMediumLineSlowCase):
1416         (bmalloc::Heap::allocateXLarge):
1417         (bmalloc::Heap::deallocateXLarge):
1418         (bmalloc::Heap::allocateLarge):
1419         (bmalloc::Heap::deallocateLarge):
1420         * bmalloc/Heap.h: Added.
1421         (bmalloc::Heap::deallocateSmallLine):
1422         (bmalloc::Heap::allocateSmallLine):
1423         (bmalloc::Heap::deallocateMediumLine):
1424         (bmalloc::Heap::allocateMediumLine):
1425         * bmalloc/Inline.h: Added.
1426         * bmalloc/LargeChunk.h: Added.
1427         (bmalloc::LargeChunk::begin):
1428         (bmalloc::LargeChunk::end):
1429         (bmalloc::LargeChunk::create):
1430         (bmalloc::LargeChunk::get):
1431         (bmalloc::LargeChunk::beginTag):
1432         (bmalloc::LargeChunk::endTag):
1433         * bmalloc/Line.h: Added.
1434         (bmalloc::Line<Traits>::begin):
1435         (bmalloc::Line<Traits>::end):
1436         (bmalloc::Line<Traits>::concurrentRef):
1437         (bmalloc::Line<Traits>::deref):
1438         * bmalloc/MediumAllocator.h: Added.
1439         (bmalloc::MediumAllocator::isNull):
1440         (bmalloc::MediumAllocator::MediumAllocator):
1441         (bmalloc::MediumAllocator::line):
1442         (bmalloc::MediumAllocator::allocate):
1443         (bmalloc::MediumAllocator::derefCount):
1444         (bmalloc::MediumAllocator::refill):
1445         * bmalloc/MediumChunk.h: Added.
1446         * bmalloc/MediumLine.h: Added.
1447         * bmalloc/MediumPage.h: Added.
1448         * bmalloc/MediumTraits.h: Added.
1449         * bmalloc/Mutex.cpp: Added.
1450         (bmalloc::Mutex::lockSlowCase):
1451         * bmalloc/Mutex.h: Added.
1452         (bmalloc::Mutex::Mutex):
1453         (bmalloc::Mutex::try_lock):
1454         (bmalloc::Mutex::lock):
1455         (bmalloc::Mutex::unlock):
1456         * bmalloc/ObjectType.cpp: Added.
1457         (bmalloc::objectType):
1458         * bmalloc/ObjectType.h: Added.
1459         (bmalloc::isSmallOrMedium):
1460         (bmalloc::isSmall):
1461         * bmalloc/Page.h: Added.
1462         (bmalloc::Page<Traits>::ref):
1463         (bmalloc::Page<Traits>::deref):
1464         (bmalloc::Page<Traits>::refCount):
1465         * bmalloc/PerProcess.h: Added.
1466         (bmalloc::PerProcess::mutex):
1467         (bmalloc::PerProcess<T>::getFastCase):
1468         (bmalloc::PerProcess<T>::get):
1469         (bmalloc::PerProcess<T>::getSlowCase):
1470         * bmalloc/PerThread.h: Added.
1471         (bmalloc::PerThreadStorage<Cache>::get):
1472         (bmalloc::PerThreadStorage<Cache>::init):
1473         (bmalloc::PerThreadStorage::get):
1474         (bmalloc::PerThreadStorage::init):
1475         (bmalloc::PerThread<T>::getFastCase):
1476         (bmalloc::PerThread<T>::get):
1477         (bmalloc::PerThread<T>::destructor):
1478         (bmalloc::PerThread<T>::getSlowCase):
1479         * bmalloc/Range.h: Added.
1480         (bmalloc::Range::Range):
1481         (bmalloc::Range::begin):
1482         (bmalloc::Range::end):
1483         (bmalloc::Range::size):
1484         (bmalloc::Range::operator!):
1485         (bmalloc::Range::operator<):
1486         * bmalloc/SegregatedFreeList.cpp: Added.
1487         (bmalloc::SegregatedFreeList::SegregatedFreeList):
1488         (bmalloc::SegregatedFreeList::insert):
1489         (bmalloc::SegregatedFreeList::takeGreedy):
1490         (bmalloc::SegregatedFreeList::take):
1491         * bmalloc/SegregatedFreeList.h: Added.
1492         * bmalloc/Sizes.h: Added.
1493         * bmalloc/SmallAllocator.h: Added.
1494         (bmalloc::SmallAllocator::isNull):
1495         (bmalloc::SmallAllocator::canAllocate):
1496         (bmalloc::SmallAllocator::SmallAllocator):
1497         (bmalloc::SmallAllocator::line):
1498         (bmalloc::SmallAllocator::allocate):
1499         (bmalloc::SmallAllocator::objectCount):
1500         (bmalloc::SmallAllocator::derefCount):
1501         (bmalloc::SmallAllocator::refill):
1502         * bmalloc/SmallChunk.h: Added.
1503         * bmalloc/SmallLine.h: Added.
1504         * bmalloc/SmallPage.h: Added.
1505         * bmalloc/SmallTraits.h: Added.
1506         * bmalloc/Syscall.h: Added.
1507         * bmalloc/VMAllocate.h: Added.
1508         (bmalloc::vmSize):
1509         (bmalloc::vmValidate):
1510         (bmalloc::vmAllocate):
1511         (bmalloc::vmDeallocate):
1512         (bmalloc::vmDeallocatePhysicalPages):
1513         (bmalloc::vmAllocatePhysicalPages):
1514         (bmalloc::vmDeallocatePhysicalPagesSloppy):
1515         (bmalloc::vmAllocatePhysicalPagesSloppy):
1516         * bmalloc/VMHeap.cpp: Added.
1517         (bmalloc::VMHeap::VMHeap):
1518         (bmalloc::VMHeap::allocateSmallChunk):
1519         (bmalloc::VMHeap::allocateMediumChunk):
1520         (bmalloc::VMHeap::allocateLargeChunk):
1521         * bmalloc/VMHeap.h: Added.
1522         (bmalloc::VMHeap::allocateSmallPage):
1523         (bmalloc::VMHeap::allocateMediumPage):
1524         (bmalloc::VMHeap::allocateLargeRange):
1525         (bmalloc::VMHeap::deallocateSmallPage):
1526         (bmalloc::VMHeap::deallocateMediumPage):
1527         (bmalloc::VMHeap::deallocateLargeRange):
1528         * bmalloc/Vector.h: Added.
1529         (bmalloc::Vector::begin):
1530         (bmalloc::Vector::end):
1531         (bmalloc::Vector::size):
1532         (bmalloc::Vector::capacity):
1533         (bmalloc::Vector::last):
1534         (bmalloc::Vector::pop):
1535         (bmalloc::Vector<T>::Vector):
1536         (bmalloc::Vector<T>::~Vector):
1537         (bmalloc::Vector<T>::operator):
1538         (bmalloc::Vector<T>::push):
1539         (bmalloc::Vector<T>::pop):
1540         (bmalloc::Vector<T>::shrink):
1541         (bmalloc::Vector<T>::reallocateBuffer):
1542         (bmalloc::Vector<T>::shrinkCapacity):
1543         (bmalloc::Vector<T>::growCapacity):
1544         * bmalloc/XLargeChunk.h: Added.
1545         (bmalloc::XLargeChunk::get):
1546         (bmalloc::XLargeChunk::begin):
1547         (bmalloc::XLargeChunk::XLargeChunk):
1548         (bmalloc::XLargeChunk::create):
1549         (bmalloc::XLargeChunk::destroy):
1550         (bmalloc::XLargeChunk::range):
1551         (bmalloc::XLargeChunk::size):
1552         * bmalloc/bmalloc.h: Added.
1553         (bmalloc::api::malloc):
1554         (bmalloc::api::free):
1555         (bmalloc::api::realloc):
1556         * bmalloc/mbmalloc.cpp: Added.
1557