366f3e3a7f0e7e63b2427bdef4ca843d2c347dfc
[WebKit-https.git] / Source / bmalloc / ChangeLog
1 2015-02-18  Zan Dobersek  <zdobersek@igalia.com>
2
3         Build bmalloc through CMake as a static library. It's then linked either
4         into the WTF library (if built as a shared library) or into the JSC and
5         WebKit2 libraries. There's no need to build it as a standalone shared library.
6
7         Rubber-stamped by Carlos Garcia Campos.
8
9         * CMakeLists.txt:
10
11 2015-02-13  Gyuyoung Kim  <gyuyoung.kim@samsung.com>
12
13         [BMalloc] Add a FIXME comment for memory alignas
14         https://bugs.webkit.org/show_bug.cgi?id=141556
15
16         Reviewed by Csaba Osztrogonác.
17
18         * bmalloc/Chunk.h: Add a FIXME comment.
19         * bmalloc/LargeChunk.h: ditto.
20
21 2015-02-11  Csaba Osztrogonác  <ossy@webkit.org>
22
23         bmalloc buildfix on 32 bit Linux (x86/ARM)
24         https://bugs.webkit.org/show_bug.cgi?id=141472
25
26         Reviewed by Gyuyoung Kim.
27
28         * bmalloc/Algorithm.h:
29         (bmalloc::roundUpToMultipleOf):
30         * bmalloc/FixedVector.h:
31         (bmalloc::FixedVector::clear):
32         * bmalloc/Sizes.h:
33         (bmalloc::Sizes::sizeClass):
34
35 2015-02-11  Gyuyoung Kim  <gyuyoung.kim@samsung.com>
36
37         [EFL][GTK] Use bmalloc instead of tcmalloc
38         https://bugs.webkit.org/show_bug.cgi?id=140162
39
40         Reviewed by Carlos Garcia Campos.
41
42         Support to use bmalloc on EFL and GTK ports.
43
44         * CMakeLists.txt: Added.
45         * bmalloc/Allocator.cpp:
46         (bmalloc::Allocator::allocate):
47             Fix unused return value caused by posix_memalign().
48         * bmalloc/AsyncTask.h:
49         * bmalloc/BoundaryTag.h:
50         (bmalloc::BoundaryTag::clear):
51         * bmalloc/Chunk.h:
52             Change Traits::Page with Traits::PageType in order to fix
53             -fpermitive build error on EFL and GTK port. 
54         * bmalloc/EndTag.h:
55         (bmalloc::EndTag::operator=):
56         * bmalloc/Line.h: ditto.
57         * bmalloc/MediumTraits.h:
58         * bmalloc/Page.h: ditto.
59         * bmalloc/PerThread.h:
60             EFL port doesn't support __has_include definition yet.
61             Define HAVE_PTHREAD_MACHDEP_H according to check if __has_include is supported.
62         * bmalloc/SmallTraits.h: ditto.
63         * bmalloc/VMAllocate.h:
64         (bmalloc::vmDeallocatePhysicalPages):
65         (bmalloc::vmAllocatePhysicalPages):
66         * bmalloc/Vector.h:
67         (bmalloc::Vector<T>::push):
68         (bmalloc::Vector<T>::reallocateBuffer):
69
70 2015-01-31  Sam Weinig  <sam@webkit.org>
71
72         Remove even more Mountain Lion support
73         https://bugs.webkit.org/show_bug.cgi?id=141124
74
75         Reviewed by Alexey Proskuryakov.
76
77         * Configurations/Base.xcconfig:
78         * Configurations/DebugRelease.xcconfig:
79
80 2015-01-30  Geoffrey Garen  <ggaren@apple.com>
81
82         GC marking threads should clear malloc caches
83         https://bugs.webkit.org/show_bug.cgi?id=141097
84
85         Reviewed by Andreas Kling.
86
87         Split the scavenging API into per-thread vs global, so that you can
88         request to scavenge your own thread without scavenging the whole heap.
89
90         * bmalloc/Cache.cpp:
91         (bmalloc::Cache::scavenge):
92         * bmalloc/bmalloc.h:
93         (bmalloc::api::scavengeThisThread):
94         (bmalloc::api::scavenge):
95
96 2015-01-28  Dana Burkart  <dburkart@apple.com>
97
98         Move ASan flag settings from DebugRelease.xcconfig to Base.xcconfig
99         https://bugs.webkit.org/show_bug.cgi?id=136765
100
101         Reviewed by Alexey Proskuryakov.
102
103         * Configurations/Base.xcconfig:
104         * Configurations/DebugRelease.xcconfig:
105
106 2015-01-21  Geoffrey Garen  <ggaren@apple.com>
107
108         bmalloc: support aligned allocation
109         https://bugs.webkit.org/show_bug.cgi?id=140732
110
111         Reviewed by Andreas Kling.
112
113         * bmalloc/Allocator.cpp:
114         (bmalloc::Allocator::allocate): New function for aligned allocation.
115
116         Small and medium requests just allocate and free until they find an
117         aligned pointer. This is slightly inefficient in the worst case, but
118         still constant-time with little-to-no space overhead.
119
120         Large requests use a new API that requires the client to specify both
121         its ideal size and alignment, and the worst-case size you would have to
122         allocate in order to produce some interior pointer of the requested size
123         and alignment. We put the burden of this calculation on the client
124         because it simplifies things if we guarantee that allocation won't fail.
125
126         XLarge requests are easy: we just forward them to vmAllocate, which
127         already supported aligned requests.
128
129         * bmalloc/BoundaryTag.h:
130         * bmalloc/BoundaryTagInlines.h:
131         (bmalloc::BoundaryTag::mergeLeft):
132         (bmalloc::BoundaryTag::mergeRight):
133         (bmalloc::BoundaryTag::merge):
134         (bmalloc::BoundaryTag::deallocate):
135         (bmalloc::BoundaryTag::split):
136         (bmalloc::BoundaryTag::allocate): No behavior change here. I just
137         refactored the interface to remove some reference out parameters in
138         order to clarify what changes and what doesn't.
139
140         * bmalloc/Heap.cpp:
141         (bmalloc::Heap::allocateXLarge): Added an alignment API.
142
143         (bmalloc::Heap::allocateLarge):
144         * bmalloc/Heap.h: Added an alignment API. I split out allocateLarge into
145         a few variants, so aligned and unaligned allocation could share some code.
146
147         * bmalloc/SegregatedFreeList.cpp:
148         (bmalloc::SegregatedFreeList::take):
149         * bmalloc/SegregatedFreeList.h: Changed to use a separate, explicit API
150         for aligned allocation. It turns out that the aligned path is pretty
151         different, since it ends up searching for two potential ways to satisfy
152         an allocation: either large enough and aligned, or large enough to split
153         into something not aligned and something large enough and aligned.
154
155         * bmalloc/VMAllocate.h:
156         (bmalloc::vmAllocate): Switched alignment to come before size because
157         that's how the memalign API specifies it.
158
159         * bmalloc/VMHeap.h:
160         (bmalloc::VMHeap::allocateLargeRange): Added an alignment API.
161
162 2015-01-20  Geoffrey Garen  <ggaren@apple.com>
163
164         bmalloc: a little bit of cleanup
165         https://bugs.webkit.org/show_bug.cgi?id=140687
166
167         Reviewed by Anders Carlsson.
168
169         * bmalloc/Algorithm.h:
170         (bmalloc::isPowerOfTwo): Added a check for 0, since 0 would break a lot
171         of code.
172
173         * bmalloc/BoundaryTag.h:
174         * bmalloc/BoundaryTagInlines.h:
175         (bmalloc::BoundaryTag::mergeLeft):
176         (bmalloc::BoundaryTag::mergeRight):
177         (bmalloc::BoundaryTag::merge):
178         (bmalloc::BoundaryTag::deallocate):
179         (bmalloc::BoundaryTag::split):
180         (bmalloc::BoundaryTag::allocate):
181         (bmalloc::BoundaryTag::mergeLargeLeft): Deleted.
182         (bmalloc::BoundaryTag::mergeLargeRight): Deleted.
183         (bmalloc::BoundaryTag::mergeLarge): Deleted.
184         (bmalloc::BoundaryTag::splitLarge): Deleted. Removed the word "Large"
185         from all these functions, since boundary tags always pertain to large
186         objects, and putting the word "Large" everywhere wasn't helping to
187         explain that.
188
189         * bmalloc/Heap.cpp:
190         (bmalloc::Heap::allocateXLarge):
191         (bmalloc::Heap::findXLarge):
192         (bmalloc::Heap::deallocateXLarge):
193         * bmalloc/Heap.h:
194         * bmalloc/VMHeap.h:
195         (bmalloc::VMHeap::allocateXLarge): Deleted.
196         (bmalloc::VMHeap::findXLarge): Deleted.
197         (bmalloc::VMHeap::deallocateXLarge): Deleted. Moved XLarge allocation
198         from VMHeap to Heap. Since the purpose of the VMHeap is to cache VM
199         ranges, and the VMHeap never caches any XLarge ranges, it doesn't
200         really make sense for the VMHeap to be involved.
201
202 2015-01-16  Geoffrey Garen  <ggaren@apple.com>
203
204         bmalloc: refactored XLarge allocation for better alignment
205         https://bugs.webkit.org/show_bug.cgi?id=140582
206
207         Reviewed by Andreas Kling.
208
209         XLarge objects used to be Large objects with an extra bit of metadata
210         that said "actually, I'm not large -- I'm extra large".
211
212         The metadata header in an XLarge allocation made it impossible for the
213         XLarge object to honor a very large alignment request.
214
215         The solution is to stop using a metadata header for XLarge objects, and
216         instead to store explicit metadata on the side.
217
218         This is a bit less astonishing, which is also nice.
219
220         Finding XLarge metadata is now a linear search. That's probably OK, since
221         it was always so in TCMalloc, and the usual number of XLarge allocations
222         in a process is 0.
223
224         This design makes it possible for the heap to cache XLarge allocations
225         with and/or without physical pages. I haven't actually done that yet
226         because the tradeoffs are subtle, so I don't want to do anything without
227         a motivating test case.
228
229         * bmalloc.xcodeproj/project.pbxproj:
230         * bmalloc/Allocator.cpp:
231         (bmalloc::Allocator::reallocate): Removed the concept of an XLargeChunk,
232         since an XLarge allocation is now just a naked buffer without a header.
233
234         (bmalloc::Allocator::allocateXLarge): Added an explicit qualifier for
235         XLarge alignment, since XLargeChunk won't give this to us implicitly
236         anymore.
237
238         * bmalloc/BoundaryTag.h:
239         (bmalloc::BoundaryTag::setRange):
240         (bmalloc::BoundaryTag::isXLarge): Deleted.
241         (bmalloc::BoundaryTag::setXLarge): Deleted.
242         * bmalloc/BoundaryTagInlines.h:
243         (bmalloc::validate):
244         (bmalloc::BoundaryTag::deallocate): Removed the XLarge hacks from Large allocations.
245
246         * bmalloc/Deallocator.cpp:
247         (bmalloc::Deallocator::deallocateXLarge):
248         (bmalloc::Deallocator::deallocateSlowCase):
249         * bmalloc/Heap.cpp:
250         (bmalloc::Heap::findXLarge):
251         (bmalloc::Heap::allocateXLarge):
252         (bmalloc::Heap::deallocateXLarge):
253         * bmalloc/Heap.h: Updated for interface changes.
254
255         * bmalloc/ObjectType.cpp:
256         (bmalloc::objectType):
257         * bmalloc/ObjectType.h:
258         (bmalloc::isXLarge): We can now tell if a pointer is XLarge just by
259         examining its bit pattern -- just like we do for other kinds of
260         allocations -- which is nice.
261
262         * bmalloc/Sizes.h:
263         * bmalloc/VMHeap.h:
264         (bmalloc::VMHeap::allocateXLarge):
265         (bmalloc::VMHeap::findXLarge):
266         (bmalloc::VMHeap::deallocateXLarge): Keep an explicit vector of metadata
267         for XLarge allocations.
268
269         * bmalloc/XLargeChunk.h: Removed.
270
271 2015-01-16  Geoffrey Garen  <ggaren@apple.com>
272
273         bmalloc: added some infrastructure for aligned allocation
274         https://bugs.webkit.org/show_bug.cgi?id=140572
275
276         Reviewed by Andreas Kling.
277
278         * bmalloc/Algorithm.h:
279         (bmalloc::isPowerOfTwo):
280         (bmalloc::roundUpToMultipleOf):
281         (bmalloc::roundDownToMultipleOf): Refactored some duplicate code to use our
282         isPowerOfTwo helper function.
283
284         * bmalloc/Allocator.cpp:
285         (bmalloc::Allocator::allocate):
286         * bmalloc/Allocator.h: Stubbed out an implementation of aligned allocation.
287         Doesn't do anything yet, but does correctly forward to system malloc
288         when bmalloc is disabled.
289
290         * bmalloc/Cache.cpp:
291         (bmalloc::Cache::allocateSlowCaseNullCache):
292         * bmalloc/Cache.h:
293         (bmalloc::Cache::allocate):
294         * bmalloc/bmalloc.h:
295         (bmalloc::api::memalign):
296         * bmalloc/mbmalloc.cpp: Stubbed out an API for aligned allocation.
297
298 2015-01-13  Geoffrey Garen  <ggaren@apple.com>
299
300         Consider alignment when allocating from a SegregatedFreeList
301         https://bugs.webkit.org/show_bug.cgi?id=140408
302
303         Reviewed by Sam Weinig.
304
305         In preparation for supporting aligned allocation.
306
307         No performance change.
308
309         Since this is just one extra branch in an already expensive function,
310         I decided not to duplicate the function just to avoid the branch in
311         the un-aligned case.
312
313         * bmalloc/SegregatedFreeList.cpp:
314         (bmalloc::SegregatedFreeList::take):
315         * bmalloc/SegregatedFreeList.h:
316
317 2015-01-13  Geoffrey Garen  <ggaren@apple.com>
318
319         Renamed minimum to size in SegregatedFreeList
320         https://bugs.webkit.org/show_bug.cgi?id=140406
321
322         Reviewed by Sam Weinig.
323
324         In preparation for supporting aligned allocation.
325
326         * bmalloc/SegregatedFreeList.cpp:
327         (bmalloc::SegregatedFreeList::takeGreedy):
328         (bmalloc::SegregatedFreeList::take): Every size passed to malloc is
329         really just a minimum. Let's not imply that this value is special.
330
331 2015-01-11  Dan Bernstein  <mitz@apple.com>
332
333         Geoff is organized, but he is not an organization.
334
335         Rubber-stamped by Anders Carlsson.
336
337         * bmalloc.xcodeproj/project.pbxproj: Removed the ORGANIZATIONNAME project attribute.
338
339 2015-01-07  Geoffrey Garen  <ggaren@apple.com>
340
341         Make bmalloc work with ASan
342         https://bugs.webkit.org/show_bug.cgi?id=140194
343
344         Reviewed by Mark Lam.
345
346         * bmalloc/BPlatform.h: Added a way to detect Darwin OSes, since we need
347         an OS-specific API to test for loaded runtime libraries.
348
349         * bmalloc/Environment.cpp:
350         (bmalloc::isASanEnabled):
351         (bmalloc::Environment::computeIsBmallocEnabled): Disabled bmalloc if
352         ASan is enabled, since system malloc has the Asan hooks we need.
353
354         You could check for the ASan compile-time flag instead, but doing this
355         check at runtime prepares bmalloc for a world where it is a dynamic
356         library that might be loaded into projects it did not compile with.
357
358 2015-01-05  Geoffrey Garen  <ggaren@apple.com>
359
360         Fix up bmalloc's PerThread for use on Linux
361         https://bugs.webkit.org/show_bug.cgi?id=139804
362
363         Reviewed by Anders Carlsson.
364
365         The previous implementation was a bit slow.
366
367         * bmalloc/PerThread.h:
368         (bmalloc::PerThreadStorage<Cache>::get):
369         (bmalloc::PerThreadStorage::get):
370         (bmalloc::PerThreadStorage::init): Added a catch-all cross-platform Unix
371         way to do fast per-thread access without taking a lock every time. This
372         probably works on all the platforms we care about, and it matches other
373         techniques we use elsewhere in WebKit.
374
375         (bmalloc::PerThread<T>::getFastCase): Removed the conditional from
376         this class because PerThreadStorage now encapsulates everything that
377         needs to be conditional.
378
379         (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Deleted.
380
381 2014-12-26  Dan Bernstein  <mitz@apple.com>
382
383         <rdar://problem/19348208> REGRESSION (r177027): iOS builds use the wrong toolchain
384         https://bugs.webkit.org/show_bug.cgi?id=139950
385
386         Reviewed by David Kilzer.
387
388         * Configurations/Base.xcconfig: Only define TOOLCHAINS when building for OS X, doing so
389         in a manner that works with Xcode 5.1.1.
390
391 2014-12-15  Geoffrey Garen  <ggaren@apple.com>
392
393         Safari crashes when you set Malloc environment variables
394         https://bugs.webkit.org/show_bug.cgi?id=139656
395
396         Reviewed by Michael Saboff.
397
398         I forgot to cover the realloc() case. Whoops. (OoPS?)
399
400         This time around, I ran the full MallocBench test suite in Malloc=1
401         mode, and it passed.
402
403         * bmalloc/Allocator.cpp:
404         (bmalloc::Allocator::reallocate):
405         * bmalloc/Allocator.h: Pushed realloc() logic down into the allocator.
406         It needs to be down there so that we can do the short-circuiting check
407         for whether bmalloc is enabled first.
408
409         Also added the check.
410
411         * bmalloc/Cache.cpp:
412         (bmalloc::Cache::scavenge):
413         (bmalloc::Cache::Cache):
414         (bmalloc::Cache::reallocateSlowCaseNullCache):
415         * bmalloc/Cache.h:
416         (bmalloc::Cache::deallocator):
417         (bmalloc::Cache::reallocate): Ditto.
418
419         * bmalloc/bmalloc.h:
420         (bmalloc::api::free):
421         (bmalloc::api::realloc): Ditto.
422
423         (bmalloc::api::scavenge): Pushed this down into Cache to match the
424         surrounding functions.
425
426 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
427
428         bmalloc should support system memory analysis tools (part 2)
429         https://bugs.webkit.org/show_bug.cgi?id=139565
430
431         Reviewed by Mark Lam.
432
433         This patch actually queries the environment to see if memory analysis
434         tools have been enabled.
435
436         * bmalloc/Deallocator.cpp:
437         (bmalloc::Deallocator::scavenge): Don't process the object log if
438         we've disabled bmalloc because it will be full of invalid nullptrs.
439
440         * bmalloc/Environment.cpp:
441         (bmalloc::isMallocEnvironmentVariableSet): Test for the list of known
442         Malloc debugging flags. I also added a plain "Malloc" catch-all for
443         when you want to disable bmalloc without enabling any kind of funny
444         business.
445
446         It would be slightly nicer just to iterate the list of environment
447         variables and strstr them, but getenv is the more portable option,
448         and performance here doesn't really matter.
449
450         (bmalloc::isLibgmallocEnabled): Test for the libgmalloc insertion
451         environment variable.
452
453         (bmalloc::Environment::computeIsBmallocEnabled):
454
455 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
456
457         Try to fix the iOS simulator build.
458
459         #include the declaration of malloc / free.
460
461         * bmalloc/Allocator.cpp:
462         * bmalloc/Deallocator.cpp:
463
464 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
465
466         Try to fix the build.
467
468         * bmalloc.xcodeproj/project.pbxproj: Marked a header exported.
469
470 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
471
472         bmalloc should support system memory analysis tools (part 1)
473         https://bugs.webkit.org/show_bug.cgi?id=139559
474
475         Reviewed by Mark Lam.
476
477         This patch adds the hooks to disable bmalloc at runtime if certain
478         environment variables are set, but doesn't actually read from the
479         environment yet.
480
481         No performance change.
482
483         * bmalloc.xcodeproj/project.pbxproj: Added the Environment class, which
484         we'll use to read environment variables and see if memory analysis tools
485         have been enabled.
486
487         * bmalloc/Allocator.cpp:
488         (bmalloc::Allocator::Allocator):
489         (bmalloc::Allocator::allocateSlowCase): Added a hook to disable bmalloc
490         on the allocation path. We cache the setting to make the check fast.
491
492         * bmalloc/Allocator.h: Interface changes.
493
494         * bmalloc/Cache.cpp:
495         (bmalloc::Cache::Cache): Pass a heap pointer through to our allocator
496         and deallocator. This main purpose is to enable them to query the
497         environment for whether bmalloc is enabled; but this is also a slightly
498         cleaner way to guarantee to them that the Heap has been pre-initialized.
499
500         * bmalloc/Deallocator.cpp:
501         (bmalloc::Deallocator::Deallocator): If bmalloc is disable, artificially
502         fill the object log to force us to take the slow path on all deallocations.
503
504         (bmalloc::Deallocator::deallocateSlowCase): Do the disabled check.
505
506         * bmalloc/Deallocator.h: Interface changes.
507
508         * bmalloc/Environment.cpp: Added.
509         (bmalloc::Environment::Environment):
510         (bmalloc::Environment::computeIsBmallocEnabled):
511         * bmalloc/Environment.h: Added.
512         (bmalloc::Environment::isBmallocEnabled): This is the class that will
513         encapsulate looking for environment variables that turn on heap
514         analysis tools.
515
516         * bmalloc/Heap.h:
517         (bmalloc::Heap::environment):
518
519         * bmalloc/Mutex.h:
520         (bmalloc::Mutex::Mutex):
521         * bmalloc/StaticMutex.h: A little refactoring to clarify these comments,
522         since I got super confused about them while writing this patch.
523
524         * bmalloc/VMHeap.cpp: Fixed an #include.
525
526 2014-12-09  David Kilzer  <ddkilzer@apple.com>
527
528         Switch from using PLATFORM_NAME to SDK selectors in ANGLE, bmalloc, gtest, JavaScriptCore, WTF
529         <http://webkit.org/b/139212>
530
531         Reviewed by Joseph Pecoraro.
532
533         * Configurations/Base.xcconfig:
534         - Only set GCC_ENABLE_OBJC_GC, GCC_MODEL_TUNING and TOOLCHAINS
535           on OS X.
536         * Configurations/DebugRelease.xcconfig:
537         - Only set MACOSX_DEPLOYMENT_TARGET and SDKROOT on OS X.
538
539 2014-11-07  Geoffrey Garen  <ggaren@apple.com>
540
541         bmalloc uses 8X more virtual memory than necessary
542         https://bugs.webkit.org/show_bug.cgi?id=138495
543
544         Reviewed by Mark Lam.
545
546         iOS has a per-process virtual memory cap around 1GB, so there's some
547         value to not going totally ham with virtual memory.
548
549         We currently use about 8X the necessary amount:
550             - 2X to align our VM allocation
551             - 4X to reserve small / medium / (2) large chunk VM ranges per superchunk
552
553         We can cut that down:
554             - Return the unaligned portion of our VM allocation (-2X)
555             - Use all the chunks in a superchunk, instead of allocating one
556               chunk per superchunk (-4X)
557
558         * bmalloc/Algorithm.h:
559         (bmalloc::roundUpToMultipleOf): Added a non-constant version of this
560         function so we can call it with getpagesize() at runtime.
561
562         * bmalloc/Chunk.h:
563         * bmalloc/LargeChunk.h:
564         (bmalloc::LargeChunk::create): Deleted. Instead of each chunk allocating
565         its own VM, VMHeap allocates the superchunk and all the chunks in it at a time.
566
567         * bmalloc/VMAllocate.h:
568         (bmalloc::vmValidate):
569         (bmalloc::vmAllocate): ASSERT that mmap succeeds to make crashes clearer
570         if it does not succeed. Allocate precisely, and give back the extra.
571
572         * bmalloc/VMHeap.cpp:
573         (bmalloc::VMHeap::allocateSuperChunk):
574         (bmalloc::VMHeap::allocateSmallChunk): Deleted.
575         (bmalloc::VMHeap::allocateMediumChunk): Deleted.
576         (bmalloc::VMHeap::allocateLargeChunk): Deleted. Use all the chunks
577         in a superchunk, instead of just one.
578
579         * bmalloc/VMHeap.h:
580         (bmalloc::VMHeap::allocateSmallPage):
581         (bmalloc::VMHeap::allocateMediumPage):
582         (bmalloc::VMHeap::allocateLargeRange):
583         * bmalloc/XLargeChunk.h:
584         (bmalloc::XLargeChunk::create): Updated to match changes above.
585
586 2014-11-01  David Kilzer  <ddkilzer@apple.com>
587
588         JavaScriptCore is missing debug info for bmalloc because libbmalloc.a is stripped
589         <https://webkit.org/b/138286>
590         <rdar://problem/18847087>
591
592         Reviewed by Dan Bernstein.
593
594         * Configurations/bmalloc.xcconfig: Set STRIP_INSTALLED_PRODUCT
595         to NO for the target that produces libbmalloc.a so that the
596         debug symbols will be linked into JavaScriptCore and end up in
597         its dSYM file.
598
599 2014-10-30  Dana Burkart  <dburkart@apple.com>
600
601         <rdar://problem/18821260> Prepare for the mysterious future
602
603         Reviewed by Lucas Forschler.
604
605         * Configurations/Base.xcconfig:
606         * Configurations/DebugRelease.xcconfig:
607
608 2014-09-24  Geoffrey Garen  <ggaren@apple.com>
609
610         bmalloc: cleaned up fast path vs slow path
611         https://bugs.webkit.org/show_bug.cgi?id=137081
612
613         Reviewed by Sam Weinig.
614
615         Might be a 1% speedup on MallocBench. Also cleans up the code a bit.
616
617         * bmalloc/Allocator.cpp:
618         (bmalloc::Allocator::Allocator): Merged the small and medium range
619         caches, just like the small and medium allocators. Ranges are abstract
620         objects that don't really care whether they hold small or medium objects,
621         so they don't need to be segregated.
622
623         (bmalloc::Allocator::scavenge): Ditto.
624
625         (bmalloc::Allocator::allocateBumpRangeSlowCase):
626         (bmalloc::Allocator::allocateBumpRange): Same thing here, except that
627         we do care a tiny bit, because we need to specify small vs medium when
628         allocating new ranges from the heap, to ensure that the heap allocates
629         from the right segment of VM.
630
631         (bmalloc::Allocator::allocateLarge):
632         (bmalloc::Allocator::allocateXLarge): NO_INLINE because this was clouding
633         up the fast path. Large allocation performance is dominated by allocation
634         logic and initialization, so inlining it doesn't help.
635
636         (bmalloc::Allocator::allocateSlowCase): Slow path got a bit cleaner since
637         it doesn't need to distinguish small vs medium objects.
638
639         (bmalloc::Allocator::allocateSmallBumpRange): Deleted.
640         (bmalloc::Allocator::allocateMediumBumpRange): Deleted.
641
642         * bmalloc/Allocator.h:
643         * bmalloc/BumpRange.h:
644
645         * bmalloc/Cache.cpp:
646         (bmalloc::Cache::allocateSlowCase): Deleted.
647         (bmalloc::Cache::deallocateSlowCase): Deleted.
648         * bmalloc/Cache.h:
649         (bmalloc::Cache::allocate):
650         (bmalloc::Cache::deallocate):
651         (bmalloc::Cache::allocateFastCase): Deleted.
652         (bmalloc::Cache::deallocateFastCase): Deleted. Removed the Cache slow
653         paths. The downside to this change is that the fast path branches to two
654         distinct failure cases instead of one. The upside is that the slow path
655         doesn't need to re-read the segment register, which is not as cheap as a
656         normal register, and it doesn't need to do an extra level of function 
657         call. Seems to be worth it.
658
659         * bmalloc/Deallocator.h:
660         * bmalloc/Heap.cpp:
661         (bmalloc::Heap::refillSmallBumpRangeCache):
662         (bmalloc::Heap::refillMediumBumpRangeCache):
663         * bmalloc/Heap.h: Updated for interface changes.
664
665         * bmalloc/Sizes.h: The most ranges a cache will hold is the number of
666         small lines in a page / 2, since any other free lines will coalesce
667         with their neighbors.
668
669 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
670
671         Rolled out r173346.
672
673             bmalloc should honor the FastMalloc statistics API
674             https://bugs.webkit.org/show_bug.cgi?id=136592
675
676         This didn't really work. Because we allow ranges with and without
677         physical pages to merge, and we allow double-committing and
678         double-decommitting, we can't rely on commit actions to track memory
679         footprint.
680
681         * bmalloc/Heap.cpp:
682         (bmalloc::Heap::size): Deleted.
683         (bmalloc::Heap::capacity): Deleted.
684         * bmalloc/Heap.h:
685         * bmalloc/VMHeap.cpp:
686         (bmalloc::VMHeap::VMHeap):
687         (bmalloc::VMHeap::allocateSmallChunk):
688         (bmalloc::VMHeap::allocateMediumChunk):
689         (bmalloc::VMHeap::allocateLargeChunk):
690         * bmalloc/VMHeap.h:
691         (bmalloc::VMHeap::allocateSmallPage):
692         (bmalloc::VMHeap::allocateMediumPage):
693         (bmalloc::VMHeap::allocateLargeRange):
694         (bmalloc::VMHeap::deallocateSmallPage):
695         (bmalloc::VMHeap::deallocateMediumPage):
696         (bmalloc::VMHeap::deallocateLargeRange):
697         (bmalloc::VMHeap::size): Deleted.
698         (bmalloc::VMHeap::capacity): Deleted.
699         * bmalloc/bmalloc.h:
700         (bmalloc::api::heapSize): Deleted.
701         (bmalloc::api::heapCapacity): Deleted.
702
703 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
704
705         bmalloc: Allocation should be more precise
706         https://bugs.webkit.org/show_bug.cgi?id=136993
707
708         Reviewed by Gavin Barraclough.
709
710         13% reduction in heap size on the MallocBench *_memory_warning benchmarks.
711
712         This patch teaches the allocator to merge adjacent free lines into a
713         single allocatable range. This allows us to shrink the size of an
714         individual line without increasing fragmentation or the rate of allocator
715         slow paths.
716
717         We'll only take more slow paths when available memory is sparse, which
718         is exactly when it's worth it. When available memory is dense, we'll
719         take fewer slow paths.
720
721         * bmalloc.xcodeproj/project.pbxproj:
722         * bmalloc/Algorithm.h:
723         (bmalloc::divideRoundingUp):
724
725         * bmalloc/Allocator.cpp:
726         (bmalloc::Allocator::Allocator): Updated for interface changes.
727
728         (bmalloc::Allocator::scavenge): Scavenge by object instead of by line.
729         Now that we merge lines, it's not convenient to scavenge by line.
730
731         (bmalloc::Allocator::allocateSmallBumpRange):
732         (bmalloc::Allocator::allocateMediumBumpRange): Allocate whole ranges
733         instead of individual lines.
734
735         (bmalloc::Allocator::allocateSlowCase):
736         (bmalloc::Allocator::allocateSmallLine): Deleted.
737         (bmalloc::Allocator::allocateMediumLine): Deleted.
738         (bmalloc::Allocator::allocateMedium): Deleted.
739         * bmalloc/Allocator.h:
740         (bmalloc::Allocator::allocateFastCase): Folded medium allocations
741         into the standard fast path with small allocations. Since a BumpAllocator
742         just allocates out of an arbitrary range, it doesn't need to distinguish
743         between small and medium lines.
744
745         * bmalloc/BumpAllocator.h:
746         (bmalloc::BumpAllocator::size):
747         (bmalloc::BumpAllocator::BumpAllocator):
748         (bmalloc::BumpAllocator::init):
749         (bmalloc::BumpAllocator::refill):
750         (bmalloc::BumpAllocator::line): Deleted. No need to track line information
751         anymore: the heap just gives us a pointer and a pre-computed number of
752         objects, and we allocate them.
753
754         * bmalloc/Deallocator.cpp:
755         (bmalloc::Deallocator::processObjectLog): Updated for interface changes.
756
757         * bmalloc/Heap.cpp:
758         (bmalloc::Heap::Heap):
759         (bmalloc::Heap::initializeLineMetadata): Pre-compute precise metadata
760         detailing where all objects will lie in memory. After we merge two lines,
761         we might allocate an object that spans from one line to the next. This
762         metadata details which bits of memory overlap in that way, and how they
763         overlap.
764
765         (bmalloc::Heap::refillSmallBumpRangeCache):
766         (bmalloc::Heap::refillMediumBumpRangeCache): Scan a whole page at a time,
767         and merge adjacent free lines into BumpRanges.
768
769         (bmalloc::Heap::allocateSmallPage):
770         (bmalloc::Heap::allocateMediumPage):
771         (bmalloc::Heap::deallocateSmallLine):
772         (bmalloc::Heap::deallocateMediumLine): Track pages rather than lines,
773         since we scan for free memory a page at a time.
774
775         (bmalloc::Heap::allocateSmallLineSlowCase): Deleted.
776         (bmalloc::Heap::allocateMediumLineSlowCase): Deleted. Folded into the
777         fast path.
778
779         * bmalloc/Heap.h:
780         (bmalloc::Heap::derefSmallLine):
781         (bmalloc::Heap::derefMediumLine):
782         (bmalloc::Heap::deallocateSmallLine): Deleted.
783         (bmalloc::Heap::allocateSmallLine): Deleted.
784         (bmalloc::Heap::deallocateMediumLine): Deleted.
785         (bmalloc::Heap::allocateMediumLine): Deleted. Updated for interface changes.
786
787         * bmalloc/Line.h:
788         (bmalloc::Line<Traits>::ref):
789         (bmalloc::Line<Traits>::deref):
790         (bmalloc::Line<Traits>::concurrentRef): Deleted. We don't pass a derefCount
791         anymore, since we only ever deref by 1 now.
792
793         * bmalloc/MediumAllocator.h:
794         (bmalloc::MediumAllocator::isNull): Deleted.
795         (bmalloc::MediumAllocator::MediumAllocator): Deleted.
796         (bmalloc::MediumAllocator::line): Deleted.
797         (bmalloc::MediumAllocator::allocate): Deleted.
798         (bmalloc::MediumAllocator::derefCount): Deleted.
799         (bmalloc::MediumAllocator::refill): Deleted.
800         (bmalloc::MediumAllocator::clear): Deleted. Deleted some code that's
801         been dead for a while, since it doesn't build anymore with this patch.
802
803         * bmalloc/Page.h:
804         (bmalloc::Page::sizeClass):
805         (bmalloc::Page::setSizeClass):
806         (bmalloc::Page::smallSizeClass): Deleted.
807         (bmalloc::Page::setSmallSizeClass): Deleted. Renamed setSmallSizeClass
808         to sizeClass, since we use it for medium sizes too.
809
810         * bmalloc/Sizes.h:
811         (bmalloc::Sizes::sizeClass):
812         (bmalloc::Sizes::objectSize): Shrank line sizes to save memory.
813
814         (bmalloc::Sizes::smallSizeClassFor): Deleted.
815         (bmalloc::Sizes::mediumSizeClassFor): Deleted.
816
817         * bmalloc/bmalloc.h:
818         (bmalloc::api::realloc): Now that we have precise objects sizes, realloc
819         can be a bit more precise. It also has to be, since we can't guarantee
820         that an object ends at the end of a line anymore.
821
822 2014-09-19  Daniel Bates  <dabates@apple.com>
823
824         Always assume internal SDK when building configuration Production
825         https://bugs.webkit.org/show_bug.cgi?id=136925
826         <rdar://problem/18362399>
827
828         Reviewed by Dan Bernstein.
829
830         * Configurations/Base.xcconfig:
831
832 2014-09-16  Geoffrey Garen  <ggaren@apple.com>
833
834         bmalloc: moved line caches from the deallocator to the allocator
835         https://bugs.webkit.org/show_bug.cgi?id=136868
836
837         Reviewed by Gavin Barraclough.
838
839         I did this mostly as a simplification, to make it easier to change the
840         allocation strategy.
841
842         No throughput change on MallocBench. Saves about 50kB.
843
844         Since the deallocator needs to lock the heap when freeing lines anyway,
845         there isn't much benefit to giving the deallocator a local cache of
846         deallocated lines.
847
848         We still give the allocator a local cache of lines because that does
849         reduce the frequency at which it needs to lock the heap in order to
850         acquire more lines.
851
852         * bmalloc/Allocator.cpp:
853         (bmalloc::Allocator::scavenge):
854         (bmalloc::Allocator::allocateSmallLine):
855         (bmalloc::Allocator::allocateMediumLine):
856         (bmalloc::Allocator::allocateMedium):
857         (bmalloc::Allocator::allocateSlowCase):
858         * bmalloc/Allocator.h:
859         * bmalloc/Deallocator.cpp:
860         (bmalloc::Deallocator::Deallocator):
861         (bmalloc::Deallocator::scavenge):
862         (bmalloc::Deallocator::processObjectLog):
863         (bmalloc::Deallocator::deallocateSmallLine): Deleted.
864         (bmalloc::Deallocator::allocateSmallLine): Deleted.
865         (bmalloc::Deallocator::deallocateMediumLine): Deleted.
866         (bmalloc::Deallocator::allocateMediumLine): Deleted.
867         * bmalloc/Deallocator.h:
868
869         * bmalloc/Sizes.h:
870         * bmalloc/VMAllocate.h: Took the opportunity to make the line cache size
871         exactly one page in size. That's about what we were shooting for anyway,
872         and it may make it easier to switch to per-page allocation in future.
873
874 2014-09-15  Geoffrey Garen  <ggaren@apple.com>
875
876         bmalloc: allocate small and medium objects using the same bump pointer class
877         https://bugs.webkit.org/show_bug.cgi?id=136843
878
879         Reviewed by Gavin Barraclough.
880
881         4% speedup on MallocBench.
882
883         Now that medium-sized objects have dedicated per-size allocators, they
884         don't need to use an arbitrary bump pointer allocator. This means that
885         every allocator knows how many objects it will allocate from the start,
886         and we don't need a post-processing step to adjust refcounts based on
887         real allocation count.
888
889         * bmalloc.xcodeproj/project.pbxproj: Renamed SmallAllocator to BumpAllocator
890         since it's used for small and medium objects now.
891
892         * bmalloc/Allocator.cpp:
893         (bmalloc::Allocator::Allocator): Updated to use new interface.
894         (bmalloc::Allocator::scavenge): To "retire" an allocator, we just need
895         to make sure that we finish allocating all the objects in it.
896
897         (bmalloc::Allocator::allocateMedium):
898         (bmalloc::Allocator::allocateSlowCase):
899         (bmalloc::Allocator::retire): Deleted.
900         (bmalloc::Allocator::processSmallAllocatorLog): Deleted.
901         (bmalloc::Allocator::processMediumAllocatorLog): Deleted.
902         * bmalloc/Allocator.h:
903         (bmalloc::Allocator::allocateFastCase): Removed abstractions and data
904         used to post-process an allocator based on how many objects it allocated.
905
906         * bmalloc/BumpAllocator.h: Copied from Source/bmalloc/bmalloc/SmallAllocator.h.
907         (bmalloc::BumpAllocator::BumpAllocator):
908         (bmalloc::BumpAllocator::init):
909         (bmalloc::BumpAllocator::line):
910         (bmalloc::BumpAllocator::validate):
911         (bmalloc::BumpAllocator::allocate):
912         (bmalloc::BumpAllocator::refill):
913         (bmalloc::BumpAllocator::clear): Updated these functions to be agnostic
914         about the kinds of lines they allocate into. In some cases, the line
915         type must be provided as a template parameter by the caller.
916
917         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
918         (bmalloc::SmallAllocator::line): Deleted.
919         (bmalloc::SmallAllocator::allocate): Deleted.
920         (bmalloc::SmallAllocator::objectCount): Deleted.
921         (bmalloc::SmallAllocator::derefCount): Deleted.
922         (bmalloc::SmallAllocator::refill): Deleted.
923         (bmalloc::SmallAllocator::clear): Deleted.
924
925         * bmalloc/ObjectType.h:
926         (bmalloc::isMedium):
927
928         * bmalloc/SmallAllocator.h:
929         (bmalloc::SmallAllocator::isNull): Deleted.
930         (bmalloc::SmallAllocator::canAllocate): Deleted.
931         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
932         (bmalloc::SmallAllocator::line): Deleted.
933         (bmalloc::SmallAllocator::allocate): Deleted.
934         (bmalloc::SmallAllocator::objectCount): Deleted.
935         (bmalloc::SmallAllocator::derefCount): Deleted.
936         (bmalloc::SmallAllocator::refill): Deleted.
937         (bmalloc::SmallAllocator::clear): Deleted.
938
939 2014-09-12  Geoffrey Garen  <ggaren@apple.com>
940
941         Fixed a goof in bmalloc Vector sizing
942         https://bugs.webkit.org/show_bug.cgi?id=136795
943
944         Reviewed by Gavin Barraclough and Sam Weinig.
945
946         We want our minimum vector to be page-sized since the OS will give us
947         a page no matter what -- but we want that many bytes, and not enough
948         bytes to store that many elements.
949
950         * bmalloc/Vector.h: Math is hard.
951
952 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
953
954         bmalloc should segregate medium-sized objects by line like it does for small-sized objects
955         https://bugs.webkit.org/show_bug.cgi?id=136693
956
957         Reviewed by Gavin Barraclough.
958
959         4% reduction in heap size on the MallocBench *_memory_warning benchmarks.
960
961         No throughput change.
962
963         We keep an array of medium allocators, just like our array of small
964         allocators.
965
966         In future, we can simplify the allocation fast path by merging the small
967         and medium allocator arrays. For now, this is the simplest change that
968         gets the win.
969
970         * bmalloc/Allocator.cpp:
971         (bmalloc::Allocator::Allocator):
972         (bmalloc::Allocator::scavenge):
973         (bmalloc::Allocator::allocateMedium):
974         * bmalloc/Allocator.h:
975         * bmalloc/Sizes.h:
976         (bmalloc::Sizes::mediumSizeClassFor):
977
978 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
979
980         Reviewed by Sam Weinig.
981
982         Renamed log => retire for clarity.
983
984         * bmalloc/Allocator.cpp:
985         (bmalloc::Allocator::scavenge):
986         (bmalloc::Allocator::retire):
987         (bmalloc::Allocator::allocateMedium):
988         (bmalloc::Allocator::allocateSlowCase):
989         (bmalloc::Allocator::log): Deleted.
990         * bmalloc/Allocator.h:
991
992 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
993
994         bmalloc: eager scavenge leaves behind a bogus allocator
995         https://bugs.webkit.org/show_bug.cgi?id=136743
996
997         Reviewed by Sam Weinig.
998
999         Be sure to clear the allocator after logging it in the eager scavenge
1000         case, so that we don't later try to allocate out of the lines that we
1001         have thrown away.
1002
1003         We didn't need to do this previously because scavenge would only happen
1004         at thread exit time, after which no further allocation from the per-thread
1005         cache would take place.
1006
1007         * bmalloc/Allocator.cpp:
1008         (bmalloc::Allocator::scavenge):
1009         * bmalloc/MediumAllocator.h:
1010         (bmalloc::MediumAllocator::clear):
1011         * bmalloc/SmallAllocator.h:
1012         (bmalloc::SmallAllocator::clear):
1013
1014 2014-09-05  Geoffrey Garen  <ggaren@apple.com>
1015
1016         bmalloc should honor the FastMalloc statistics API
1017         https://bugs.webkit.org/show_bug.cgi?id=136592
1018
1019         Reviewed by Gavin Barraclough.
1020
1021         We do this by tracking "size" and "capacity" in the VM heap.
1022
1023         The VM heap's "capacity" is all the VM we ever allocated.
1024
1025         The VM heap's "size" the subset of VM currently held onto by the
1026         VM heap (and therefore not in use by the regular heap).
1027
1028         Somewhat ironically, reducing the process's memory footprint, increases
1029         the size of the VM heap, since the VM heap holds the pages that are
1030         purely virtual and not physical.
1031
1032         * bmalloc/Heap.cpp:
1033         (bmalloc::Heap::size):
1034         (bmalloc::Heap::capacity):
1035         * bmalloc/Heap.h:
1036         * bmalloc/VMHeap.cpp:
1037         (bmalloc::VMHeap::VMHeap):
1038         (bmalloc::VMHeap::allocateSmallChunk):
1039         (bmalloc::VMHeap::allocateMediumChunk):
1040         (bmalloc::VMHeap::allocateLargeChunk):
1041         * bmalloc/VMHeap.h:
1042         (bmalloc::VMHeap::size):
1043         (bmalloc::VMHeap::capacity):
1044         (bmalloc::VMHeap::allocateSmallPage):
1045         (bmalloc::VMHeap::allocateMediumPage):
1046         (bmalloc::VMHeap::allocateLargeRange):
1047         (bmalloc::VMHeap::deallocateSmallPage):
1048         (bmalloc::VMHeap::deallocateMediumPage):
1049         (bmalloc::VMHeap::deallocateLargeRange):
1050         * bmalloc/bmalloc.h:
1051         (bmalloc::api::heapSize):
1052         (bmalloc::api::heapCapacity):
1053
1054 2014-09-02  Geoffrey Garen  <ggaren@apple.com>
1055
1056         bmalloc crashes on the EWS bots (due to bad large object allocation)
1057         https://bugs.webkit.org/show_bug.cgi?id=136469
1058
1059         Reviewed by Andreas Kling.
1060
1061         It's possible to convince bmalloc to perform a bad large object allocation,
1062         through these steps:
1063
1064         (1) Insert object A into freelist F0.
1065
1066         (2) Split, merge and split again A's neighbors such that object B is
1067         inserted into freelist F0, with boundary tag and size equal to object A,
1068         but pointer not completely equal to object A. Put object B at the head of F0.
1069
1070         (3) Allocate some other object from F0, swapping its position in the
1071         freelist with object B, such that object A is now ahead of object B.
1072
1073         --> Now, the next allocation for size A/B will allocate object A, which
1074         has a slightly wrong idea about where the object actually begins.
1075         Immediately, you'll corrupt a little memory, and over time, you'll also
1076         corrupt boundary tag metadata.
1077
1078         The solution is to store the begin pointer in the boundary tag. Luckily,
1079         this doesn't make the tag any bigger, and it's not a noticeable slowdown
1080         on MallocBench.
1081
1082         * bmalloc/Algorithm.h:
1083         (bmalloc::rightShift):
1084         * bmalloc/BeginTag.h:
1085         (bmalloc::BeginTag::isInFreeList): This is the bug fix. Make sure to
1086         validate the start pointer when popping off the free list. Through a
1087         very uncommon set of steps, it is possible to have an item in the free
1088         list that is valid by all accounts except for its start pointer.
1089
1090         * bmalloc/BoundaryTag.h:
1091         (bmalloc::BoundaryTag::compactBegin):
1092         (bmalloc::BoundaryTag::setRange):
1093         (bmalloc::BoundaryTag::setSize): Deleted. Record a compact version of the
1094         start pointer. We don't need the whole pointer -- just the offset, in
1095         largeAlignment increments, into the relevant boundary tag bucket.
1096
1097         * bmalloc/BoundaryTagInlines.h:
1098         (bmalloc::validateNext):
1099         (bmalloc::BoundaryTag::init):
1100         (bmalloc::BoundaryTag::mergeLarge):
1101         (bmalloc::BoundaryTag::splitLarge):
1102         * bmalloc/SegregatedFreeList.cpp:
1103         (bmalloc::SegregatedFreeList::insert):
1104         (bmalloc::SegregatedFreeList::takeGreedy):
1105         (bmalloc::SegregatedFreeList::take): Provide the whole range instead of
1106         the size when establishing a boundary tag, as required by the new
1107         interface.
1108
1109         * bmalloc/Sizes.h:
1110
1111 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
1112
1113         Fixed a bmalloc crash seen on the EWS bot
1114         https://bugs.webkit.org/show_bug.cgi?id=135955
1115
1116         Reviewed by Andreas Kling.
1117
1118         * bmalloc/Syscall.h: Some CG APIs vm_copy their input buffers. If the
1119         input buffer is a malloc region, that region will get marked Copy-On-Write
1120         by the kernel. Calls to madvise() for COW regions fail and return EINVAL
1121         on older OS X's. In 10.10, they still fail, but they do not return
1122         EINVAL.
1123
1124         So, we can only ASSERT that our syscalls succeed starting with 10.10.
1125
1126 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
1127
1128         Fixed the bmalloc build
1129         https://bugs.webkit.org/show_bug.cgi?id=135953
1130
1131         Reviewed by Andreas Kling.
1132
1133         * bmalloc.xcodeproj/project.pbxproj: Marked a few headers as private.
1134         These headers are used, so they must be available outside the project.
1135
1136 2014-08-13  Daniel Bates  <dabates@apple.com>
1137
1138         Attempt to fix the build following <http://trac.webkit.org/changeset/172576>
1139         (https://bugs.webkit.org/show_bug.cgi?id=135895)
1140
1141         Substitute PerThreadStorage<T>::initSharedKeyIfNeeded() for initSharedKeyIfNeeded() in
1142         implementation of PerThread<T>::getFastCase().
1143
1144         * bmalloc/PerThread.h:
1145         (bmalloc::PerThread<T>::getFastCase):
1146
1147 2014-08-13  Daniel Bates  <dabates@apple.com>
1148
1149         Make bmalloc::PerThread work without C++ thread local storage
1150         https://bugs.webkit.org/show_bug.cgi?id=135895
1151
1152         Reviewed by Geoffrey Garen.
1153
1154         Implement support for building bmalloc without C++ thread local storage.
1155
1156         * bmalloc/BPlatform.h: Remove macro define BPLATFORM_IOS_SIMULATOR. Added macro function
1157         BCOMPILER_SUPPORTS() and macro define BCOMPILER_SUPPORTS_CXX_THREAD_LOCAL that can be used
1158         to determine whether the compiler supports C++ thread local storage.
1159         * bmalloc/PerThread.h:
1160         (bmalloc::PerThreadStorage::get): Modified to call pthread_getspecific() when building
1161         without C++ thread local storage.
1162         (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Added.
1163         (bmalloc::PerThreadStorage::init): Moved logic to initialize shared Pthread key from here to
1164         PerThreadStorage::initSharedKeyIfNeeded().
1165         (bmalloc::PerThread<T>::getFastCase): Modified to call PerThreadStorage::initSharedKeyIfNeeded()
1166         before querying PerThreadStorage::get() when building without C++ thread local storage so as to
1167         ensure that the shared key has been initialized.
1168         (_pthread_setspecific_direct): Deleted.
1169         (_pthread_getspecific_direct): Deleted.
1170
1171 2014-08-13  Daniel Bates  <dabates@apple.com>
1172
1173         [iOS] Make JavaScriptCore and bmalloc build with the public SDK
1174         https://bugs.webkit.org/show_bug.cgi?id=135848
1175
1176         Reviewed by Geoffrey Garen.
1177
1178         * bmalloc/BPlatform.h: Added macro BPLATFORM_IOS_SIMULATOR, which evaluates to true
1179         when building for the iOS Simulator.
1180         * bmalloc/PerThread.h: Use pthread_machdep.h code path when building for iOS Simulator
1181         using the public SDK.
1182         (_pthread_setspecific_direct): Added; only defined when building for the iOS Simulator
1183         using the public SDK.
1184         (_pthread_getspecific_direct): Added; only defined when building for the iOS Simulator
1185         using the public SDK.
1186
1187 2014-08-12  Daniel Bates  <dabates@apple.com>
1188
1189         BPLATFORM(IOS) always evaluates to false
1190         https://bugs.webkit.org/show_bug.cgi?id=135843
1191
1192         Reviewed by Geoffrey Garen.
1193
1194         Fix typo in definition of BPLATFORM() and include system header TargetConditionals.h
1195         (when building on an Apple platform) so that BPLATFORM(X) evaluates to true when
1196         building for platform X. In particular, so that BPLATFORM(IOS) evaluates to true when
1197         building for iOS.
1198
1199         As a side effect of this change, the change made in <http://trac.webkit.org/changeset/167289>
1200         will be honored and iOS will assume a VM page size of 16kB (again) instead of 4kB.
1201
1202         * bmalloc/BPlatform.h:
1203
1204 2014-08-11  Andy Estes  <aestes@apple.com>
1205
1206         [iOS] Get rid of iOS.xcconfig
1207         https://bugs.webkit.org/show_bug.cgi?id=135809
1208
1209         Reviewed by Joseph Pecoraro.
1210
1211         All iOS.xcconfig did was include AspenFamily.xcconfig, so there's no need for the indirection.
1212
1213         * Configurations/Base.xcconfig:
1214         * Configurations/iOS.xcconfig: Removed.
1215         * bmalloc.xcodeproj/project.pbxproj:
1216
1217 2014-05-01  Dan Bernstein  <mitz@apple.com>
1218
1219         Fixed production builds for the iOS Simulator.
1220         <rdar://problem/16792221>
1221
1222         * Configurations/bmalloc.xcconfig: Include INSTALL_PATH_PREFIX in
1223         PRIVATE_HEADERS_FOLDER_PATH when installing.
1224
1225 2014-04-20  Geoffrey Garen  <ggaren@apple.com>
1226
1227         bmalloc: Segregate pages by objects size
1228         https://bugs.webkit.org/show_bug.cgi?id=131909
1229
1230         Reviewed by Andreas Kling.
1231
1232         2% reduction in memory-at-end on the Membuster memory_warning benchmarks.
1233
1234         * bmalloc/Allocator.cpp:
1235         (bmalloc::Allocator::allocateSlowCase):
1236         * bmalloc/Allocator.h:
1237         (bmalloc::Allocator::allocateFastCase):
1238         (bmalloc::Allocator::smallAllocatorFor): Use the new shared helper
1239         function for size class calculation.
1240
1241         * bmalloc/Deallocator.cpp:
1242         (bmalloc::Deallocator::Deallocator):
1243         (bmalloc::Deallocator::scavenge):
1244         (bmalloc::Deallocator::deallocateSmallLine):
1245         (bmalloc::Deallocator::allocateSmallLine):
1246         * bmalloc/Deallocator.h: Keep a cache for every size class, since the
1247         cache can't be shared anymore.
1248
1249         * bmalloc/Heap.cpp:
1250         (bmalloc::Heap::allocateSmallLineSlowCase):
1251         * bmalloc/Heap.h:
1252         (bmalloc::Heap::deallocateSmallLine): Ditto.
1253
1254         (bmalloc::Heap::allocateSmallLine): Check size class in addition to
1255         page refcount when allocating a line because we might have deallocated
1256         the page and the recycled it for another size class.
1257
1258         (bmalloc::Heap::deallocateMediumLine):
1259         (bmalloc::Heap::allocateMediumLine):
1260         * bmalloc/Line.h:
1261         (bmalloc::Line::refCount):
1262         * bmalloc/Page.h:
1263         (bmalloc::Page::refCount):
1264         (bmalloc::Page::smallSizeClass):
1265         (bmalloc::Page::setSmallSizeClass):
1266         (bmalloc::Page<Traits>::refCount): Deleted.
1267         * bmalloc/Sizes.h:
1268         (bmalloc::Sizes::smallSizeClassFor): New shared API for computing
1269         an index into an array from a size.
1270
1271 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1272
1273         bmalloc: Improved alignment in LargeChunk
1274         https://bugs.webkit.org/show_bug.cgi?id=131895
1275
1276         Reviewed by Andreas Kling.
1277
1278         * bmalloc/Chunk.h:
1279         * bmalloc/LargeChunk.h: Align to vmPageSize just like Chunk does.
1280         Technically, the previous alignment was harmless, but I would prefer,
1281         dear reader, not to have to explain the interlocking set of
1282         circumstances that made it so.
1283
1284 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1285
1286         Rolled out r167502 because it caused a crash on the facebook benchmark.
1287
1288         Unreviewed.
1289
1290             bmalloc: Added an XSmall line size
1291             https://bugs.webkit.org/show_bug.cgi?id=131851
1292
1293             Reviewed by Sam Weinig.
1294
1295 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1296
1297         bmalloc: Mutex should be harder to use wrong
1298         https://bugs.webkit.org/show_bug.cgi?id=131879
1299
1300         Reviewed by Andreas Kling.
1301
1302         Mutex now has a proper constructor, so you can't deadlock by forgetting
1303         to initialize it.
1304
1305         * bmalloc.xcodeproj/project.pbxproj:
1306         * bmalloc/Allocator.cpp:
1307         (bmalloc::Allocator::processXSmallAllocatorLog):
1308         (bmalloc::Allocator::processSmallAllocatorLog):
1309         (bmalloc::Allocator::processMediumAllocatorLog):
1310         (bmalloc::Allocator::allocateLarge):
1311         (bmalloc::Allocator::allocateXLarge): Global replace Mutex => StaticMutex,
1312         since the Heap mutex is a static.
1313
1314         * bmalloc/AsyncTask.h:
1315         (bmalloc::Function>::AsyncTask): Use Mutex, since we're not static. No
1316         need for explicit initialization anymore.
1317
1318         * bmalloc/Deallocator.cpp:
1319         (bmalloc::Deallocator::scavenge):
1320         (bmalloc::Deallocator::deallocateLarge):
1321         (bmalloc::Deallocator::deallocateXLarge):
1322         (bmalloc::Deallocator::processObjectLog):
1323         (bmalloc::Deallocator::deallocateSmallLine):
1324         (bmalloc::Deallocator::deallocateXSmallLine):
1325         (bmalloc::Deallocator::allocateSmallLine):
1326         (bmalloc::Deallocator::allocateXSmallLine):
1327         (bmalloc::Deallocator::deallocateMediumLine):
1328         (bmalloc::Deallocator::allocateMediumLine):
1329         * bmalloc/Deallocator.h:
1330         * bmalloc/Heap.cpp:
1331         (bmalloc::sleep):
1332         (bmalloc::Heap::Heap):
1333         (bmalloc::Heap::concurrentScavenge):
1334         (bmalloc::Heap::scavenge):
1335         (bmalloc::Heap::scavengeSmallPages):
1336         (bmalloc::Heap::scavengeXSmallPages):
1337         (bmalloc::Heap::scavengeMediumPages):
1338         (bmalloc::Heap::scavengeLargeRanges):
1339         (bmalloc::Heap::allocateXSmallLineSlowCase):
1340         (bmalloc::Heap::allocateSmallLineSlowCase):
1341         (bmalloc::Heap::allocateMediumLineSlowCase):
1342         (bmalloc::Heap::allocateXLarge):
1343         (bmalloc::Heap::deallocateXLarge):
1344         (bmalloc::Heap::allocateLarge):
1345         (bmalloc::Heap::deallocateLarge):
1346         * bmalloc/Heap.h:
1347         (bmalloc::Heap::deallocateXSmallLine):
1348         (bmalloc::Heap::allocateXSmallLine):
1349         (bmalloc::Heap::deallocateSmallLine):
1350         (bmalloc::Heap::allocateSmallLine):
1351         (bmalloc::Heap::deallocateMediumLine):
1352         (bmalloc::Heap::allocateMediumLine):
1353         * bmalloc/Line.h:
1354         (bmalloc::Line<Traits>::deref):
1355         * bmalloc/Mutex.cpp: Removed.
1356         * bmalloc/Mutex.h:
1357         (bmalloc::Mutex::Mutex):
1358         (bmalloc::Mutex::init): Deleted.
1359         (bmalloc::Mutex::try_lock): Deleted.
1360         (bmalloc::Mutex::lock): Deleted.
1361         (bmalloc::Mutex::unlock): Deleted.
1362         * bmalloc/Page.h:
1363         (bmalloc::Page<Traits>::ref):
1364         (bmalloc::Page<Traits>::deref):
1365         (bmalloc::Page<Traits>::refCount):
1366         * bmalloc/PerProcess.h:
1367         (bmalloc::PerProcess::mutex):
1368         (bmalloc::PerProcess<T>::getSlowCase):
1369         * bmalloc/StaticMutex.cpp: Added.
1370         (bmalloc::StaticMutex::lockSlowCase):
1371         * bmalloc/StaticMutex.h: Added.
1372         (bmalloc::StaticMutex::init):
1373         (bmalloc::StaticMutex::try_lock):
1374         (bmalloc::StaticMutex::lock):
1375         (bmalloc::StaticMutex::unlock):
1376         * bmalloc/VMHeap.h:
1377         (bmalloc::VMHeap::deallocateXSmallPage):
1378         (bmalloc::VMHeap::deallocateSmallPage):
1379         (bmalloc::VMHeap::deallocateMediumPage):
1380         (bmalloc::VMHeap::deallocateLargeRange):
1381         * bmalloc/bmalloc.h:
1382         (bmalloc::api::scavenge): Global replace Mutex => StaticMutex,
1383         since the Heap mutex is a static.
1384
1385 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
1386
1387         bmalloc: AsyncTask should use Mutex instead of std::mutex
1388         https://bugs.webkit.org/show_bug.cgi?id=131865
1389
1390         Reviewed by Gavin Barraclough.
1391
1392         std::mutex is so slow that it makes parallelizing simple tasks through
1393         AsyncTask a net regression. Mutex fixes this.
1394
1395         * bmalloc/AsyncTask.h:
1396         (bmalloc::Function>::AsyncTask):
1397         (bmalloc::Function>::join):
1398         (bmalloc::Function>::runSlowCase):
1399         (bmalloc::Function>::entryPoint):
1400         * bmalloc/Mutex.h:
1401         (bmalloc::Mutex::init):
1402
1403 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
1404
1405         bmalloc: Added an XSmall line size
1406         https://bugs.webkit.org/show_bug.cgi?id=131851
1407
1408         Reviewed by Sam Weinig.
1409
1410         Reduces malloc footprint on Membuster recordings by 10%.
1411
1412         This is a throughput regression, but we're still way ahead of TCMalloc.
1413         I have some ideas for how to recover the regression -- but I wanted to
1414         get this win in first.
1415
1416         Full set of benchmark results:
1417
1418                 bmalloc> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks --measure-heap nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1419                                                                                                 
1420                                                                nopatch                      patch                                Δ
1421                 Peak Memory:
1422                     reddit_memory_warning                      7,896kB                    7,532kB                  ^ 1.05x smaller
1423                     flickr_memory_warning                     12,968kB                   12,324kB                  ^ 1.05x smaller
1424                     theverge_memory_warning                   16,672kB                   15,200kB                   ^ 1.1x smaller
1425
1426                     <geometric mean>                          11,952kB                   11,216kB                  ^ 1.07x smaller
1427                     <arithmetic mean>                         12,512kB                   11,685kB                  ^ 1.07x smaller
1428                     <harmonic mean>                           11,375kB                   10,726kB                  ^ 1.06x smaller
1429
1430                 Memory at End:
1431                     reddit_memory_warning                      7,320kB                    6,856kB                  ^ 1.07x smaller
1432                     flickr_memory_warning                     10,848kB                    9,692kB                  ^ 1.12x smaller
1433                     theverge_memory_warning                   16,380kB                   14,872kB                   ^ 1.1x smaller
1434
1435                     <geometric mean>                          10,916kB                    9,961kB                   ^ 1.1x smaller
1436                     <arithmetic mean>                         11,516kB                   10,473kB                   ^ 1.1x smaller
1437                     <harmonic mean>                           10,350kB                    9,485kB                  ^ 1.09x smaller
1438
1439                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1440                                                                                                 
1441                                                    nopatch                patch                         Δ
1442                 Execution Time:
1443                     churn                            127ms                151ms            ! 1.19x slower
1444                     list_allocate                    130ms                164ms            ! 1.26x slower
1445                     tree_allocate                    109ms                127ms            ! 1.17x slower
1446                     tree_churn                       115ms                120ms            ! 1.04x slower
1447                     facebook                         240ms                259ms            ! 1.08x slower
1448                     fragment                          91ms                131ms            ! 1.44x slower
1449                     fragment_iterate                 105ms                106ms            ! 1.01x slower
1450                     message_one                      260ms                259ms             ^ 1.0x faster
1451                     message_many                     149ms                154ms            ! 1.03x slower
1452                     medium                           194ms                248ms            ! 1.28x slower
1453                     big                              157ms                160ms            ! 1.02x slower
1454
1455                     <geometric mean>                 144ms                163ms            ! 1.13x slower
1456                     <arithmetic mean>                152ms                171ms            ! 1.12x slower
1457                     <harmonic mean>                  137ms                156ms            ! 1.14x slower
1458
1459                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1460                                                                                                 
1461                                                                        nopatch                          patch                                     Δ
1462                 Execution Time:
1463                     churn                                                126ms                          148ms                        ! 1.17x slower
1464                     churn --parallel                                      62ms                           76ms                        ! 1.23x slower
1465                     list_allocate                                        130ms                          164ms                        ! 1.26x slower
1466                     list_allocate --parallel                             120ms                          175ms                        ! 1.46x slower
1467                     tree_allocate                                        111ms                          127ms                        ! 1.14x slower
1468                     tree_allocate --parallel                              95ms                          135ms                        ! 1.42x slower
1469                     tree_churn                                           115ms                          124ms                        ! 1.08x slower
1470                     tree_churn --parallel                                107ms                          126ms                        ! 1.18x slower
1471                     facebook                                             240ms                          276ms                        ! 1.15x slower
1472                     facebook --parallel                                  802ms                        1,088ms                        ! 1.36x slower
1473                     fragment                                              92ms                          130ms                        ! 1.41x slower
1474                     fragment --parallel                                   66ms                          124ms                        ! 1.88x slower
1475                     fragment_iterate                                     109ms                          127ms                        ! 1.17x slower
1476                     fragment_iterate --parallel                           55ms                           64ms                        ! 1.16x slower
1477                     message_one                                          260ms                          260ms                                      
1478                     message_many                                         170ms                          238ms                         ! 1.4x slower
1479                     medium                                               185ms                          250ms                        ! 1.35x slower
1480                     medium --parallel                                    210ms                          334ms                        ! 1.59x slower
1481                     big                                                  150ms                          169ms                        ! 1.13x slower
1482                     big --parallel                                       138ms                          144ms                        ! 1.04x slower
1483
1484                     <geometric mean>                                     135ms                          170ms                        ! 1.26x slower
1485                     <arithmetic mean>                                    167ms                          214ms                        ! 1.28x slower
1486                     <harmonic mean>                                      117ms                          148ms                        ! 1.26x slower
1487
1488                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
1489
1490                                                                     TC                      patch                                Δ
1491                 Peak Memory:
1492                     reddit_memory_warning                     13,836kB                   13,436kB                  ^ 1.03x smaller
1493                     flickr_memory_warning                     24,868kB                   25,188kB                   ! 1.01x bigger
1494                     theverge_memory_warning                   24,504kB                   26,636kB                   ! 1.09x bigger
1495
1496                     <geometric mean>                          20,353kB                   20,812kB                   ! 1.02x bigger
1497                     <arithmetic mean>                         21,069kB                   21,753kB                   ! 1.03x bigger
1498                     <harmonic mean>                           19,570kB                   19,780kB                   ! 1.01x bigger
1499
1500                 Memory at End:
1501                     reddit_memory_warning                      8,656kB                   10,016kB                   ! 1.16x bigger
1502                     flickr_memory_warning                     11,844kB                   13,784kB                   ! 1.16x bigger
1503                     theverge_memory_warning                   18,516kB                   22,748kB                   ! 1.23x bigger
1504
1505                     <geometric mean>                          12,382kB                   14,644kB                   ! 1.18x bigger
1506                     <arithmetic mean>                         13,005kB                   15,516kB                   ! 1.19x bigger
1507                     <harmonic mean>                           11,813kB                   13,867kB                   ! 1.17x bigger
1508
1509                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
1510                                                                                                 
1511                                                         TC                patch                         Δ
1512                 Execution Time:
1513                     churn                            416ms                148ms            ^ 2.81x faster
1514                     list_allocate                    463ms                164ms            ^ 2.82x faster
1515                     tree_allocate                    292ms                127ms             ^ 2.3x faster
1516                     tree_churn                       157ms                120ms            ^ 1.31x faster
1517                     facebook                         327ms                276ms            ^ 1.18x faster
1518                     fragment                         335ms                129ms             ^ 2.6x faster
1519                     fragment_iterate                 344ms                108ms            ^ 3.19x faster
1520                     message_one                      386ms                258ms             ^ 1.5x faster
1521                     message_many                     410ms                154ms            ^ 2.66x faster
1522                     medium                           391ms                245ms             ^ 1.6x faster
1523                     big                              261ms                167ms            ^ 1.56x faster
1524
1525                     <geometric mean>                 332ms                164ms            ^ 2.02x faster
1526                     <arithmetic mean>                344ms                172ms            ^ 1.99x faster
1527                     <harmonic mean>                  317ms                157ms            ^ 2.02x faster
1528
1529         * bmalloc.xcodeproj/project.pbxproj:
1530         * bmalloc/Allocator.cpp:
1531         (bmalloc::Allocator::Allocator): Don't assume that each allocator's
1532         index corresponds with its size. Instead, use the size selection function
1533         explicitly. Now that we have XSmall, some small allocator entries are
1534         unused.
1535
1536         (bmalloc::Allocator::scavenge):
1537         (bmalloc::Allocator::log):
1538         (bmalloc::Allocator::processXSmallAllocatorLog):
1539         (bmalloc::Allocator::allocateSlowCase):
1540         * bmalloc/Allocator.h:
1541         (bmalloc::Allocator::xSmallAllocatorFor):
1542         (bmalloc::Allocator::allocateFastCase):
1543         * bmalloc/Chunk.h:
1544         * bmalloc/Deallocator.cpp:
1545         (bmalloc::Deallocator::scavenge):
1546         (bmalloc::Deallocator::processObjectLog):
1547         (bmalloc::Deallocator::deallocateSlowCase):
1548         (bmalloc::Deallocator::deallocateXSmallLine):
1549         (bmalloc::Deallocator::allocateXSmallLine):
1550         * bmalloc/Deallocator.h:
1551         (bmalloc::Deallocator::deallocateFastCase):
1552         * bmalloc/Heap.cpp:
1553         (bmalloc::Heap::scavenge):
1554         (bmalloc::Heap::scavengeXSmallPages):
1555         (bmalloc::Heap::allocateXSmallLineSlowCase):
1556         * bmalloc/Heap.h:
1557         (bmalloc::Heap::deallocateXSmallLine):
1558         (bmalloc::Heap::allocateXSmallLine):
1559         * bmalloc/LargeChunk.h:
1560         (bmalloc::LargeChunk::get):
1561         (bmalloc::LargeChunk::endTag):
1562         * bmalloc/Line.h:
1563         * bmalloc/MediumAllocator.h:
1564         (bmalloc::MediumAllocator::allocate):
1565         (bmalloc::MediumAllocator::refill):
1566         * bmalloc/ObjectType.cpp:
1567         (bmalloc::objectType):
1568         * bmalloc/ObjectType.h:
1569         (bmalloc::isXSmall):
1570         (bmalloc::isSmall):
1571         (bmalloc::isMedium):
1572         (bmalloc::isLarge):
1573         (bmalloc::isSmallOrMedium): Deleted.
1574         * bmalloc/SegregatedFreeList.h: I boiler-plate copied existing code for
1575         handling small objects. There's probably a reasonable way to share this
1576         code in the future -- I'll look into that once it's stopped changing.
1577
1578         * bmalloc/Sizes.h: Tweaked size classes to make Membuster happy. This
1579         is the main reason things got slower.
1580
1581         * bmalloc/SmallAllocator.h:
1582         (bmalloc::SmallAllocator::allocate):
1583         * bmalloc/SmallTraits.h:
1584         * bmalloc/VMHeap.cpp:
1585         (bmalloc::VMHeap::allocateXSmallChunk):
1586         * bmalloc/VMHeap.h:
1587         (bmalloc::VMHeap::allocateXSmallPage):
1588         (bmalloc::VMHeap::deallocateXSmallPage):
1589         * bmalloc/XSmallAllocator.h: Added.
1590         (bmalloc::XSmallAllocator::isNull):
1591         (bmalloc::XSmallAllocator::canAllocate):
1592         (bmalloc::XSmallAllocator::XSmallAllocator):
1593         (bmalloc::XSmallAllocator::line):
1594         (bmalloc::XSmallAllocator::allocate):
1595         (bmalloc::XSmallAllocator::objectCount):
1596         (bmalloc::XSmallAllocator::derefCount):
1597         (bmalloc::XSmallAllocator::refill):
1598         * bmalloc/XSmallChunk.h: Added.
1599         * bmalloc/XSmallLine.h: Added.
1600         * bmalloc/XSmallPage.h: Added.
1601         * bmalloc/XSmallTraits.h: Added.
1602         * bmalloc/bmalloc.h:
1603         (bmalloc::api::realloc): Boiler-plate copy, as above.
1604
1605 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
1606
1607         MallocBench should scavenge explicitly instead of waiting
1608         https://bugs.webkit.org/show_bug.cgi?id=131661
1609
1610         Reviewed by Andreas Kling.
1611
1612         Added explicit scavenge support to bmalloc. This isn't a memory win,
1613         since bmalloc's per-thread cache is so small. But it makes testing
1614         simpler.
1615
1616         * bmalloc/Allocator.cpp:
1617         (bmalloc::Allocator::~Allocator):
1618         (bmalloc::Allocator::scavenge):
1619         * bmalloc/Allocator.h:
1620         * bmalloc/Cache.cpp:
1621         (bmalloc::Cache::operator new):
1622         (bmalloc::Cache::operator delete):
1623         (bmalloc::Cache::Cache):
1624         (bmalloc::Cache::scavenge):
1625         * bmalloc/Cache.h:
1626         * bmalloc/Deallocator.cpp:
1627         (bmalloc::Deallocator::~Deallocator):
1628         (bmalloc::Deallocator::scavenge):
1629         * bmalloc/Deallocator.h: Factored existing scavenging code into helper
1630         functions, for reuse.
1631
1632         * bmalloc/Heap.cpp:
1633         (bmalloc::sleep):
1634         (bmalloc::Heap::concurrentScavenge):
1635         (bmalloc::Heap::scavenge):
1636         (bmalloc::Heap::scavengeSmallPages):
1637         (bmalloc::Heap::scavengeMediumPages):
1638         (bmalloc::Heap::scavengeLargeRanges):
1639         * bmalloc/Heap.h: Made scavenge sleep duration a parameter. Forced
1640         scavenging -- in response to a benchmark or a low memory warning --
1641         wants to complete as soon as possible, so its sleep duration is 0.
1642
1643         * bmalloc/bmalloc.h:
1644         (bmalloc::api::scavenge):
1645         * bmalloc/mbmalloc.cpp: Exported the scavenge API for MallocBench's use.
1646
1647 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
1648
1649         Use 4kB pages on Mac
1650         https://bugs.webkit.org/show_bug.cgi?id=131658
1651
1652         Reviewed by Sam Weinig.
1653
1654         This reduces memory use a lot on Membuster:
1655
1656                                                                   base                      patch                                Δ
1657                 Execution Time:
1658                     reddit_memory_warning                         18ms                       17ms                   ^ 1.06x faster
1659                     flickr_memory_warning                         34ms                       36ms                   ! 1.06x slower
1660                     theverge_memory_warning                       39ms                       41ms                   ! 1.05x slower
1661
1662                     <geometric mean>                              29ms                       29ms                   ! 1.02x slower
1663                     <arithmetic mean>                             30ms                       31ms                   ! 1.03x slower
1664                     <harmonic mean>                               27ms                       27ms                    ^ 1.0x faster
1665
1666                 Peak Memory:
1667                     reddit_memory_warning                     16,412kB                   16,436kB                    ! 1.0x bigger
1668                     flickr_memory_warning                     30,120kB                   30,184kB                    ! 1.0x bigger
1669                     theverge_memory_warning                   33,408kB                   33,420kB                    ! 1.0x bigger
1670
1671                     <geometric mean>                          25,466kB                   25,499kB                    ! 1.0x bigger
1672                     <arithmetic mean>                         26,647kB                   26,680kB                    ! 1.0x bigger
1673                     <harmonic mean>                           24,181kB                   24,214kB                    ! 1.0x bigger
1674
1675                 Memory at End:
1676                     reddit_memory_warning                      2,404kB                    1,920kB                  ^ 1.25x smaller
1677                     flickr_memory_warning                      3,764kB                    3,072kB                  ^ 1.23x smaller
1678                     theverge_memory_warning                    3,648kB                    3,132kB                  ^ 1.16x smaller
1679
1680                     <geometric mean>                           3,208kB                    2,644kB                  ^ 1.21x smaller
1681                     <arithmetic mean>                          3,272kB                    2,708kB                  ^ 1.21x smaller
1682                     <harmonic mean>                            3,139kB                    2,574kB                  ^ 1.22x smaller
1683
1684
1685         * bmalloc.xcodeproj/project.pbxproj:
1686         * bmalloc/BPlatform.h: Added.
1687         * bmalloc/VMAllocate.h: Only use 16kB pages on iOS because the page size
1688         is 4kB on Mac.
1689
1690 2014-04-14  Alexey Proskuryakov  <ap@apple.com>
1691
1692         Fixed svn:ignore on bmalloc.xcodeproj, it had erroneous leading spaces.
1693
1694         * bmalloc.xcodeproj: Modified property svn:ignore.
1695
1696 2014-04-13  Geoffrey Garen  <ggaren@apple.com>
1697
1698         Fixed some mbmalloc exports
1699         https://bugs.webkit.org/show_bug.cgi?id=131599
1700
1701         Reviewed by Ryosuke Niwa.
1702
1703         * bmalloc.xcodeproj/project.pbxproj: Made some headers a private part
1704         of the project, so we can call them from API.
1705
1706         * bmalloc/mbmalloc.cpp: Marked the mbmalloc functions with default
1707         visibility, so they show up as exported in the .dylib.
1708
1709 2014-04-09  Geoffrey Garen  <ggaren@apple.com>
1710
1711         Put bmalloc headers in the right place
1712         https://bugs.webkit.org/show_bug.cgi?id=131464
1713
1714         Reviewed by Mark Rowe.
1715
1716         * Configurations/bmalloc.xcconfig: Set PRIVATE_HEADERS_FOLDER_PATH to
1717         specify that we don't just want to dump all of our generically-named
1718         headers into /usr/local/include.
1719
1720 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
1721
1722         Made bmalloc more #include friendly
1723         https://bugs.webkit.org/show_bug.cgi?id=131386
1724
1725         Reviewed by Andreas Kling.
1726
1727         Marked a bunch of headers private so they can be used from client code
1728         that #includes bmalloc.h.
1729
1730         Renamed ASSERT macros to BASSERT. This matches their header, which already
1731         had to be renamed, and fixes conflicts with WTF's ASSERT macros.
1732
1733         * bmalloc.xcodeproj/project.pbxproj:
1734         * bmalloc/Allocator.cpp:
1735         (bmalloc::Allocator::allocateSlowCase):
1736         * bmalloc/AsyncTask.h:
1737         (bmalloc::Function>::runSlowCase):
1738         * bmalloc/BAssert.h:
1739         * bmalloc/BoundaryTag.h:
1740         (bmalloc::BoundaryTag::setSize):
1741         * bmalloc/BoundaryTagInlines.h:
1742         (bmalloc::validate):
1743         (bmalloc::BoundaryTag::init):
1744         (bmalloc::BoundaryTag::deallocate):
1745         (bmalloc::BoundaryTag::splitLarge):
1746         (bmalloc::BoundaryTag::allocate):
1747         * bmalloc/Chunk.h:
1748         * bmalloc/Deallocator.cpp:
1749         (bmalloc::Deallocator::processObjectLog):
1750         (bmalloc::Deallocator::deallocateSlowCase):
1751         * bmalloc/Deallocator.h:
1752         (bmalloc::Deallocator::deallocateFastCase):
1753         * bmalloc/FixedVector.h:
1754         (bmalloc::Capacity>::operator):
1755         (bmalloc::Capacity>::push):
1756         (bmalloc::Capacity>::pop):
1757         (bmalloc::Capacity>::shrink):
1758         * bmalloc/Heap.cpp:
1759         (bmalloc::Heap::allocateLarge):
1760         * bmalloc/LargeChunk.h:
1761         (bmalloc::LargeChunk::get):
1762         (bmalloc::LargeChunk::endTag):
1763         * bmalloc/Line.h:
1764         (bmalloc::Line<Traits>::concurrentRef):
1765         (bmalloc::Line<Traits>::deref):
1766         * bmalloc/MediumAllocator.h:
1767         (bmalloc::MediumAllocator::allocate):
1768         * bmalloc/ObjectType.h:
1769         (bmalloc::isSmall):
1770         * bmalloc/Page.h:
1771         (bmalloc::Page<Traits>::ref):
1772         (bmalloc::Page<Traits>::deref):
1773         * bmalloc/PerThread.h:
1774         (bmalloc::PerThread<T>::getSlowCase):
1775         * bmalloc/SegregatedFreeList.cpp:
1776         (bmalloc::SegregatedFreeList::SegregatedFreeList):
1777         (bmalloc::SegregatedFreeList::insert):
1778         * bmalloc/SmallAllocator.h:
1779         (bmalloc::SmallAllocator::allocate):
1780         (bmalloc::SmallAllocator::refill):
1781         * bmalloc/Syscall.h:
1782         * bmalloc/VMAllocate.h:
1783         (bmalloc::vmValidate):
1784         (bmalloc::vmAllocate):
1785         (bmalloc::vmDeallocatePhysicalPagesSloppy):
1786         * bmalloc/Vector.h:
1787         (bmalloc::Vector<T>::operator):
1788         (bmalloc::Vector<T>::pop):
1789         (bmalloc::Vector<T>::shrink):
1790         * bmalloc/XLargeChunk.h:
1791         (bmalloc::XLargeChunk::range):
1792         (bmalloc::XLargeChunk::size):
1793
1794 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
1795
1796         Removed an unused file.
1797
1798         Unreviewed.
1799
1800         * bmalloc/AsyncTask.cpp: Removed.
1801
1802 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1803
1804         Build bmalloc on Mac
1805         https://bugs.webkit.org/show_bug.cgi?id=131333
1806
1807         Reviewed by Mark Rowe.
1808
1809         * Makefile: Added. For make clients.
1810
1811         These files are required for building any project in WebKit. I copied
1812         them from WTF:
1813         * Configurations: Added.
1814         * Configurations/Base.xcconfig: Added.
1815         * Configurations/DebugRelease.xcconfig: Added.
1816         * Configurations/bmalloc.xcconfig: Added.
1817         * Configurations/iOS.xcconfig: Added.
1818         * Configurations/mbmalloc.xcconfig: Added.
1819
1820         * bmalloc.xcodeproj/project.pbxproj: I removed per-project-file stuff
1821         from here because everything is in .xcconfig files now.
1822
1823         I had to fix a bunch of minor warnings, since they're enabled in our
1824         .xcconfig files:
1825
1826         * bmalloc/AsyncTask.h:
1827         (bmalloc::Function>::AsyncTask):
1828         * bmalloc/BAssert.h:
1829         * bmalloc/BoundaryTagInlines.h:
1830         (bmalloc::validate):
1831         * bmalloc/Heap.cpp:
1832         (bmalloc::Heap::Heap):
1833         (bmalloc::Heap::allocateLarge):
1834         (bmalloc::Heap::deallocateLarge):
1835         * bmalloc/Mutex.h:
1836         (bmalloc::Mutex::Mutex): Deleted.
1837         * bmalloc/VMAllocate.h:
1838         (bmalloc::vmValidate):
1839         * bmalloc/mbmalloc.cpp:
1840
1841 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1842
1843         bmalloc: Fixed a leak in the per-thread cache
1844         https://bugs.webkit.org/show_bug.cgi?id=131330
1845
1846         Reviewed by Andreas Kling.
1847
1848         Remember to deallocate our line caches upon thread exit.
1849
1850         * bmalloc/Deallocator.cpp:
1851         (bmalloc::Deallocator::~Deallocator):
1852
1853 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1854
1855         bmalloc: rolled out the tryLock experiment
1856         https://bugs.webkit.org/show_bug.cgi?id=131328
1857
1858         Reviewed by Andreas Kling.
1859
1860         It wasn't a speedup.
1861
1862         * bmalloc.xcodeproj/project.pbxproj:
1863         * bmalloc/Allocator.cpp:
1864         (bmalloc::Allocator::processSmallAllocatorLog):
1865         (bmalloc::Allocator::processMediumAllocatorLog):
1866         * bmalloc/Deallocator.cpp:
1867         (bmalloc::Deallocator::processObjectLog):
1868         (bmalloc::Deallocator::deallocateSlowCase):
1869         (bmalloc::Deallocator::deallocateSmallLine):
1870         (bmalloc::Deallocator::deallocateMediumLine):
1871         * bmalloc/Deallocator.h:
1872         (bmalloc::Deallocator::deallocateFastCase):
1873         * bmalloc/Heap.h:
1874         (bmalloc::Heap::deallocateSmallLine):
1875         (bmalloc::Heap::deallocateMediumLine):
1876         * bmalloc/Line.h:
1877         (bmalloc::Line<Traits>::deref):
1878         * bmalloc/Page.h:
1879         (bmalloc::Page<Traits>::deref):
1880
1881 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1882
1883         bmalloc
1884         https://bugs.webkit.org/show_bug.cgi?id=131170
1885
1886         Reviewed by Andreas Kling.
1887
1888         Initial commit.
1889
1890         * bmalloc: Added.
1891         * bmalloc.xcodeproj: Added.
1892         * bmalloc.xcodeproj/project.pbxproj: Added.
1893         * bmalloc/Algorithm.h: Added.
1894         (bmalloc::max):
1895         (bmalloc::min):
1896         (bmalloc::mask):
1897         (bmalloc::test):
1898         (bmalloc::roundUpToMultipleOf):
1899         (bmalloc::roundDownToMultipleOf):
1900         (bmalloc::sizeOf):
1901         (bmalloc::bitCount):
1902         (bmalloc::isPowerOfTwo):
1903         * bmalloc/Allocator.cpp: Added.
1904         (bmalloc::Allocator::Allocator):
1905         (bmalloc::Allocator::~Allocator):
1906         (bmalloc::Allocator::log):
1907         (bmalloc::Allocator::processSmallAllocatorLog):
1908         (bmalloc::Allocator::processMediumAllocatorLog):
1909         (bmalloc::Allocator::allocateLarge):
1910         (bmalloc::Allocator::allocateXLarge):
1911         (bmalloc::Allocator::allocateMedium):
1912         (bmalloc::Allocator::allocateSlowCase):
1913         * bmalloc/Allocator.h: Added.
1914         (bmalloc::Allocator::smallAllocatorFor):
1915         (bmalloc::Allocator::allocateFastCase):
1916         (bmalloc::Allocator::allocate):
1917         * bmalloc/AsyncTask.cpp: Added.
1918         (bmalloc::AsyncTask<Function>::runSlowCase):
1919         (bmalloc::AsyncTask<Function>::pthreadEntryPoint):
1920         (bmalloc::AsyncTask<Function>::entryPoint):
1921         * bmalloc/AsyncTask.h: Added.
1922         (bmalloc::Function>::AsyncTask):
1923         (bmalloc::Function>::join):
1924         (bmalloc::Function>::run):
1925         (bmalloc::Function>::runSlowCase):
1926         (bmalloc::Function>::pthreadEntryPoint):
1927         (bmalloc::Function>::entryPoint):
1928         * bmalloc/BAssert.h: Added.
1929         * bmalloc/BeginTag.h: Added.
1930         (bmalloc::BeginTag::isInFreeList):
1931         * bmalloc/BoundaryTag.h: Added.
1932         (bmalloc::BoundaryTag::isXLarge):
1933         (bmalloc::BoundaryTag::setXLarge):
1934         (bmalloc::BoundaryTag::isFree):
1935         (bmalloc::BoundaryTag::setFree):
1936         (bmalloc::BoundaryTag::isEnd):
1937         (bmalloc::BoundaryTag::setEnd):
1938         (bmalloc::BoundaryTag::hasPhysicalPages):
1939         (bmalloc::BoundaryTag::setHasPhysicalPages):
1940         (bmalloc::BoundaryTag::isNull):
1941         (bmalloc::BoundaryTag::clear):
1942         (bmalloc::BoundaryTag::size):
1943         (bmalloc::BoundaryTag::setSize):
1944         (bmalloc::BoundaryTag::prev):
1945         (bmalloc::BoundaryTag::next):
1946         * bmalloc/BoundaryTagInlines.h: Added.
1947         (bmalloc::validate):
1948         (bmalloc::validatePrev):
1949         (bmalloc::validateNext):
1950         (bmalloc::BoundaryTag::init):
1951         (bmalloc::BoundaryTag::mergeLargeLeft):
1952         (bmalloc::BoundaryTag::mergeLargeRight):
1953         (bmalloc::BoundaryTag::mergeLarge):
1954         (bmalloc::BoundaryTag::deallocate):
1955         (bmalloc::BoundaryTag::splitLarge):
1956         (bmalloc::BoundaryTag::allocate):
1957         * bmalloc/Cache.cpp: Added.
1958         (bmalloc::Cache::operator new):
1959         (bmalloc::Cache::operator delete):
1960         (bmalloc::Cache::Cache):
1961         (bmalloc::Cache::allocateSlowCase):
1962         (bmalloc::Cache::allocateSlowCaseNullCache):
1963         (bmalloc::Cache::deallocateSlowCase):
1964         (bmalloc::Cache::deallocateSlowCaseNullCache):
1965         * bmalloc/Cache.h: Added.
1966         (bmalloc::Cache::allocator):
1967         (bmalloc::Cache::deallocator):
1968         (bmalloc::Cache::allocateFastCase):
1969         (bmalloc::Cache::deallocateFastCase):
1970         (bmalloc::Cache::allocate):
1971         (bmalloc::Cache::deallocate):
1972         * bmalloc/Chunk.h: Added.
1973         (bmalloc::Chunk::begin):
1974         (bmalloc::Chunk::end):
1975         (bmalloc::Chunk::lines):
1976         (bmalloc::Chunk::pages):
1977         * bmalloc/Deallocator.cpp: Added.
1978         (bmalloc::Deallocator::Deallocator):
1979         (bmalloc::Deallocator::~Deallocator):
1980         (bmalloc::Deallocator::deallocateLarge):
1981         (bmalloc::Deallocator::deallocateXLarge):
1982         (bmalloc::Deallocator::processObjectLog):
1983         (bmalloc::Deallocator::deallocateSlowCase):
1984         (bmalloc::Deallocator::deallocateSmallLine):
1985         (bmalloc::Deallocator::allocateSmallLine):
1986         (bmalloc::Deallocator::deallocateMediumLine):
1987         (bmalloc::Deallocator::allocateMediumLine):
1988         * bmalloc/Deallocator.h: Added.
1989         (bmalloc::Deallocator::deallocateFastCase):
1990         (bmalloc::Deallocator::deallocate):
1991         * bmalloc/EndTag.h: Added.
1992         (bmalloc::EndTag::operator=):
1993         * bmalloc/FixedVector.h: Added.
1994         (bmalloc::FixedVector::begin):
1995         (bmalloc::FixedVector::end):
1996         (bmalloc::FixedVector::size):
1997         (bmalloc::FixedVector::capacity):
1998         (bmalloc::FixedVector::clear):
1999         (bmalloc::FixedVector::isEmpty):
2000         (bmalloc::Capacity>::FixedVector):
2001         (bmalloc::Capacity>::operator):
2002         (bmalloc::Capacity>::push):
2003         (bmalloc::Capacity>::pop):
2004         (bmalloc::Capacity>::shrink):
2005         * bmalloc/Heap.cpp: Added.
2006         (bmalloc::sleep):
2007         (bmalloc::Heap::Heap):
2008         (bmalloc::Heap::concurrentScavenge):
2009         (bmalloc::Heap::scavengeSmallPages):
2010         (bmalloc::Heap::scavengeMediumPages):
2011         (bmalloc::Heap::scavengeLargeRanges):
2012         (bmalloc::Heap::allocateSmallLineSlowCase):
2013         (bmalloc::Heap::allocateMediumLineSlowCase):
2014         (bmalloc::Heap::allocateXLarge):
2015         (bmalloc::Heap::deallocateXLarge):
2016         (bmalloc::Heap::allocateLarge):
2017         (bmalloc::Heap::deallocateLarge):
2018         * bmalloc/Heap.h: Added.
2019         (bmalloc::Heap::deallocateSmallLine):
2020         (bmalloc::Heap::allocateSmallLine):
2021         (bmalloc::Heap::deallocateMediumLine):
2022         (bmalloc::Heap::allocateMediumLine):
2023         * bmalloc/Inline.h: Added.
2024         * bmalloc/LargeChunk.h: Added.
2025         (bmalloc::LargeChunk::begin):
2026         (bmalloc::LargeChunk::end):
2027         (bmalloc::LargeChunk::create):
2028         (bmalloc::LargeChunk::get):
2029         (bmalloc::LargeChunk::beginTag):
2030         (bmalloc::LargeChunk::endTag):
2031         * bmalloc/Line.h: Added.
2032         (bmalloc::Line<Traits>::begin):
2033         (bmalloc::Line<Traits>::end):
2034         (bmalloc::Line<Traits>::concurrentRef):
2035         (bmalloc::Line<Traits>::deref):
2036         * bmalloc/MediumAllocator.h: Added.
2037         (bmalloc::MediumAllocator::isNull):
2038         (bmalloc::MediumAllocator::MediumAllocator):
2039         (bmalloc::MediumAllocator::line):
2040         (bmalloc::MediumAllocator::allocate):
2041         (bmalloc::MediumAllocator::derefCount):
2042         (bmalloc::MediumAllocator::refill):
2043         * bmalloc/MediumChunk.h: Added.
2044         * bmalloc/MediumLine.h: Added.
2045         * bmalloc/MediumPage.h: Added.
2046         * bmalloc/MediumTraits.h: Added.
2047         * bmalloc/Mutex.cpp: Added.
2048         (bmalloc::Mutex::lockSlowCase):
2049         * bmalloc/Mutex.h: Added.
2050         (bmalloc::Mutex::Mutex):
2051         (bmalloc::Mutex::try_lock):
2052         (bmalloc::Mutex::lock):
2053         (bmalloc::Mutex::unlock):
2054         * bmalloc/ObjectType.cpp: Added.
2055         (bmalloc::objectType):
2056         * bmalloc/ObjectType.h: Added.
2057         (bmalloc::isSmallOrMedium):
2058         (bmalloc::isSmall):
2059         * bmalloc/Page.h: Added.
2060         (bmalloc::Page<Traits>::ref):
2061         (bmalloc::Page<Traits>::deref):
2062         (bmalloc::Page<Traits>::refCount):
2063         * bmalloc/PerProcess.h: Added.
2064         (bmalloc::PerProcess::mutex):
2065         (bmalloc::PerProcess<T>::getFastCase):
2066         (bmalloc::PerProcess<T>::get):
2067         (bmalloc::PerProcess<T>::getSlowCase):
2068         * bmalloc/PerThread.h: Added.
2069         (bmalloc::PerThreadStorage<Cache>::get):
2070         (bmalloc::PerThreadStorage<Cache>::init):
2071         (bmalloc::PerThreadStorage::get):
2072         (bmalloc::PerThreadStorage::init):
2073         (bmalloc::PerThread<T>::getFastCase):
2074         (bmalloc::PerThread<T>::get):
2075         (bmalloc::PerThread<T>::destructor):
2076         (bmalloc::PerThread<T>::getSlowCase):
2077         * bmalloc/Range.h: Added.
2078         (bmalloc::Range::Range):
2079         (bmalloc::Range::begin):
2080         (bmalloc::Range::end):
2081         (bmalloc::Range::size):
2082         (bmalloc::Range::operator!):
2083         (bmalloc::Range::operator<):
2084         * bmalloc/SegregatedFreeList.cpp: Added.
2085         (bmalloc::SegregatedFreeList::SegregatedFreeList):
2086         (bmalloc::SegregatedFreeList::insert):
2087         (bmalloc::SegregatedFreeList::takeGreedy):
2088         (bmalloc::SegregatedFreeList::take):
2089         * bmalloc/SegregatedFreeList.h: Added.
2090         * bmalloc/Sizes.h: Added.
2091         * bmalloc/SmallAllocator.h: Added.
2092         (bmalloc::SmallAllocator::isNull):
2093         (bmalloc::SmallAllocator::canAllocate):
2094         (bmalloc::SmallAllocator::SmallAllocator):
2095         (bmalloc::SmallAllocator::line):
2096         (bmalloc::SmallAllocator::allocate):
2097         (bmalloc::SmallAllocator::objectCount):
2098         (bmalloc::SmallAllocator::derefCount):
2099         (bmalloc::SmallAllocator::refill):
2100         * bmalloc/SmallChunk.h: Added.
2101         * bmalloc/SmallLine.h: Added.
2102         * bmalloc/SmallPage.h: Added.
2103         * bmalloc/SmallTraits.h: Added.
2104         * bmalloc/Syscall.h: Added.
2105         * bmalloc/VMAllocate.h: Added.
2106         (bmalloc::vmSize):
2107         (bmalloc::vmValidate):
2108         (bmalloc::vmAllocate):
2109         (bmalloc::vmDeallocate):
2110         (bmalloc::vmDeallocatePhysicalPages):
2111         (bmalloc::vmAllocatePhysicalPages):
2112         (bmalloc::vmDeallocatePhysicalPagesSloppy):
2113         (bmalloc::vmAllocatePhysicalPagesSloppy):
2114         * bmalloc/VMHeap.cpp: Added.
2115         (bmalloc::VMHeap::VMHeap):
2116         (bmalloc::VMHeap::allocateSmallChunk):
2117         (bmalloc::VMHeap::allocateMediumChunk):
2118         (bmalloc::VMHeap::allocateLargeChunk):
2119         * bmalloc/VMHeap.h: Added.
2120         (bmalloc::VMHeap::allocateSmallPage):
2121         (bmalloc::VMHeap::allocateMediumPage):
2122         (bmalloc::VMHeap::allocateLargeRange):
2123         (bmalloc::VMHeap::deallocateSmallPage):
2124         (bmalloc::VMHeap::deallocateMediumPage):
2125         (bmalloc::VMHeap::deallocateLargeRange):
2126         * bmalloc/Vector.h: Added.
2127         (bmalloc::Vector::begin):
2128         (bmalloc::Vector::end):
2129         (bmalloc::Vector::size):
2130         (bmalloc::Vector::capacity):
2131         (bmalloc::Vector::last):
2132         (bmalloc::Vector::pop):
2133         (bmalloc::Vector<T>::Vector):
2134         (bmalloc::Vector<T>::~Vector):
2135         (bmalloc::Vector<T>::operator):
2136         (bmalloc::Vector<T>::push):
2137         (bmalloc::Vector<T>::pop):
2138         (bmalloc::Vector<T>::shrink):
2139         (bmalloc::Vector<T>::reallocateBuffer):
2140         (bmalloc::Vector<T>::shrinkCapacity):
2141         (bmalloc::Vector<T>::growCapacity):
2142         * bmalloc/XLargeChunk.h: Added.
2143         (bmalloc::XLargeChunk::get):
2144         (bmalloc::XLargeChunk::begin):
2145         (bmalloc::XLargeChunk::XLargeChunk):
2146         (bmalloc::XLargeChunk::create):
2147         (bmalloc::XLargeChunk::destroy):
2148         (bmalloc::XLargeChunk::range):
2149         (bmalloc::XLargeChunk::size):
2150         * bmalloc/bmalloc.h: Added.
2151         (bmalloc::api::malloc):
2152         (bmalloc::api::free):
2153         (bmalloc::api::realloc):
2154         * bmalloc/mbmalloc.cpp: Added.
2155