bmalloc should implement malloc introspection (to stop false-positive leaks when...
[WebKit-https.git] / Source / bmalloc / ChangeLog
1 2015-02-19  Geoffrey Garen  <ggaren@apple.com>
2
3         bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
4         https://bugs.webkit.org/show_bug.cgi?id=141802
5
6         Reviewed by Andreas Kling.
7
8         Fixed a last-minute type.
9
10         The macro is OS, not PLATFORM.
11
12         * bmalloc/VMHeap.cpp:
13         (bmalloc::VMHeap::grow):
14         * bmalloc/VMHeap.h:
15         * bmalloc/Zone.h:
16
17 2015-02-19  Geoffrey Garen  <ggaren@apple.com>
18
19         bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
20         https://bugs.webkit.org/show_bug.cgi?id=141802
21
22         Reviewed by Andreas Kling.
23
24         This patch does the bare minimum to stop false positive leaks from
25         being reported by the Darwin leaks tool. We register each super chunk
26         as a single object, and then request that the leaks tool scan it.
27
28         * bmalloc.xcodeproj/project.pbxproj: Added an abstraction for the malloc
29         zone introspection API.
30
31         * bmalloc/Algorithm.h: Missing #include.
32
33         * bmalloc/VMHeap.cpp:
34         (bmalloc::VMHeap::grow):
35         * bmalloc/VMHeap.h: Adopt the new abstraction.
36
37         * bmalloc/Zone.cpp: Added.
38         (bmalloc::remoteRead): Helper for reading an object out of another process.
39         (bmalloc::Zone::enumerator):
40         (bmalloc::Zone::Zone): Register a malloc zone so that we will participate
41         in introspection.
42
43         * bmalloc/Zone.h: Added.
44         (bmalloc::Zone::superChunks):
45         (bmalloc::Zone::addSuperChunk): Use a non-dynamically-allocated vector
46         since our dynamic allocations will not be scanned by leaks since they
47         will have the malloc VM tag.
48
49 2015-02-18  Geoffrey Garen  <ggaren@apple.com>
50
51         bmalloc: VMHeap should keep a record of all of its VM ranges (for malloc introspection)
52         https://bugs.webkit.org/show_bug.cgi?id=141759
53
54         Reviewed by Andreas Kling.
55
56         * bmalloc.xcodeproj/project.pbxproj:
57         * bmalloc/SuperChunk.h: Added.
58         (bmalloc::SuperChunk::create):
59         (bmalloc::SuperChunk::SuperChunk):
60         (bmalloc::SuperChunk::smallChunk):
61         (bmalloc::SuperChunk::mediumChunk):
62         (bmalloc::SuperChunk::largeChunk): Factored out super chunk creation
63         into a separate class, for clarity and type safety.
64
65         * bmalloc/VMHeap.cpp:
66         (bmalloc::VMHeap::grow):
67         (bmalloc::VMHeap::allocateSuperChunk): Renamed "allocateSuperChunk" to
68         "grow" because Andreas found "allocateSuperChunk" to be unclear.
69
70         * bmalloc/VMHeap.h: Track all our VM ranges. We will use this information
71         for malloc introspection.
72
73         (bmalloc::VMHeap::allocateSmallPage):
74         (bmalloc::VMHeap::allocateMediumPage):
75         (bmalloc::VMHeap::allocateLargeRange): Updated for renames.
76
77 2015-02-18  Zan Dobersek  <zdobersek@igalia.com>
78
79         Build bmalloc through CMake as a static library. It's then linked either
80         into the WTF library (if built as a shared library) or into the JSC and
81         WebKit2 libraries. There's no need to build it as a standalone shared library.
82
83         Rubber-stamped by Carlos Garcia Campos.
84
85         * CMakeLists.txt:
86
87 2015-02-13  Gyuyoung Kim  <gyuyoung.kim@samsung.com>
88
89         [BMalloc] Add a FIXME comment for memory alignas
90         https://bugs.webkit.org/show_bug.cgi?id=141556
91
92         Reviewed by Csaba Osztrogonác.
93
94         * bmalloc/Chunk.h: Add a FIXME comment.
95         * bmalloc/LargeChunk.h: ditto.
96
97 2015-02-11  Csaba Osztrogonác  <ossy@webkit.org>
98
99         bmalloc buildfix on 32 bit Linux (x86/ARM)
100         https://bugs.webkit.org/show_bug.cgi?id=141472
101
102         Reviewed by Gyuyoung Kim.
103
104         * bmalloc/Algorithm.h:
105         (bmalloc::roundUpToMultipleOf):
106         * bmalloc/FixedVector.h:
107         (bmalloc::FixedVector::clear):
108         * bmalloc/Sizes.h:
109         (bmalloc::Sizes::sizeClass):
110
111 2015-02-11  Gyuyoung Kim  <gyuyoung.kim@samsung.com>
112
113         [EFL][GTK] Use bmalloc instead of tcmalloc
114         https://bugs.webkit.org/show_bug.cgi?id=140162
115
116         Reviewed by Carlos Garcia Campos.
117
118         Support to use bmalloc on EFL and GTK ports.
119
120         * CMakeLists.txt: Added.
121         * bmalloc/Allocator.cpp:
122         (bmalloc::Allocator::allocate):
123             Fix unused return value caused by posix_memalign().
124         * bmalloc/AsyncTask.h:
125         * bmalloc/BoundaryTag.h:
126         (bmalloc::BoundaryTag::clear):
127         * bmalloc/Chunk.h:
128             Change Traits::Page with Traits::PageType in order to fix
129             -fpermitive build error on EFL and GTK port. 
130         * bmalloc/EndTag.h:
131         (bmalloc::EndTag::operator=):
132         * bmalloc/Line.h: ditto.
133         * bmalloc/MediumTraits.h:
134         * bmalloc/Page.h: ditto.
135         * bmalloc/PerThread.h:
136             EFL port doesn't support __has_include definition yet.
137             Define HAVE_PTHREAD_MACHDEP_H according to check if __has_include is supported.
138         * bmalloc/SmallTraits.h: ditto.
139         * bmalloc/VMAllocate.h:
140         (bmalloc::vmDeallocatePhysicalPages):
141         (bmalloc::vmAllocatePhysicalPages):
142         * bmalloc/Vector.h:
143         (bmalloc::Vector<T>::push):
144         (bmalloc::Vector<T>::reallocateBuffer):
145
146 2015-01-31  Sam Weinig  <sam@webkit.org>
147
148         Remove even more Mountain Lion support
149         https://bugs.webkit.org/show_bug.cgi?id=141124
150
151         Reviewed by Alexey Proskuryakov.
152
153         * Configurations/Base.xcconfig:
154         * Configurations/DebugRelease.xcconfig:
155
156 2015-01-30  Geoffrey Garen  <ggaren@apple.com>
157
158         GC marking threads should clear malloc caches
159         https://bugs.webkit.org/show_bug.cgi?id=141097
160
161         Reviewed by Andreas Kling.
162
163         Split the scavenging API into per-thread vs global, so that you can
164         request to scavenge your own thread without scavenging the whole heap.
165
166         * bmalloc/Cache.cpp:
167         (bmalloc::Cache::scavenge):
168         * bmalloc/bmalloc.h:
169         (bmalloc::api::scavengeThisThread):
170         (bmalloc::api::scavenge):
171
172 2015-01-28  Dana Burkart  <dburkart@apple.com>
173
174         Move ASan flag settings from DebugRelease.xcconfig to Base.xcconfig
175         https://bugs.webkit.org/show_bug.cgi?id=136765
176
177         Reviewed by Alexey Proskuryakov.
178
179         * Configurations/Base.xcconfig:
180         * Configurations/DebugRelease.xcconfig:
181
182 2015-01-21  Geoffrey Garen  <ggaren@apple.com>
183
184         bmalloc: support aligned allocation
185         https://bugs.webkit.org/show_bug.cgi?id=140732
186
187         Reviewed by Andreas Kling.
188
189         * bmalloc/Allocator.cpp:
190         (bmalloc::Allocator::allocate): New function for aligned allocation.
191
192         Small and medium requests just allocate and free until they find an
193         aligned pointer. This is slightly inefficient in the worst case, but
194         still constant-time with little-to-no space overhead.
195
196         Large requests use a new API that requires the client to specify both
197         its ideal size and alignment, and the worst-case size you would have to
198         allocate in order to produce some interior pointer of the requested size
199         and alignment. We put the burden of this calculation on the client
200         because it simplifies things if we guarantee that allocation won't fail.
201
202         XLarge requests are easy: we just forward them to vmAllocate, which
203         already supported aligned requests.
204
205         * bmalloc/BoundaryTag.h:
206         * bmalloc/BoundaryTagInlines.h:
207         (bmalloc::BoundaryTag::mergeLeft):
208         (bmalloc::BoundaryTag::mergeRight):
209         (bmalloc::BoundaryTag::merge):
210         (bmalloc::BoundaryTag::deallocate):
211         (bmalloc::BoundaryTag::split):
212         (bmalloc::BoundaryTag::allocate): No behavior change here. I just
213         refactored the interface to remove some reference out parameters in
214         order to clarify what changes and what doesn't.
215
216         * bmalloc/Heap.cpp:
217         (bmalloc::Heap::allocateXLarge): Added an alignment API.
218
219         (bmalloc::Heap::allocateLarge):
220         * bmalloc/Heap.h: Added an alignment API. I split out allocateLarge into
221         a few variants, so aligned and unaligned allocation could share some code.
222
223         * bmalloc/SegregatedFreeList.cpp:
224         (bmalloc::SegregatedFreeList::take):
225         * bmalloc/SegregatedFreeList.h: Changed to use a separate, explicit API
226         for aligned allocation. It turns out that the aligned path is pretty
227         different, since it ends up searching for two potential ways to satisfy
228         an allocation: either large enough and aligned, or large enough to split
229         into something not aligned and something large enough and aligned.
230
231         * bmalloc/VMAllocate.h:
232         (bmalloc::vmAllocate): Switched alignment to come before size because
233         that's how the memalign API specifies it.
234
235         * bmalloc/VMHeap.h:
236         (bmalloc::VMHeap::allocateLargeRange): Added an alignment API.
237
238 2015-01-20  Geoffrey Garen  <ggaren@apple.com>
239
240         bmalloc: a little bit of cleanup
241         https://bugs.webkit.org/show_bug.cgi?id=140687
242
243         Reviewed by Anders Carlsson.
244
245         * bmalloc/Algorithm.h:
246         (bmalloc::isPowerOfTwo): Added a check for 0, since 0 would break a lot
247         of code.
248
249         * bmalloc/BoundaryTag.h:
250         * bmalloc/BoundaryTagInlines.h:
251         (bmalloc::BoundaryTag::mergeLeft):
252         (bmalloc::BoundaryTag::mergeRight):
253         (bmalloc::BoundaryTag::merge):
254         (bmalloc::BoundaryTag::deallocate):
255         (bmalloc::BoundaryTag::split):
256         (bmalloc::BoundaryTag::allocate):
257         (bmalloc::BoundaryTag::mergeLargeLeft): Deleted.
258         (bmalloc::BoundaryTag::mergeLargeRight): Deleted.
259         (bmalloc::BoundaryTag::mergeLarge): Deleted.
260         (bmalloc::BoundaryTag::splitLarge): Deleted. Removed the word "Large"
261         from all these functions, since boundary tags always pertain to large
262         objects, and putting the word "Large" everywhere wasn't helping to
263         explain that.
264
265         * bmalloc/Heap.cpp:
266         (bmalloc::Heap::allocateXLarge):
267         (bmalloc::Heap::findXLarge):
268         (bmalloc::Heap::deallocateXLarge):
269         * bmalloc/Heap.h:
270         * bmalloc/VMHeap.h:
271         (bmalloc::VMHeap::allocateXLarge): Deleted.
272         (bmalloc::VMHeap::findXLarge): Deleted.
273         (bmalloc::VMHeap::deallocateXLarge): Deleted. Moved XLarge allocation
274         from VMHeap to Heap. Since the purpose of the VMHeap is to cache VM
275         ranges, and the VMHeap never caches any XLarge ranges, it doesn't
276         really make sense for the VMHeap to be involved.
277
278 2015-01-16  Geoffrey Garen  <ggaren@apple.com>
279
280         bmalloc: refactored XLarge allocation for better alignment
281         https://bugs.webkit.org/show_bug.cgi?id=140582
282
283         Reviewed by Andreas Kling.
284
285         XLarge objects used to be Large objects with an extra bit of metadata
286         that said "actually, I'm not large -- I'm extra large".
287
288         The metadata header in an XLarge allocation made it impossible for the
289         XLarge object to honor a very large alignment request.
290
291         The solution is to stop using a metadata header for XLarge objects, and
292         instead to store explicit metadata on the side.
293
294         This is a bit less astonishing, which is also nice.
295
296         Finding XLarge metadata is now a linear search. That's probably OK, since
297         it was always so in TCMalloc, and the usual number of XLarge allocations
298         in a process is 0.
299
300         This design makes it possible for the heap to cache XLarge allocations
301         with and/or without physical pages. I haven't actually done that yet
302         because the tradeoffs are subtle, so I don't want to do anything without
303         a motivating test case.
304
305         * bmalloc.xcodeproj/project.pbxproj:
306         * bmalloc/Allocator.cpp:
307         (bmalloc::Allocator::reallocate): Removed the concept of an XLargeChunk,
308         since an XLarge allocation is now just a naked buffer without a header.
309
310         (bmalloc::Allocator::allocateXLarge): Added an explicit qualifier for
311         XLarge alignment, since XLargeChunk won't give this to us implicitly
312         anymore.
313
314         * bmalloc/BoundaryTag.h:
315         (bmalloc::BoundaryTag::setRange):
316         (bmalloc::BoundaryTag::isXLarge): Deleted.
317         (bmalloc::BoundaryTag::setXLarge): Deleted.
318         * bmalloc/BoundaryTagInlines.h:
319         (bmalloc::validate):
320         (bmalloc::BoundaryTag::deallocate): Removed the XLarge hacks from Large allocations.
321
322         * bmalloc/Deallocator.cpp:
323         (bmalloc::Deallocator::deallocateXLarge):
324         (bmalloc::Deallocator::deallocateSlowCase):
325         * bmalloc/Heap.cpp:
326         (bmalloc::Heap::findXLarge):
327         (bmalloc::Heap::allocateXLarge):
328         (bmalloc::Heap::deallocateXLarge):
329         * bmalloc/Heap.h: Updated for interface changes.
330
331         * bmalloc/ObjectType.cpp:
332         (bmalloc::objectType):
333         * bmalloc/ObjectType.h:
334         (bmalloc::isXLarge): We can now tell if a pointer is XLarge just by
335         examining its bit pattern -- just like we do for other kinds of
336         allocations -- which is nice.
337
338         * bmalloc/Sizes.h:
339         * bmalloc/VMHeap.h:
340         (bmalloc::VMHeap::allocateXLarge):
341         (bmalloc::VMHeap::findXLarge):
342         (bmalloc::VMHeap::deallocateXLarge): Keep an explicit vector of metadata
343         for XLarge allocations.
344
345         * bmalloc/XLargeChunk.h: Removed.
346
347 2015-01-16  Geoffrey Garen  <ggaren@apple.com>
348
349         bmalloc: added some infrastructure for aligned allocation
350         https://bugs.webkit.org/show_bug.cgi?id=140572
351
352         Reviewed by Andreas Kling.
353
354         * bmalloc/Algorithm.h:
355         (bmalloc::isPowerOfTwo):
356         (bmalloc::roundUpToMultipleOf):
357         (bmalloc::roundDownToMultipleOf): Refactored some duplicate code to use our
358         isPowerOfTwo helper function.
359
360         * bmalloc/Allocator.cpp:
361         (bmalloc::Allocator::allocate):
362         * bmalloc/Allocator.h: Stubbed out an implementation of aligned allocation.
363         Doesn't do anything yet, but does correctly forward to system malloc
364         when bmalloc is disabled.
365
366         * bmalloc/Cache.cpp:
367         (bmalloc::Cache::allocateSlowCaseNullCache):
368         * bmalloc/Cache.h:
369         (bmalloc::Cache::allocate):
370         * bmalloc/bmalloc.h:
371         (bmalloc::api::memalign):
372         * bmalloc/mbmalloc.cpp: Stubbed out an API for aligned allocation.
373
374 2015-01-13  Geoffrey Garen  <ggaren@apple.com>
375
376         Consider alignment when allocating from a SegregatedFreeList
377         https://bugs.webkit.org/show_bug.cgi?id=140408
378
379         Reviewed by Sam Weinig.
380
381         In preparation for supporting aligned allocation.
382
383         No performance change.
384
385         Since this is just one extra branch in an already expensive function,
386         I decided not to duplicate the function just to avoid the branch in
387         the un-aligned case.
388
389         * bmalloc/SegregatedFreeList.cpp:
390         (bmalloc::SegregatedFreeList::take):
391         * bmalloc/SegregatedFreeList.h:
392
393 2015-01-13  Geoffrey Garen  <ggaren@apple.com>
394
395         Renamed minimum to size in SegregatedFreeList
396         https://bugs.webkit.org/show_bug.cgi?id=140406
397
398         Reviewed by Sam Weinig.
399
400         In preparation for supporting aligned allocation.
401
402         * bmalloc/SegregatedFreeList.cpp:
403         (bmalloc::SegregatedFreeList::takeGreedy):
404         (bmalloc::SegregatedFreeList::take): Every size passed to malloc is
405         really just a minimum. Let's not imply that this value is special.
406
407 2015-01-11  Dan Bernstein  <mitz@apple.com>
408
409         Geoff is organized, but he is not an organization.
410
411         Rubber-stamped by Anders Carlsson.
412
413         * bmalloc.xcodeproj/project.pbxproj: Removed the ORGANIZATIONNAME project attribute.
414
415 2015-01-07  Geoffrey Garen  <ggaren@apple.com>
416
417         Make bmalloc work with ASan
418         https://bugs.webkit.org/show_bug.cgi?id=140194
419
420         Reviewed by Mark Lam.
421
422         * bmalloc/BPlatform.h: Added a way to detect Darwin OSes, since we need
423         an OS-specific API to test for loaded runtime libraries.
424
425         * bmalloc/Environment.cpp:
426         (bmalloc::isASanEnabled):
427         (bmalloc::Environment::computeIsBmallocEnabled): Disabled bmalloc if
428         ASan is enabled, since system malloc has the Asan hooks we need.
429
430         You could check for the ASan compile-time flag instead, but doing this
431         check at runtime prepares bmalloc for a world where it is a dynamic
432         library that might be loaded into projects it did not compile with.
433
434 2015-01-05  Geoffrey Garen  <ggaren@apple.com>
435
436         Fix up bmalloc's PerThread for use on Linux
437         https://bugs.webkit.org/show_bug.cgi?id=139804
438
439         Reviewed by Anders Carlsson.
440
441         The previous implementation was a bit slow.
442
443         * bmalloc/PerThread.h:
444         (bmalloc::PerThreadStorage<Cache>::get):
445         (bmalloc::PerThreadStorage::get):
446         (bmalloc::PerThreadStorage::init): Added a catch-all cross-platform Unix
447         way to do fast per-thread access without taking a lock every time. This
448         probably works on all the platforms we care about, and it matches other
449         techniques we use elsewhere in WebKit.
450
451         (bmalloc::PerThread<T>::getFastCase): Removed the conditional from
452         this class because PerThreadStorage now encapsulates everything that
453         needs to be conditional.
454
455         (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Deleted.
456
457 2014-12-26  Dan Bernstein  <mitz@apple.com>
458
459         <rdar://problem/19348208> REGRESSION (r177027): iOS builds use the wrong toolchain
460         https://bugs.webkit.org/show_bug.cgi?id=139950
461
462         Reviewed by David Kilzer.
463
464         * Configurations/Base.xcconfig: Only define TOOLCHAINS when building for OS X, doing so
465         in a manner that works with Xcode 5.1.1.
466
467 2014-12-15  Geoffrey Garen  <ggaren@apple.com>
468
469         Safari crashes when you set Malloc environment variables
470         https://bugs.webkit.org/show_bug.cgi?id=139656
471
472         Reviewed by Michael Saboff.
473
474         I forgot to cover the realloc() case. Whoops. (OoPS?)
475
476         This time around, I ran the full MallocBench test suite in Malloc=1
477         mode, and it passed.
478
479         * bmalloc/Allocator.cpp:
480         (bmalloc::Allocator::reallocate):
481         * bmalloc/Allocator.h: Pushed realloc() logic down into the allocator.
482         It needs to be down there so that we can do the short-circuiting check
483         for whether bmalloc is enabled first.
484
485         Also added the check.
486
487         * bmalloc/Cache.cpp:
488         (bmalloc::Cache::scavenge):
489         (bmalloc::Cache::Cache):
490         (bmalloc::Cache::reallocateSlowCaseNullCache):
491         * bmalloc/Cache.h:
492         (bmalloc::Cache::deallocator):
493         (bmalloc::Cache::reallocate): Ditto.
494
495         * bmalloc/bmalloc.h:
496         (bmalloc::api::free):
497         (bmalloc::api::realloc): Ditto.
498
499         (bmalloc::api::scavenge): Pushed this down into Cache to match the
500         surrounding functions.
501
502 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
503
504         bmalloc should support system memory analysis tools (part 2)
505         https://bugs.webkit.org/show_bug.cgi?id=139565
506
507         Reviewed by Mark Lam.
508
509         This patch actually queries the environment to see if memory analysis
510         tools have been enabled.
511
512         * bmalloc/Deallocator.cpp:
513         (bmalloc::Deallocator::scavenge): Don't process the object log if
514         we've disabled bmalloc because it will be full of invalid nullptrs.
515
516         * bmalloc/Environment.cpp:
517         (bmalloc::isMallocEnvironmentVariableSet): Test for the list of known
518         Malloc debugging flags. I also added a plain "Malloc" catch-all for
519         when you want to disable bmalloc without enabling any kind of funny
520         business.
521
522         It would be slightly nicer just to iterate the list of environment
523         variables and strstr them, but getenv is the more portable option,
524         and performance here doesn't really matter.
525
526         (bmalloc::isLibgmallocEnabled): Test for the libgmalloc insertion
527         environment variable.
528
529         (bmalloc::Environment::computeIsBmallocEnabled):
530
531 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
532
533         Try to fix the iOS simulator build.
534
535         #include the declaration of malloc / free.
536
537         * bmalloc/Allocator.cpp:
538         * bmalloc/Deallocator.cpp:
539
540 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
541
542         Try to fix the build.
543
544         * bmalloc.xcodeproj/project.pbxproj: Marked a header exported.
545
546 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
547
548         bmalloc should support system memory analysis tools (part 1)
549         https://bugs.webkit.org/show_bug.cgi?id=139559
550
551         Reviewed by Mark Lam.
552
553         This patch adds the hooks to disable bmalloc at runtime if certain
554         environment variables are set, but doesn't actually read from the
555         environment yet.
556
557         No performance change.
558
559         * bmalloc.xcodeproj/project.pbxproj: Added the Environment class, which
560         we'll use to read environment variables and see if memory analysis tools
561         have been enabled.
562
563         * bmalloc/Allocator.cpp:
564         (bmalloc::Allocator::Allocator):
565         (bmalloc::Allocator::allocateSlowCase): Added a hook to disable bmalloc
566         on the allocation path. We cache the setting to make the check fast.
567
568         * bmalloc/Allocator.h: Interface changes.
569
570         * bmalloc/Cache.cpp:
571         (bmalloc::Cache::Cache): Pass a heap pointer through to our allocator
572         and deallocator. This main purpose is to enable them to query the
573         environment for whether bmalloc is enabled; but this is also a slightly
574         cleaner way to guarantee to them that the Heap has been pre-initialized.
575
576         * bmalloc/Deallocator.cpp:
577         (bmalloc::Deallocator::Deallocator): If bmalloc is disable, artificially
578         fill the object log to force us to take the slow path on all deallocations.
579
580         (bmalloc::Deallocator::deallocateSlowCase): Do the disabled check.
581
582         * bmalloc/Deallocator.h: Interface changes.
583
584         * bmalloc/Environment.cpp: Added.
585         (bmalloc::Environment::Environment):
586         (bmalloc::Environment::computeIsBmallocEnabled):
587         * bmalloc/Environment.h: Added.
588         (bmalloc::Environment::isBmallocEnabled): This is the class that will
589         encapsulate looking for environment variables that turn on heap
590         analysis tools.
591
592         * bmalloc/Heap.h:
593         (bmalloc::Heap::environment):
594
595         * bmalloc/Mutex.h:
596         (bmalloc::Mutex::Mutex):
597         * bmalloc/StaticMutex.h: A little refactoring to clarify these comments,
598         since I got super confused about them while writing this patch.
599
600         * bmalloc/VMHeap.cpp: Fixed an #include.
601
602 2014-12-09  David Kilzer  <ddkilzer@apple.com>
603
604         Switch from using PLATFORM_NAME to SDK selectors in ANGLE, bmalloc, gtest, JavaScriptCore, WTF
605         <http://webkit.org/b/139212>
606
607         Reviewed by Joseph Pecoraro.
608
609         * Configurations/Base.xcconfig:
610         - Only set GCC_ENABLE_OBJC_GC, GCC_MODEL_TUNING and TOOLCHAINS
611           on OS X.
612         * Configurations/DebugRelease.xcconfig:
613         - Only set MACOSX_DEPLOYMENT_TARGET and SDKROOT on OS X.
614
615 2014-11-07  Geoffrey Garen  <ggaren@apple.com>
616
617         bmalloc uses 8X more virtual memory than necessary
618         https://bugs.webkit.org/show_bug.cgi?id=138495
619
620         Reviewed by Mark Lam.
621
622         iOS has a per-process virtual memory cap around 1GB, so there's some
623         value to not going totally ham with virtual memory.
624
625         We currently use about 8X the necessary amount:
626             - 2X to align our VM allocation
627             - 4X to reserve small / medium / (2) large chunk VM ranges per superchunk
628
629         We can cut that down:
630             - Return the unaligned portion of our VM allocation (-2X)
631             - Use all the chunks in a superchunk, instead of allocating one
632               chunk per superchunk (-4X)
633
634         * bmalloc/Algorithm.h:
635         (bmalloc::roundUpToMultipleOf): Added a non-constant version of this
636         function so we can call it with getpagesize() at runtime.
637
638         * bmalloc/Chunk.h:
639         * bmalloc/LargeChunk.h:
640         (bmalloc::LargeChunk::create): Deleted. Instead of each chunk allocating
641         its own VM, VMHeap allocates the superchunk and all the chunks in it at a time.
642
643         * bmalloc/VMAllocate.h:
644         (bmalloc::vmValidate):
645         (bmalloc::vmAllocate): ASSERT that mmap succeeds to make crashes clearer
646         if it does not succeed. Allocate precisely, and give back the extra.
647
648         * bmalloc/VMHeap.cpp:
649         (bmalloc::VMHeap::allocateSuperChunk):
650         (bmalloc::VMHeap::allocateSmallChunk): Deleted.
651         (bmalloc::VMHeap::allocateMediumChunk): Deleted.
652         (bmalloc::VMHeap::allocateLargeChunk): Deleted. Use all the chunks
653         in a superchunk, instead of just one.
654
655         * bmalloc/VMHeap.h:
656         (bmalloc::VMHeap::allocateSmallPage):
657         (bmalloc::VMHeap::allocateMediumPage):
658         (bmalloc::VMHeap::allocateLargeRange):
659         * bmalloc/XLargeChunk.h:
660         (bmalloc::XLargeChunk::create): Updated to match changes above.
661
662 2014-11-01  David Kilzer  <ddkilzer@apple.com>
663
664         JavaScriptCore is missing debug info for bmalloc because libbmalloc.a is stripped
665         <https://webkit.org/b/138286>
666         <rdar://problem/18847087>
667
668         Reviewed by Dan Bernstein.
669
670         * Configurations/bmalloc.xcconfig: Set STRIP_INSTALLED_PRODUCT
671         to NO for the target that produces libbmalloc.a so that the
672         debug symbols will be linked into JavaScriptCore and end up in
673         its dSYM file.
674
675 2014-10-30  Dana Burkart  <dburkart@apple.com>
676
677         <rdar://problem/18821260> Prepare for the mysterious future
678
679         Reviewed by Lucas Forschler.
680
681         * Configurations/Base.xcconfig:
682         * Configurations/DebugRelease.xcconfig:
683
684 2014-09-24  Geoffrey Garen  <ggaren@apple.com>
685
686         bmalloc: cleaned up fast path vs slow path
687         https://bugs.webkit.org/show_bug.cgi?id=137081
688
689         Reviewed by Sam Weinig.
690
691         Might be a 1% speedup on MallocBench. Also cleans up the code a bit.
692
693         * bmalloc/Allocator.cpp:
694         (bmalloc::Allocator::Allocator): Merged the small and medium range
695         caches, just like the small and medium allocators. Ranges are abstract
696         objects that don't really care whether they hold small or medium objects,
697         so they don't need to be segregated.
698
699         (bmalloc::Allocator::scavenge): Ditto.
700
701         (bmalloc::Allocator::allocateBumpRangeSlowCase):
702         (bmalloc::Allocator::allocateBumpRange): Same thing here, except that
703         we do care a tiny bit, because we need to specify small vs medium when
704         allocating new ranges from the heap, to ensure that the heap allocates
705         from the right segment of VM.
706
707         (bmalloc::Allocator::allocateLarge):
708         (bmalloc::Allocator::allocateXLarge): NO_INLINE because this was clouding
709         up the fast path. Large allocation performance is dominated by allocation
710         logic and initialization, so inlining it doesn't help.
711
712         (bmalloc::Allocator::allocateSlowCase): Slow path got a bit cleaner since
713         it doesn't need to distinguish small vs medium objects.
714
715         (bmalloc::Allocator::allocateSmallBumpRange): Deleted.
716         (bmalloc::Allocator::allocateMediumBumpRange): Deleted.
717
718         * bmalloc/Allocator.h:
719         * bmalloc/BumpRange.h:
720
721         * bmalloc/Cache.cpp:
722         (bmalloc::Cache::allocateSlowCase): Deleted.
723         (bmalloc::Cache::deallocateSlowCase): Deleted.
724         * bmalloc/Cache.h:
725         (bmalloc::Cache::allocate):
726         (bmalloc::Cache::deallocate):
727         (bmalloc::Cache::allocateFastCase): Deleted.
728         (bmalloc::Cache::deallocateFastCase): Deleted. Removed the Cache slow
729         paths. The downside to this change is that the fast path branches to two
730         distinct failure cases instead of one. The upside is that the slow path
731         doesn't need to re-read the segment register, which is not as cheap as a
732         normal register, and it doesn't need to do an extra level of function 
733         call. Seems to be worth it.
734
735         * bmalloc/Deallocator.h:
736         * bmalloc/Heap.cpp:
737         (bmalloc::Heap::refillSmallBumpRangeCache):
738         (bmalloc::Heap::refillMediumBumpRangeCache):
739         * bmalloc/Heap.h: Updated for interface changes.
740
741         * bmalloc/Sizes.h: The most ranges a cache will hold is the number of
742         small lines in a page / 2, since any other free lines will coalesce
743         with their neighbors.
744
745 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
746
747         Rolled out r173346.
748
749             bmalloc should honor the FastMalloc statistics API
750             https://bugs.webkit.org/show_bug.cgi?id=136592
751
752         This didn't really work. Because we allow ranges with and without
753         physical pages to merge, and we allow double-committing and
754         double-decommitting, we can't rely on commit actions to track memory
755         footprint.
756
757         * bmalloc/Heap.cpp:
758         (bmalloc::Heap::size): Deleted.
759         (bmalloc::Heap::capacity): Deleted.
760         * bmalloc/Heap.h:
761         * bmalloc/VMHeap.cpp:
762         (bmalloc::VMHeap::VMHeap):
763         (bmalloc::VMHeap::allocateSmallChunk):
764         (bmalloc::VMHeap::allocateMediumChunk):
765         (bmalloc::VMHeap::allocateLargeChunk):
766         * bmalloc/VMHeap.h:
767         (bmalloc::VMHeap::allocateSmallPage):
768         (bmalloc::VMHeap::allocateMediumPage):
769         (bmalloc::VMHeap::allocateLargeRange):
770         (bmalloc::VMHeap::deallocateSmallPage):
771         (bmalloc::VMHeap::deallocateMediumPage):
772         (bmalloc::VMHeap::deallocateLargeRange):
773         (bmalloc::VMHeap::size): Deleted.
774         (bmalloc::VMHeap::capacity): Deleted.
775         * bmalloc/bmalloc.h:
776         (bmalloc::api::heapSize): Deleted.
777         (bmalloc::api::heapCapacity): Deleted.
778
779 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
780
781         bmalloc: Allocation should be more precise
782         https://bugs.webkit.org/show_bug.cgi?id=136993
783
784         Reviewed by Gavin Barraclough.
785
786         13% reduction in heap size on the MallocBench *_memory_warning benchmarks.
787
788         This patch teaches the allocator to merge adjacent free lines into a
789         single allocatable range. This allows us to shrink the size of an
790         individual line without increasing fragmentation or the rate of allocator
791         slow paths.
792
793         We'll only take more slow paths when available memory is sparse, which
794         is exactly when it's worth it. When available memory is dense, we'll
795         take fewer slow paths.
796
797         * bmalloc.xcodeproj/project.pbxproj:
798         * bmalloc/Algorithm.h:
799         (bmalloc::divideRoundingUp):
800
801         * bmalloc/Allocator.cpp:
802         (bmalloc::Allocator::Allocator): Updated for interface changes.
803
804         (bmalloc::Allocator::scavenge): Scavenge by object instead of by line.
805         Now that we merge lines, it's not convenient to scavenge by line.
806
807         (bmalloc::Allocator::allocateSmallBumpRange):
808         (bmalloc::Allocator::allocateMediumBumpRange): Allocate whole ranges
809         instead of individual lines.
810
811         (bmalloc::Allocator::allocateSlowCase):
812         (bmalloc::Allocator::allocateSmallLine): Deleted.
813         (bmalloc::Allocator::allocateMediumLine): Deleted.
814         (bmalloc::Allocator::allocateMedium): Deleted.
815         * bmalloc/Allocator.h:
816         (bmalloc::Allocator::allocateFastCase): Folded medium allocations
817         into the standard fast path with small allocations. Since a BumpAllocator
818         just allocates out of an arbitrary range, it doesn't need to distinguish
819         between small and medium lines.
820
821         * bmalloc/BumpAllocator.h:
822         (bmalloc::BumpAllocator::size):
823         (bmalloc::BumpAllocator::BumpAllocator):
824         (bmalloc::BumpAllocator::init):
825         (bmalloc::BumpAllocator::refill):
826         (bmalloc::BumpAllocator::line): Deleted. No need to track line information
827         anymore: the heap just gives us a pointer and a pre-computed number of
828         objects, and we allocate them.
829
830         * bmalloc/Deallocator.cpp:
831         (bmalloc::Deallocator::processObjectLog): Updated for interface changes.
832
833         * bmalloc/Heap.cpp:
834         (bmalloc::Heap::Heap):
835         (bmalloc::Heap::initializeLineMetadata): Pre-compute precise metadata
836         detailing where all objects will lie in memory. After we merge two lines,
837         we might allocate an object that spans from one line to the next. This
838         metadata details which bits of memory overlap in that way, and how they
839         overlap.
840
841         (bmalloc::Heap::refillSmallBumpRangeCache):
842         (bmalloc::Heap::refillMediumBumpRangeCache): Scan a whole page at a time,
843         and merge adjacent free lines into BumpRanges.
844
845         (bmalloc::Heap::allocateSmallPage):
846         (bmalloc::Heap::allocateMediumPage):
847         (bmalloc::Heap::deallocateSmallLine):
848         (bmalloc::Heap::deallocateMediumLine): Track pages rather than lines,
849         since we scan for free memory a page at a time.
850
851         (bmalloc::Heap::allocateSmallLineSlowCase): Deleted.
852         (bmalloc::Heap::allocateMediumLineSlowCase): Deleted. Folded into the
853         fast path.
854
855         * bmalloc/Heap.h:
856         (bmalloc::Heap::derefSmallLine):
857         (bmalloc::Heap::derefMediumLine):
858         (bmalloc::Heap::deallocateSmallLine): Deleted.
859         (bmalloc::Heap::allocateSmallLine): Deleted.
860         (bmalloc::Heap::deallocateMediumLine): Deleted.
861         (bmalloc::Heap::allocateMediumLine): Deleted. Updated for interface changes.
862
863         * bmalloc/Line.h:
864         (bmalloc::Line<Traits>::ref):
865         (bmalloc::Line<Traits>::deref):
866         (bmalloc::Line<Traits>::concurrentRef): Deleted. We don't pass a derefCount
867         anymore, since we only ever deref by 1 now.
868
869         * bmalloc/MediumAllocator.h:
870         (bmalloc::MediumAllocator::isNull): Deleted.
871         (bmalloc::MediumAllocator::MediumAllocator): Deleted.
872         (bmalloc::MediumAllocator::line): Deleted.
873         (bmalloc::MediumAllocator::allocate): Deleted.
874         (bmalloc::MediumAllocator::derefCount): Deleted.
875         (bmalloc::MediumAllocator::refill): Deleted.
876         (bmalloc::MediumAllocator::clear): Deleted. Deleted some code that's
877         been dead for a while, since it doesn't build anymore with this patch.
878
879         * bmalloc/Page.h:
880         (bmalloc::Page::sizeClass):
881         (bmalloc::Page::setSizeClass):
882         (bmalloc::Page::smallSizeClass): Deleted.
883         (bmalloc::Page::setSmallSizeClass): Deleted. Renamed setSmallSizeClass
884         to sizeClass, since we use it for medium sizes too.
885
886         * bmalloc/Sizes.h:
887         (bmalloc::Sizes::sizeClass):
888         (bmalloc::Sizes::objectSize): Shrank line sizes to save memory.
889
890         (bmalloc::Sizes::smallSizeClassFor): Deleted.
891         (bmalloc::Sizes::mediumSizeClassFor): Deleted.
892
893         * bmalloc/bmalloc.h:
894         (bmalloc::api::realloc): Now that we have precise objects sizes, realloc
895         can be a bit more precise. It also has to be, since we can't guarantee
896         that an object ends at the end of a line anymore.
897
898 2014-09-19  Daniel Bates  <dabates@apple.com>
899
900         Always assume internal SDK when building configuration Production
901         https://bugs.webkit.org/show_bug.cgi?id=136925
902         <rdar://problem/18362399>
903
904         Reviewed by Dan Bernstein.
905
906         * Configurations/Base.xcconfig:
907
908 2014-09-16  Geoffrey Garen  <ggaren@apple.com>
909
910         bmalloc: moved line caches from the deallocator to the allocator
911         https://bugs.webkit.org/show_bug.cgi?id=136868
912
913         Reviewed by Gavin Barraclough.
914
915         I did this mostly as a simplification, to make it easier to change the
916         allocation strategy.
917
918         No throughput change on MallocBench. Saves about 50kB.
919
920         Since the deallocator needs to lock the heap when freeing lines anyway,
921         there isn't much benefit to giving the deallocator a local cache of
922         deallocated lines.
923
924         We still give the allocator a local cache of lines because that does
925         reduce the frequency at which it needs to lock the heap in order to
926         acquire more lines.
927
928         * bmalloc/Allocator.cpp:
929         (bmalloc::Allocator::scavenge):
930         (bmalloc::Allocator::allocateSmallLine):
931         (bmalloc::Allocator::allocateMediumLine):
932         (bmalloc::Allocator::allocateMedium):
933         (bmalloc::Allocator::allocateSlowCase):
934         * bmalloc/Allocator.h:
935         * bmalloc/Deallocator.cpp:
936         (bmalloc::Deallocator::Deallocator):
937         (bmalloc::Deallocator::scavenge):
938         (bmalloc::Deallocator::processObjectLog):
939         (bmalloc::Deallocator::deallocateSmallLine): Deleted.
940         (bmalloc::Deallocator::allocateSmallLine): Deleted.
941         (bmalloc::Deallocator::deallocateMediumLine): Deleted.
942         (bmalloc::Deallocator::allocateMediumLine): Deleted.
943         * bmalloc/Deallocator.h:
944
945         * bmalloc/Sizes.h:
946         * bmalloc/VMAllocate.h: Took the opportunity to make the line cache size
947         exactly one page in size. That's about what we were shooting for anyway,
948         and it may make it easier to switch to per-page allocation in future.
949
950 2014-09-15  Geoffrey Garen  <ggaren@apple.com>
951
952         bmalloc: allocate small and medium objects using the same bump pointer class
953         https://bugs.webkit.org/show_bug.cgi?id=136843
954
955         Reviewed by Gavin Barraclough.
956
957         4% speedup on MallocBench.
958
959         Now that medium-sized objects have dedicated per-size allocators, they
960         don't need to use an arbitrary bump pointer allocator. This means that
961         every allocator knows how many objects it will allocate from the start,
962         and we don't need a post-processing step to adjust refcounts based on
963         real allocation count.
964
965         * bmalloc.xcodeproj/project.pbxproj: Renamed SmallAllocator to BumpAllocator
966         since it's used for small and medium objects now.
967
968         * bmalloc/Allocator.cpp:
969         (bmalloc::Allocator::Allocator): Updated to use new interface.
970         (bmalloc::Allocator::scavenge): To "retire" an allocator, we just need
971         to make sure that we finish allocating all the objects in it.
972
973         (bmalloc::Allocator::allocateMedium):
974         (bmalloc::Allocator::allocateSlowCase):
975         (bmalloc::Allocator::retire): Deleted.
976         (bmalloc::Allocator::processSmallAllocatorLog): Deleted.
977         (bmalloc::Allocator::processMediumAllocatorLog): Deleted.
978         * bmalloc/Allocator.h:
979         (bmalloc::Allocator::allocateFastCase): Removed abstractions and data
980         used to post-process an allocator based on how many objects it allocated.
981
982         * bmalloc/BumpAllocator.h: Copied from Source/bmalloc/bmalloc/SmallAllocator.h.
983         (bmalloc::BumpAllocator::BumpAllocator):
984         (bmalloc::BumpAllocator::init):
985         (bmalloc::BumpAllocator::line):
986         (bmalloc::BumpAllocator::validate):
987         (bmalloc::BumpAllocator::allocate):
988         (bmalloc::BumpAllocator::refill):
989         (bmalloc::BumpAllocator::clear): Updated these functions to be agnostic
990         about the kinds of lines they allocate into. In some cases, the line
991         type must be provided as a template parameter by the caller.
992
993         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
994         (bmalloc::SmallAllocator::line): Deleted.
995         (bmalloc::SmallAllocator::allocate): Deleted.
996         (bmalloc::SmallAllocator::objectCount): Deleted.
997         (bmalloc::SmallAllocator::derefCount): Deleted.
998         (bmalloc::SmallAllocator::refill): Deleted.
999         (bmalloc::SmallAllocator::clear): Deleted.
1000
1001         * bmalloc/ObjectType.h:
1002         (bmalloc::isMedium):
1003
1004         * bmalloc/SmallAllocator.h:
1005         (bmalloc::SmallAllocator::isNull): Deleted.
1006         (bmalloc::SmallAllocator::canAllocate): Deleted.
1007         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
1008         (bmalloc::SmallAllocator::line): Deleted.
1009         (bmalloc::SmallAllocator::allocate): Deleted.
1010         (bmalloc::SmallAllocator::objectCount): Deleted.
1011         (bmalloc::SmallAllocator::derefCount): Deleted.
1012         (bmalloc::SmallAllocator::refill): Deleted.
1013         (bmalloc::SmallAllocator::clear): Deleted.
1014
1015 2014-09-12  Geoffrey Garen  <ggaren@apple.com>
1016
1017         Fixed a goof in bmalloc Vector sizing
1018         https://bugs.webkit.org/show_bug.cgi?id=136795
1019
1020         Reviewed by Gavin Barraclough and Sam Weinig.
1021
1022         We want our minimum vector to be page-sized since the OS will give us
1023         a page no matter what -- but we want that many bytes, and not enough
1024         bytes to store that many elements.
1025
1026         * bmalloc/Vector.h: Math is hard.
1027
1028 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
1029
1030         bmalloc should segregate medium-sized objects by line like it does for small-sized objects
1031         https://bugs.webkit.org/show_bug.cgi?id=136693
1032
1033         Reviewed by Gavin Barraclough.
1034
1035         4% reduction in heap size on the MallocBench *_memory_warning benchmarks.
1036
1037         No throughput change.
1038
1039         We keep an array of medium allocators, just like our array of small
1040         allocators.
1041
1042         In future, we can simplify the allocation fast path by merging the small
1043         and medium allocator arrays. For now, this is the simplest change that
1044         gets the win.
1045
1046         * bmalloc/Allocator.cpp:
1047         (bmalloc::Allocator::Allocator):
1048         (bmalloc::Allocator::scavenge):
1049         (bmalloc::Allocator::allocateMedium):
1050         * bmalloc/Allocator.h:
1051         * bmalloc/Sizes.h:
1052         (bmalloc::Sizes::mediumSizeClassFor):
1053
1054 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
1055
1056         Reviewed by Sam Weinig.
1057
1058         Renamed log => retire for clarity.
1059
1060         * bmalloc/Allocator.cpp:
1061         (bmalloc::Allocator::scavenge):
1062         (bmalloc::Allocator::retire):
1063         (bmalloc::Allocator::allocateMedium):
1064         (bmalloc::Allocator::allocateSlowCase):
1065         (bmalloc::Allocator::log): Deleted.
1066         * bmalloc/Allocator.h:
1067
1068 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
1069
1070         bmalloc: eager scavenge leaves behind a bogus allocator
1071         https://bugs.webkit.org/show_bug.cgi?id=136743
1072
1073         Reviewed by Sam Weinig.
1074
1075         Be sure to clear the allocator after logging it in the eager scavenge
1076         case, so that we don't later try to allocate out of the lines that we
1077         have thrown away.
1078
1079         We didn't need to do this previously because scavenge would only happen
1080         at thread exit time, after which no further allocation from the per-thread
1081         cache would take place.
1082
1083         * bmalloc/Allocator.cpp:
1084         (bmalloc::Allocator::scavenge):
1085         * bmalloc/MediumAllocator.h:
1086         (bmalloc::MediumAllocator::clear):
1087         * bmalloc/SmallAllocator.h:
1088         (bmalloc::SmallAllocator::clear):
1089
1090 2014-09-05  Geoffrey Garen  <ggaren@apple.com>
1091
1092         bmalloc should honor the FastMalloc statistics API
1093         https://bugs.webkit.org/show_bug.cgi?id=136592
1094
1095         Reviewed by Gavin Barraclough.
1096
1097         We do this by tracking "size" and "capacity" in the VM heap.
1098
1099         The VM heap's "capacity" is all the VM we ever allocated.
1100
1101         The VM heap's "size" the subset of VM currently held onto by the
1102         VM heap (and therefore not in use by the regular heap).
1103
1104         Somewhat ironically, reducing the process's memory footprint, increases
1105         the size of the VM heap, since the VM heap holds the pages that are
1106         purely virtual and not physical.
1107
1108         * bmalloc/Heap.cpp:
1109         (bmalloc::Heap::size):
1110         (bmalloc::Heap::capacity):
1111         * bmalloc/Heap.h:
1112         * bmalloc/VMHeap.cpp:
1113         (bmalloc::VMHeap::VMHeap):
1114         (bmalloc::VMHeap::allocateSmallChunk):
1115         (bmalloc::VMHeap::allocateMediumChunk):
1116         (bmalloc::VMHeap::allocateLargeChunk):
1117         * bmalloc/VMHeap.h:
1118         (bmalloc::VMHeap::size):
1119         (bmalloc::VMHeap::capacity):
1120         (bmalloc::VMHeap::allocateSmallPage):
1121         (bmalloc::VMHeap::allocateMediumPage):
1122         (bmalloc::VMHeap::allocateLargeRange):
1123         (bmalloc::VMHeap::deallocateSmallPage):
1124         (bmalloc::VMHeap::deallocateMediumPage):
1125         (bmalloc::VMHeap::deallocateLargeRange):
1126         * bmalloc/bmalloc.h:
1127         (bmalloc::api::heapSize):
1128         (bmalloc::api::heapCapacity):
1129
1130 2014-09-02  Geoffrey Garen  <ggaren@apple.com>
1131
1132         bmalloc crashes on the EWS bots (due to bad large object allocation)
1133         https://bugs.webkit.org/show_bug.cgi?id=136469
1134
1135         Reviewed by Andreas Kling.
1136
1137         It's possible to convince bmalloc to perform a bad large object allocation,
1138         through these steps:
1139
1140         (1) Insert object A into freelist F0.
1141
1142         (2) Split, merge and split again A's neighbors such that object B is
1143         inserted into freelist F0, with boundary tag and size equal to object A,
1144         but pointer not completely equal to object A. Put object B at the head of F0.
1145
1146         (3) Allocate some other object from F0, swapping its position in the
1147         freelist with object B, such that object A is now ahead of object B.
1148
1149         --> Now, the next allocation for size A/B will allocate object A, which
1150         has a slightly wrong idea about where the object actually begins.
1151         Immediately, you'll corrupt a little memory, and over time, you'll also
1152         corrupt boundary tag metadata.
1153
1154         The solution is to store the begin pointer in the boundary tag. Luckily,
1155         this doesn't make the tag any bigger, and it's not a noticeable slowdown
1156         on MallocBench.
1157
1158         * bmalloc/Algorithm.h:
1159         (bmalloc::rightShift):
1160         * bmalloc/BeginTag.h:
1161         (bmalloc::BeginTag::isInFreeList): This is the bug fix. Make sure to
1162         validate the start pointer when popping off the free list. Through a
1163         very uncommon set of steps, it is possible to have an item in the free
1164         list that is valid by all accounts except for its start pointer.
1165
1166         * bmalloc/BoundaryTag.h:
1167         (bmalloc::BoundaryTag::compactBegin):
1168         (bmalloc::BoundaryTag::setRange):
1169         (bmalloc::BoundaryTag::setSize): Deleted. Record a compact version of the
1170         start pointer. We don't need the whole pointer -- just the offset, in
1171         largeAlignment increments, into the relevant boundary tag bucket.
1172
1173         * bmalloc/BoundaryTagInlines.h:
1174         (bmalloc::validateNext):
1175         (bmalloc::BoundaryTag::init):
1176         (bmalloc::BoundaryTag::mergeLarge):
1177         (bmalloc::BoundaryTag::splitLarge):
1178         * bmalloc/SegregatedFreeList.cpp:
1179         (bmalloc::SegregatedFreeList::insert):
1180         (bmalloc::SegregatedFreeList::takeGreedy):
1181         (bmalloc::SegregatedFreeList::take): Provide the whole range instead of
1182         the size when establishing a boundary tag, as required by the new
1183         interface.
1184
1185         * bmalloc/Sizes.h:
1186
1187 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
1188
1189         Fixed a bmalloc crash seen on the EWS bot
1190         https://bugs.webkit.org/show_bug.cgi?id=135955
1191
1192         Reviewed by Andreas Kling.
1193
1194         * bmalloc/Syscall.h: Some CG APIs vm_copy their input buffers. If the
1195         input buffer is a malloc region, that region will get marked Copy-On-Write
1196         by the kernel. Calls to madvise() for COW regions fail and return EINVAL
1197         on older OS X's. In 10.10, they still fail, but they do not return
1198         EINVAL.
1199
1200         So, we can only ASSERT that our syscalls succeed starting with 10.10.
1201
1202 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
1203
1204         Fixed the bmalloc build
1205         https://bugs.webkit.org/show_bug.cgi?id=135953
1206
1207         Reviewed by Andreas Kling.
1208
1209         * bmalloc.xcodeproj/project.pbxproj: Marked a few headers as private.
1210         These headers are used, so they must be available outside the project.
1211
1212 2014-08-13  Daniel Bates  <dabates@apple.com>
1213
1214         Attempt to fix the build following <http://trac.webkit.org/changeset/172576>
1215         (https://bugs.webkit.org/show_bug.cgi?id=135895)
1216
1217         Substitute PerThreadStorage<T>::initSharedKeyIfNeeded() for initSharedKeyIfNeeded() in
1218         implementation of PerThread<T>::getFastCase().
1219
1220         * bmalloc/PerThread.h:
1221         (bmalloc::PerThread<T>::getFastCase):
1222
1223 2014-08-13  Daniel Bates  <dabates@apple.com>
1224
1225         Make bmalloc::PerThread work without C++ thread local storage
1226         https://bugs.webkit.org/show_bug.cgi?id=135895
1227
1228         Reviewed by Geoffrey Garen.
1229
1230         Implement support for building bmalloc without C++ thread local storage.
1231
1232         * bmalloc/BPlatform.h: Remove macro define BPLATFORM_IOS_SIMULATOR. Added macro function
1233         BCOMPILER_SUPPORTS() and macro define BCOMPILER_SUPPORTS_CXX_THREAD_LOCAL that can be used
1234         to determine whether the compiler supports C++ thread local storage.
1235         * bmalloc/PerThread.h:
1236         (bmalloc::PerThreadStorage::get): Modified to call pthread_getspecific() when building
1237         without C++ thread local storage.
1238         (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Added.
1239         (bmalloc::PerThreadStorage::init): Moved logic to initialize shared Pthread key from here to
1240         PerThreadStorage::initSharedKeyIfNeeded().
1241         (bmalloc::PerThread<T>::getFastCase): Modified to call PerThreadStorage::initSharedKeyIfNeeded()
1242         before querying PerThreadStorage::get() when building without C++ thread local storage so as to
1243         ensure that the shared key has been initialized.
1244         (_pthread_setspecific_direct): Deleted.
1245         (_pthread_getspecific_direct): Deleted.
1246
1247 2014-08-13  Daniel Bates  <dabates@apple.com>
1248
1249         [iOS] Make JavaScriptCore and bmalloc build with the public SDK
1250         https://bugs.webkit.org/show_bug.cgi?id=135848
1251
1252         Reviewed by Geoffrey Garen.
1253
1254         * bmalloc/BPlatform.h: Added macro BPLATFORM_IOS_SIMULATOR, which evaluates to true
1255         when building for the iOS Simulator.
1256         * bmalloc/PerThread.h: Use pthread_machdep.h code path when building for iOS Simulator
1257         using the public SDK.
1258         (_pthread_setspecific_direct): Added; only defined when building for the iOS Simulator
1259         using the public SDK.
1260         (_pthread_getspecific_direct): Added; only defined when building for the iOS Simulator
1261         using the public SDK.
1262
1263 2014-08-12  Daniel Bates  <dabates@apple.com>
1264
1265         BPLATFORM(IOS) always evaluates to false
1266         https://bugs.webkit.org/show_bug.cgi?id=135843
1267
1268         Reviewed by Geoffrey Garen.
1269
1270         Fix typo in definition of BPLATFORM() and include system header TargetConditionals.h
1271         (when building on an Apple platform) so that BPLATFORM(X) evaluates to true when
1272         building for platform X. In particular, so that BPLATFORM(IOS) evaluates to true when
1273         building for iOS.
1274
1275         As a side effect of this change, the change made in <http://trac.webkit.org/changeset/167289>
1276         will be honored and iOS will assume a VM page size of 16kB (again) instead of 4kB.
1277
1278         * bmalloc/BPlatform.h:
1279
1280 2014-08-11  Andy Estes  <aestes@apple.com>
1281
1282         [iOS] Get rid of iOS.xcconfig
1283         https://bugs.webkit.org/show_bug.cgi?id=135809
1284
1285         Reviewed by Joseph Pecoraro.
1286
1287         All iOS.xcconfig did was include AspenFamily.xcconfig, so there's no need for the indirection.
1288
1289         * Configurations/Base.xcconfig:
1290         * Configurations/iOS.xcconfig: Removed.
1291         * bmalloc.xcodeproj/project.pbxproj:
1292
1293 2014-05-01  Dan Bernstein  <mitz@apple.com>
1294
1295         Fixed production builds for the iOS Simulator.
1296         <rdar://problem/16792221>
1297
1298         * Configurations/bmalloc.xcconfig: Include INSTALL_PATH_PREFIX in
1299         PRIVATE_HEADERS_FOLDER_PATH when installing.
1300
1301 2014-04-20  Geoffrey Garen  <ggaren@apple.com>
1302
1303         bmalloc: Segregate pages by objects size
1304         https://bugs.webkit.org/show_bug.cgi?id=131909
1305
1306         Reviewed by Andreas Kling.
1307
1308         2% reduction in memory-at-end on the Membuster memory_warning benchmarks.
1309
1310         * bmalloc/Allocator.cpp:
1311         (bmalloc::Allocator::allocateSlowCase):
1312         * bmalloc/Allocator.h:
1313         (bmalloc::Allocator::allocateFastCase):
1314         (bmalloc::Allocator::smallAllocatorFor): Use the new shared helper
1315         function for size class calculation.
1316
1317         * bmalloc/Deallocator.cpp:
1318         (bmalloc::Deallocator::Deallocator):
1319         (bmalloc::Deallocator::scavenge):
1320         (bmalloc::Deallocator::deallocateSmallLine):
1321         (bmalloc::Deallocator::allocateSmallLine):
1322         * bmalloc/Deallocator.h: Keep a cache for every size class, since the
1323         cache can't be shared anymore.
1324
1325         * bmalloc/Heap.cpp:
1326         (bmalloc::Heap::allocateSmallLineSlowCase):
1327         * bmalloc/Heap.h:
1328         (bmalloc::Heap::deallocateSmallLine): Ditto.
1329
1330         (bmalloc::Heap::allocateSmallLine): Check size class in addition to
1331         page refcount when allocating a line because we might have deallocated
1332         the page and the recycled it for another size class.
1333
1334         (bmalloc::Heap::deallocateMediumLine):
1335         (bmalloc::Heap::allocateMediumLine):
1336         * bmalloc/Line.h:
1337         (bmalloc::Line::refCount):
1338         * bmalloc/Page.h:
1339         (bmalloc::Page::refCount):
1340         (bmalloc::Page::smallSizeClass):
1341         (bmalloc::Page::setSmallSizeClass):
1342         (bmalloc::Page<Traits>::refCount): Deleted.
1343         * bmalloc/Sizes.h:
1344         (bmalloc::Sizes::smallSizeClassFor): New shared API for computing
1345         an index into an array from a size.
1346
1347 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1348
1349         bmalloc: Improved alignment in LargeChunk
1350         https://bugs.webkit.org/show_bug.cgi?id=131895
1351
1352         Reviewed by Andreas Kling.
1353
1354         * bmalloc/Chunk.h:
1355         * bmalloc/LargeChunk.h: Align to vmPageSize just like Chunk does.
1356         Technically, the previous alignment was harmless, but I would prefer,
1357         dear reader, not to have to explain the interlocking set of
1358         circumstances that made it so.
1359
1360 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1361
1362         Rolled out r167502 because it caused a crash on the facebook benchmark.
1363
1364         Unreviewed.
1365
1366             bmalloc: Added an XSmall line size
1367             https://bugs.webkit.org/show_bug.cgi?id=131851
1368
1369             Reviewed by Sam Weinig.
1370
1371 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1372
1373         bmalloc: Mutex should be harder to use wrong
1374         https://bugs.webkit.org/show_bug.cgi?id=131879
1375
1376         Reviewed by Andreas Kling.
1377
1378         Mutex now has a proper constructor, so you can't deadlock by forgetting
1379         to initialize it.
1380
1381         * bmalloc.xcodeproj/project.pbxproj:
1382         * bmalloc/Allocator.cpp:
1383         (bmalloc::Allocator::processXSmallAllocatorLog):
1384         (bmalloc::Allocator::processSmallAllocatorLog):
1385         (bmalloc::Allocator::processMediumAllocatorLog):
1386         (bmalloc::Allocator::allocateLarge):
1387         (bmalloc::Allocator::allocateXLarge): Global replace Mutex => StaticMutex,
1388         since the Heap mutex is a static.
1389
1390         * bmalloc/AsyncTask.h:
1391         (bmalloc::Function>::AsyncTask): Use Mutex, since we're not static. No
1392         need for explicit initialization anymore.
1393
1394         * bmalloc/Deallocator.cpp:
1395         (bmalloc::Deallocator::scavenge):
1396         (bmalloc::Deallocator::deallocateLarge):
1397         (bmalloc::Deallocator::deallocateXLarge):
1398         (bmalloc::Deallocator::processObjectLog):
1399         (bmalloc::Deallocator::deallocateSmallLine):
1400         (bmalloc::Deallocator::deallocateXSmallLine):
1401         (bmalloc::Deallocator::allocateSmallLine):
1402         (bmalloc::Deallocator::allocateXSmallLine):
1403         (bmalloc::Deallocator::deallocateMediumLine):
1404         (bmalloc::Deallocator::allocateMediumLine):
1405         * bmalloc/Deallocator.h:
1406         * bmalloc/Heap.cpp:
1407         (bmalloc::sleep):
1408         (bmalloc::Heap::Heap):
1409         (bmalloc::Heap::concurrentScavenge):
1410         (bmalloc::Heap::scavenge):
1411         (bmalloc::Heap::scavengeSmallPages):
1412         (bmalloc::Heap::scavengeXSmallPages):
1413         (bmalloc::Heap::scavengeMediumPages):
1414         (bmalloc::Heap::scavengeLargeRanges):
1415         (bmalloc::Heap::allocateXSmallLineSlowCase):
1416         (bmalloc::Heap::allocateSmallLineSlowCase):
1417         (bmalloc::Heap::allocateMediumLineSlowCase):
1418         (bmalloc::Heap::allocateXLarge):
1419         (bmalloc::Heap::deallocateXLarge):
1420         (bmalloc::Heap::allocateLarge):
1421         (bmalloc::Heap::deallocateLarge):
1422         * bmalloc/Heap.h:
1423         (bmalloc::Heap::deallocateXSmallLine):
1424         (bmalloc::Heap::allocateXSmallLine):
1425         (bmalloc::Heap::deallocateSmallLine):
1426         (bmalloc::Heap::allocateSmallLine):
1427         (bmalloc::Heap::deallocateMediumLine):
1428         (bmalloc::Heap::allocateMediumLine):
1429         * bmalloc/Line.h:
1430         (bmalloc::Line<Traits>::deref):
1431         * bmalloc/Mutex.cpp: Removed.
1432         * bmalloc/Mutex.h:
1433         (bmalloc::Mutex::Mutex):
1434         (bmalloc::Mutex::init): Deleted.
1435         (bmalloc::Mutex::try_lock): Deleted.
1436         (bmalloc::Mutex::lock): Deleted.
1437         (bmalloc::Mutex::unlock): Deleted.
1438         * bmalloc/Page.h:
1439         (bmalloc::Page<Traits>::ref):
1440         (bmalloc::Page<Traits>::deref):
1441         (bmalloc::Page<Traits>::refCount):
1442         * bmalloc/PerProcess.h:
1443         (bmalloc::PerProcess::mutex):
1444         (bmalloc::PerProcess<T>::getSlowCase):
1445         * bmalloc/StaticMutex.cpp: Added.
1446         (bmalloc::StaticMutex::lockSlowCase):
1447         * bmalloc/StaticMutex.h: Added.
1448         (bmalloc::StaticMutex::init):
1449         (bmalloc::StaticMutex::try_lock):
1450         (bmalloc::StaticMutex::lock):
1451         (bmalloc::StaticMutex::unlock):
1452         * bmalloc/VMHeap.h:
1453         (bmalloc::VMHeap::deallocateXSmallPage):
1454         (bmalloc::VMHeap::deallocateSmallPage):
1455         (bmalloc::VMHeap::deallocateMediumPage):
1456         (bmalloc::VMHeap::deallocateLargeRange):
1457         * bmalloc/bmalloc.h:
1458         (bmalloc::api::scavenge): Global replace Mutex => StaticMutex,
1459         since the Heap mutex is a static.
1460
1461 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
1462
1463         bmalloc: AsyncTask should use Mutex instead of std::mutex
1464         https://bugs.webkit.org/show_bug.cgi?id=131865
1465
1466         Reviewed by Gavin Barraclough.
1467
1468         std::mutex is so slow that it makes parallelizing simple tasks through
1469         AsyncTask a net regression. Mutex fixes this.
1470
1471         * bmalloc/AsyncTask.h:
1472         (bmalloc::Function>::AsyncTask):
1473         (bmalloc::Function>::join):
1474         (bmalloc::Function>::runSlowCase):
1475         (bmalloc::Function>::entryPoint):
1476         * bmalloc/Mutex.h:
1477         (bmalloc::Mutex::init):
1478
1479 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
1480
1481         bmalloc: Added an XSmall line size
1482         https://bugs.webkit.org/show_bug.cgi?id=131851
1483
1484         Reviewed by Sam Weinig.
1485
1486         Reduces malloc footprint on Membuster recordings by 10%.
1487
1488         This is a throughput regression, but we're still way ahead of TCMalloc.
1489         I have some ideas for how to recover the regression -- but I wanted to
1490         get this win in first.
1491
1492         Full set of benchmark results:
1493
1494                 bmalloc> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks --measure-heap nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1495                                                                                                 
1496                                                                nopatch                      patch                                Δ
1497                 Peak Memory:
1498                     reddit_memory_warning                      7,896kB                    7,532kB                  ^ 1.05x smaller
1499                     flickr_memory_warning                     12,968kB                   12,324kB                  ^ 1.05x smaller
1500                     theverge_memory_warning                   16,672kB                   15,200kB                   ^ 1.1x smaller
1501
1502                     <geometric mean>                          11,952kB                   11,216kB                  ^ 1.07x smaller
1503                     <arithmetic mean>                         12,512kB                   11,685kB                  ^ 1.07x smaller
1504                     <harmonic mean>                           11,375kB                   10,726kB                  ^ 1.06x smaller
1505
1506                 Memory at End:
1507                     reddit_memory_warning                      7,320kB                    6,856kB                  ^ 1.07x smaller
1508                     flickr_memory_warning                     10,848kB                    9,692kB                  ^ 1.12x smaller
1509                     theverge_memory_warning                   16,380kB                   14,872kB                   ^ 1.1x smaller
1510
1511                     <geometric mean>                          10,916kB                    9,961kB                   ^ 1.1x smaller
1512                     <arithmetic mean>                         11,516kB                   10,473kB                   ^ 1.1x smaller
1513                     <harmonic mean>                           10,350kB                    9,485kB                  ^ 1.09x smaller
1514
1515                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1516                                                                                                 
1517                                                    nopatch                patch                         Δ
1518                 Execution Time:
1519                     churn                            127ms                151ms            ! 1.19x slower
1520                     list_allocate                    130ms                164ms            ! 1.26x slower
1521                     tree_allocate                    109ms                127ms            ! 1.17x slower
1522                     tree_churn                       115ms                120ms            ! 1.04x slower
1523                     facebook                         240ms                259ms            ! 1.08x slower
1524                     fragment                          91ms                131ms            ! 1.44x slower
1525                     fragment_iterate                 105ms                106ms            ! 1.01x slower
1526                     message_one                      260ms                259ms             ^ 1.0x faster
1527                     message_many                     149ms                154ms            ! 1.03x slower
1528                     medium                           194ms                248ms            ! 1.28x slower
1529                     big                              157ms                160ms            ! 1.02x slower
1530
1531                     <geometric mean>                 144ms                163ms            ! 1.13x slower
1532                     <arithmetic mean>                152ms                171ms            ! 1.12x slower
1533                     <harmonic mean>                  137ms                156ms            ! 1.14x slower
1534
1535                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1536                                                                                                 
1537                                                                        nopatch                          patch                                     Δ
1538                 Execution Time:
1539                     churn                                                126ms                          148ms                        ! 1.17x slower
1540                     churn --parallel                                      62ms                           76ms                        ! 1.23x slower
1541                     list_allocate                                        130ms                          164ms                        ! 1.26x slower
1542                     list_allocate --parallel                             120ms                          175ms                        ! 1.46x slower
1543                     tree_allocate                                        111ms                          127ms                        ! 1.14x slower
1544                     tree_allocate --parallel                              95ms                          135ms                        ! 1.42x slower
1545                     tree_churn                                           115ms                          124ms                        ! 1.08x slower
1546                     tree_churn --parallel                                107ms                          126ms                        ! 1.18x slower
1547                     facebook                                             240ms                          276ms                        ! 1.15x slower
1548                     facebook --parallel                                  802ms                        1,088ms                        ! 1.36x slower
1549                     fragment                                              92ms                          130ms                        ! 1.41x slower
1550                     fragment --parallel                                   66ms                          124ms                        ! 1.88x slower
1551                     fragment_iterate                                     109ms                          127ms                        ! 1.17x slower
1552                     fragment_iterate --parallel                           55ms                           64ms                        ! 1.16x slower
1553                     message_one                                          260ms                          260ms                                      
1554                     message_many                                         170ms                          238ms                         ! 1.4x slower
1555                     medium                                               185ms                          250ms                        ! 1.35x slower
1556                     medium --parallel                                    210ms                          334ms                        ! 1.59x slower
1557                     big                                                  150ms                          169ms                        ! 1.13x slower
1558                     big --parallel                                       138ms                          144ms                        ! 1.04x slower
1559
1560                     <geometric mean>                                     135ms                          170ms                        ! 1.26x slower
1561                     <arithmetic mean>                                    167ms                          214ms                        ! 1.28x slower
1562                     <harmonic mean>                                      117ms                          148ms                        ! 1.26x slower
1563
1564                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
1565
1566                                                                     TC                      patch                                Δ
1567                 Peak Memory:
1568                     reddit_memory_warning                     13,836kB                   13,436kB                  ^ 1.03x smaller
1569                     flickr_memory_warning                     24,868kB                   25,188kB                   ! 1.01x bigger
1570                     theverge_memory_warning                   24,504kB                   26,636kB                   ! 1.09x bigger
1571
1572                     <geometric mean>                          20,353kB                   20,812kB                   ! 1.02x bigger
1573                     <arithmetic mean>                         21,069kB                   21,753kB                   ! 1.03x bigger
1574                     <harmonic mean>                           19,570kB                   19,780kB                   ! 1.01x bigger
1575
1576                 Memory at End:
1577                     reddit_memory_warning                      8,656kB                   10,016kB                   ! 1.16x bigger
1578                     flickr_memory_warning                     11,844kB                   13,784kB                   ! 1.16x bigger
1579                     theverge_memory_warning                   18,516kB                   22,748kB                   ! 1.23x bigger
1580
1581                     <geometric mean>                          12,382kB                   14,644kB                   ! 1.18x bigger
1582                     <arithmetic mean>                         13,005kB                   15,516kB                   ! 1.19x bigger
1583                     <harmonic mean>                           11,813kB                   13,867kB                   ! 1.17x bigger
1584
1585                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
1586                                                                                                 
1587                                                         TC                patch                         Δ
1588                 Execution Time:
1589                     churn                            416ms                148ms            ^ 2.81x faster
1590                     list_allocate                    463ms                164ms            ^ 2.82x faster
1591                     tree_allocate                    292ms                127ms             ^ 2.3x faster
1592                     tree_churn                       157ms                120ms            ^ 1.31x faster
1593                     facebook                         327ms                276ms            ^ 1.18x faster
1594                     fragment                         335ms                129ms             ^ 2.6x faster
1595                     fragment_iterate                 344ms                108ms            ^ 3.19x faster
1596                     message_one                      386ms                258ms             ^ 1.5x faster
1597                     message_many                     410ms                154ms            ^ 2.66x faster
1598                     medium                           391ms                245ms             ^ 1.6x faster
1599                     big                              261ms                167ms            ^ 1.56x faster
1600
1601                     <geometric mean>                 332ms                164ms            ^ 2.02x faster
1602                     <arithmetic mean>                344ms                172ms            ^ 1.99x faster
1603                     <harmonic mean>                  317ms                157ms            ^ 2.02x faster
1604
1605         * bmalloc.xcodeproj/project.pbxproj:
1606         * bmalloc/Allocator.cpp:
1607         (bmalloc::Allocator::Allocator): Don't assume that each allocator's
1608         index corresponds with its size. Instead, use the size selection function
1609         explicitly. Now that we have XSmall, some small allocator entries are
1610         unused.
1611
1612         (bmalloc::Allocator::scavenge):
1613         (bmalloc::Allocator::log):
1614         (bmalloc::Allocator::processXSmallAllocatorLog):
1615         (bmalloc::Allocator::allocateSlowCase):
1616         * bmalloc/Allocator.h:
1617         (bmalloc::Allocator::xSmallAllocatorFor):
1618         (bmalloc::Allocator::allocateFastCase):
1619         * bmalloc/Chunk.h:
1620         * bmalloc/Deallocator.cpp:
1621         (bmalloc::Deallocator::scavenge):
1622         (bmalloc::Deallocator::processObjectLog):
1623         (bmalloc::Deallocator::deallocateSlowCase):
1624         (bmalloc::Deallocator::deallocateXSmallLine):
1625         (bmalloc::Deallocator::allocateXSmallLine):
1626         * bmalloc/Deallocator.h:
1627         (bmalloc::Deallocator::deallocateFastCase):
1628         * bmalloc/Heap.cpp:
1629         (bmalloc::Heap::scavenge):
1630         (bmalloc::Heap::scavengeXSmallPages):
1631         (bmalloc::Heap::allocateXSmallLineSlowCase):
1632         * bmalloc/Heap.h:
1633         (bmalloc::Heap::deallocateXSmallLine):
1634         (bmalloc::Heap::allocateXSmallLine):
1635         * bmalloc/LargeChunk.h:
1636         (bmalloc::LargeChunk::get):
1637         (bmalloc::LargeChunk::endTag):
1638         * bmalloc/Line.h:
1639         * bmalloc/MediumAllocator.h:
1640         (bmalloc::MediumAllocator::allocate):
1641         (bmalloc::MediumAllocator::refill):
1642         * bmalloc/ObjectType.cpp:
1643         (bmalloc::objectType):
1644         * bmalloc/ObjectType.h:
1645         (bmalloc::isXSmall):
1646         (bmalloc::isSmall):
1647         (bmalloc::isMedium):
1648         (bmalloc::isLarge):
1649         (bmalloc::isSmallOrMedium): Deleted.
1650         * bmalloc/SegregatedFreeList.h: I boiler-plate copied existing code for
1651         handling small objects. There's probably a reasonable way to share this
1652         code in the future -- I'll look into that once it's stopped changing.
1653
1654         * bmalloc/Sizes.h: Tweaked size classes to make Membuster happy. This
1655         is the main reason things got slower.
1656
1657         * bmalloc/SmallAllocator.h:
1658         (bmalloc::SmallAllocator::allocate):
1659         * bmalloc/SmallTraits.h:
1660         * bmalloc/VMHeap.cpp:
1661         (bmalloc::VMHeap::allocateXSmallChunk):
1662         * bmalloc/VMHeap.h:
1663         (bmalloc::VMHeap::allocateXSmallPage):
1664         (bmalloc::VMHeap::deallocateXSmallPage):
1665         * bmalloc/XSmallAllocator.h: Added.
1666         (bmalloc::XSmallAllocator::isNull):
1667         (bmalloc::XSmallAllocator::canAllocate):
1668         (bmalloc::XSmallAllocator::XSmallAllocator):
1669         (bmalloc::XSmallAllocator::line):
1670         (bmalloc::XSmallAllocator::allocate):
1671         (bmalloc::XSmallAllocator::objectCount):
1672         (bmalloc::XSmallAllocator::derefCount):
1673         (bmalloc::XSmallAllocator::refill):
1674         * bmalloc/XSmallChunk.h: Added.
1675         * bmalloc/XSmallLine.h: Added.
1676         * bmalloc/XSmallPage.h: Added.
1677         * bmalloc/XSmallTraits.h: Added.
1678         * bmalloc/bmalloc.h:
1679         (bmalloc::api::realloc): Boiler-plate copy, as above.
1680
1681 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
1682
1683         MallocBench should scavenge explicitly instead of waiting
1684         https://bugs.webkit.org/show_bug.cgi?id=131661
1685
1686         Reviewed by Andreas Kling.
1687
1688         Added explicit scavenge support to bmalloc. This isn't a memory win,
1689         since bmalloc's per-thread cache is so small. But it makes testing
1690         simpler.
1691
1692         * bmalloc/Allocator.cpp:
1693         (bmalloc::Allocator::~Allocator):
1694         (bmalloc::Allocator::scavenge):
1695         * bmalloc/Allocator.h:
1696         * bmalloc/Cache.cpp:
1697         (bmalloc::Cache::operator new):
1698         (bmalloc::Cache::operator delete):
1699         (bmalloc::Cache::Cache):
1700         (bmalloc::Cache::scavenge):
1701         * bmalloc/Cache.h:
1702         * bmalloc/Deallocator.cpp:
1703         (bmalloc::Deallocator::~Deallocator):
1704         (bmalloc::Deallocator::scavenge):
1705         * bmalloc/Deallocator.h: Factored existing scavenging code into helper
1706         functions, for reuse.
1707
1708         * bmalloc/Heap.cpp:
1709         (bmalloc::sleep):
1710         (bmalloc::Heap::concurrentScavenge):
1711         (bmalloc::Heap::scavenge):
1712         (bmalloc::Heap::scavengeSmallPages):
1713         (bmalloc::Heap::scavengeMediumPages):
1714         (bmalloc::Heap::scavengeLargeRanges):
1715         * bmalloc/Heap.h: Made scavenge sleep duration a parameter. Forced
1716         scavenging -- in response to a benchmark or a low memory warning --
1717         wants to complete as soon as possible, so its sleep duration is 0.
1718
1719         * bmalloc/bmalloc.h:
1720         (bmalloc::api::scavenge):
1721         * bmalloc/mbmalloc.cpp: Exported the scavenge API for MallocBench's use.
1722
1723 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
1724
1725         Use 4kB pages on Mac
1726         https://bugs.webkit.org/show_bug.cgi?id=131658
1727
1728         Reviewed by Sam Weinig.
1729
1730         This reduces memory use a lot on Membuster:
1731
1732                                                                   base                      patch                                Δ
1733                 Execution Time:
1734                     reddit_memory_warning                         18ms                       17ms                   ^ 1.06x faster
1735                     flickr_memory_warning                         34ms                       36ms                   ! 1.06x slower
1736                     theverge_memory_warning                       39ms                       41ms                   ! 1.05x slower
1737
1738                     <geometric mean>                              29ms                       29ms                   ! 1.02x slower
1739                     <arithmetic mean>                             30ms                       31ms                   ! 1.03x slower
1740                     <harmonic mean>                               27ms                       27ms                    ^ 1.0x faster
1741
1742                 Peak Memory:
1743                     reddit_memory_warning                     16,412kB                   16,436kB                    ! 1.0x bigger
1744                     flickr_memory_warning                     30,120kB                   30,184kB                    ! 1.0x bigger
1745                     theverge_memory_warning                   33,408kB                   33,420kB                    ! 1.0x bigger
1746
1747                     <geometric mean>                          25,466kB                   25,499kB                    ! 1.0x bigger
1748                     <arithmetic mean>                         26,647kB                   26,680kB                    ! 1.0x bigger
1749                     <harmonic mean>                           24,181kB                   24,214kB                    ! 1.0x bigger
1750
1751                 Memory at End:
1752                     reddit_memory_warning                      2,404kB                    1,920kB                  ^ 1.25x smaller
1753                     flickr_memory_warning                      3,764kB                    3,072kB                  ^ 1.23x smaller
1754                     theverge_memory_warning                    3,648kB                    3,132kB                  ^ 1.16x smaller
1755
1756                     <geometric mean>                           3,208kB                    2,644kB                  ^ 1.21x smaller
1757                     <arithmetic mean>                          3,272kB                    2,708kB                  ^ 1.21x smaller
1758                     <harmonic mean>                            3,139kB                    2,574kB                  ^ 1.22x smaller
1759
1760
1761         * bmalloc.xcodeproj/project.pbxproj:
1762         * bmalloc/BPlatform.h: Added.
1763         * bmalloc/VMAllocate.h: Only use 16kB pages on iOS because the page size
1764         is 4kB on Mac.
1765
1766 2014-04-14  Alexey Proskuryakov  <ap@apple.com>
1767
1768         Fixed svn:ignore on bmalloc.xcodeproj, it had erroneous leading spaces.
1769
1770         * bmalloc.xcodeproj: Modified property svn:ignore.
1771
1772 2014-04-13  Geoffrey Garen  <ggaren@apple.com>
1773
1774         Fixed some mbmalloc exports
1775         https://bugs.webkit.org/show_bug.cgi?id=131599
1776
1777         Reviewed by Ryosuke Niwa.
1778
1779         * bmalloc.xcodeproj/project.pbxproj: Made some headers a private part
1780         of the project, so we can call them from API.
1781
1782         * bmalloc/mbmalloc.cpp: Marked the mbmalloc functions with default
1783         visibility, so they show up as exported in the .dylib.
1784
1785 2014-04-09  Geoffrey Garen  <ggaren@apple.com>
1786
1787         Put bmalloc headers in the right place
1788         https://bugs.webkit.org/show_bug.cgi?id=131464
1789
1790         Reviewed by Mark Rowe.
1791
1792         * Configurations/bmalloc.xcconfig: Set PRIVATE_HEADERS_FOLDER_PATH to
1793         specify that we don't just want to dump all of our generically-named
1794         headers into /usr/local/include.
1795
1796 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
1797
1798         Made bmalloc more #include friendly
1799         https://bugs.webkit.org/show_bug.cgi?id=131386
1800
1801         Reviewed by Andreas Kling.
1802
1803         Marked a bunch of headers private so they can be used from client code
1804         that #includes bmalloc.h.
1805
1806         Renamed ASSERT macros to BASSERT. This matches their header, which already
1807         had to be renamed, and fixes conflicts with WTF's ASSERT macros.
1808
1809         * bmalloc.xcodeproj/project.pbxproj:
1810         * bmalloc/Allocator.cpp:
1811         (bmalloc::Allocator::allocateSlowCase):
1812         * bmalloc/AsyncTask.h:
1813         (bmalloc::Function>::runSlowCase):
1814         * bmalloc/BAssert.h:
1815         * bmalloc/BoundaryTag.h:
1816         (bmalloc::BoundaryTag::setSize):
1817         * bmalloc/BoundaryTagInlines.h:
1818         (bmalloc::validate):
1819         (bmalloc::BoundaryTag::init):
1820         (bmalloc::BoundaryTag::deallocate):
1821         (bmalloc::BoundaryTag::splitLarge):
1822         (bmalloc::BoundaryTag::allocate):
1823         * bmalloc/Chunk.h:
1824         * bmalloc/Deallocator.cpp:
1825         (bmalloc::Deallocator::processObjectLog):
1826         (bmalloc::Deallocator::deallocateSlowCase):
1827         * bmalloc/Deallocator.h:
1828         (bmalloc::Deallocator::deallocateFastCase):
1829         * bmalloc/FixedVector.h:
1830         (bmalloc::Capacity>::operator):
1831         (bmalloc::Capacity>::push):
1832         (bmalloc::Capacity>::pop):
1833         (bmalloc::Capacity>::shrink):
1834         * bmalloc/Heap.cpp:
1835         (bmalloc::Heap::allocateLarge):
1836         * bmalloc/LargeChunk.h:
1837         (bmalloc::LargeChunk::get):
1838         (bmalloc::LargeChunk::endTag):
1839         * bmalloc/Line.h:
1840         (bmalloc::Line<Traits>::concurrentRef):
1841         (bmalloc::Line<Traits>::deref):
1842         * bmalloc/MediumAllocator.h:
1843         (bmalloc::MediumAllocator::allocate):
1844         * bmalloc/ObjectType.h:
1845         (bmalloc::isSmall):
1846         * bmalloc/Page.h:
1847         (bmalloc::Page<Traits>::ref):
1848         (bmalloc::Page<Traits>::deref):
1849         * bmalloc/PerThread.h:
1850         (bmalloc::PerThread<T>::getSlowCase):
1851         * bmalloc/SegregatedFreeList.cpp:
1852         (bmalloc::SegregatedFreeList::SegregatedFreeList):
1853         (bmalloc::SegregatedFreeList::insert):
1854         * bmalloc/SmallAllocator.h:
1855         (bmalloc::SmallAllocator::allocate):
1856         (bmalloc::SmallAllocator::refill):
1857         * bmalloc/Syscall.h:
1858         * bmalloc/VMAllocate.h:
1859         (bmalloc::vmValidate):
1860         (bmalloc::vmAllocate):
1861         (bmalloc::vmDeallocatePhysicalPagesSloppy):
1862         * bmalloc/Vector.h:
1863         (bmalloc::Vector<T>::operator):
1864         (bmalloc::Vector<T>::pop):
1865         (bmalloc::Vector<T>::shrink):
1866         * bmalloc/XLargeChunk.h:
1867         (bmalloc::XLargeChunk::range):
1868         (bmalloc::XLargeChunk::size):
1869
1870 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
1871
1872         Removed an unused file.
1873
1874         Unreviewed.
1875
1876         * bmalloc/AsyncTask.cpp: Removed.
1877
1878 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1879
1880         Build bmalloc on Mac
1881         https://bugs.webkit.org/show_bug.cgi?id=131333
1882
1883         Reviewed by Mark Rowe.
1884
1885         * Makefile: Added. For make clients.
1886
1887         These files are required for building any project in WebKit. I copied
1888         them from WTF:
1889         * Configurations: Added.
1890         * Configurations/Base.xcconfig: Added.
1891         * Configurations/DebugRelease.xcconfig: Added.
1892         * Configurations/bmalloc.xcconfig: Added.
1893         * Configurations/iOS.xcconfig: Added.
1894         * Configurations/mbmalloc.xcconfig: Added.
1895
1896         * bmalloc.xcodeproj/project.pbxproj: I removed per-project-file stuff
1897         from here because everything is in .xcconfig files now.
1898
1899         I had to fix a bunch of minor warnings, since they're enabled in our
1900         .xcconfig files:
1901
1902         * bmalloc/AsyncTask.h:
1903         (bmalloc::Function>::AsyncTask):
1904         * bmalloc/BAssert.h:
1905         * bmalloc/BoundaryTagInlines.h:
1906         (bmalloc::validate):
1907         * bmalloc/Heap.cpp:
1908         (bmalloc::Heap::Heap):
1909         (bmalloc::Heap::allocateLarge):
1910         (bmalloc::Heap::deallocateLarge):
1911         * bmalloc/Mutex.h:
1912         (bmalloc::Mutex::Mutex): Deleted.
1913         * bmalloc/VMAllocate.h:
1914         (bmalloc::vmValidate):
1915         * bmalloc/mbmalloc.cpp:
1916
1917 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1918
1919         bmalloc: Fixed a leak in the per-thread cache
1920         https://bugs.webkit.org/show_bug.cgi?id=131330
1921
1922         Reviewed by Andreas Kling.
1923
1924         Remember to deallocate our line caches upon thread exit.
1925
1926         * bmalloc/Deallocator.cpp:
1927         (bmalloc::Deallocator::~Deallocator):
1928
1929 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1930
1931         bmalloc: rolled out the tryLock experiment
1932         https://bugs.webkit.org/show_bug.cgi?id=131328
1933
1934         Reviewed by Andreas Kling.
1935
1936         It wasn't a speedup.
1937
1938         * bmalloc.xcodeproj/project.pbxproj:
1939         * bmalloc/Allocator.cpp:
1940         (bmalloc::Allocator::processSmallAllocatorLog):
1941         (bmalloc::Allocator::processMediumAllocatorLog):
1942         * bmalloc/Deallocator.cpp:
1943         (bmalloc::Deallocator::processObjectLog):
1944         (bmalloc::Deallocator::deallocateSlowCase):
1945         (bmalloc::Deallocator::deallocateSmallLine):
1946         (bmalloc::Deallocator::deallocateMediumLine):
1947         * bmalloc/Deallocator.h:
1948         (bmalloc::Deallocator::deallocateFastCase):
1949         * bmalloc/Heap.h:
1950         (bmalloc::Heap::deallocateSmallLine):
1951         (bmalloc::Heap::deallocateMediumLine):
1952         * bmalloc/Line.h:
1953         (bmalloc::Line<Traits>::deref):
1954         * bmalloc/Page.h:
1955         (bmalloc::Page<Traits>::deref):
1956
1957 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1958
1959         bmalloc
1960         https://bugs.webkit.org/show_bug.cgi?id=131170
1961
1962         Reviewed by Andreas Kling.
1963
1964         Initial commit.
1965
1966         * bmalloc: Added.
1967         * bmalloc.xcodeproj: Added.
1968         * bmalloc.xcodeproj/project.pbxproj: Added.
1969         * bmalloc/Algorithm.h: Added.
1970         (bmalloc::max):
1971         (bmalloc::min):
1972         (bmalloc::mask):
1973         (bmalloc::test):
1974         (bmalloc::roundUpToMultipleOf):
1975         (bmalloc::roundDownToMultipleOf):
1976         (bmalloc::sizeOf):
1977         (bmalloc::bitCount):
1978         (bmalloc::isPowerOfTwo):
1979         * bmalloc/Allocator.cpp: Added.
1980         (bmalloc::Allocator::Allocator):
1981         (bmalloc::Allocator::~Allocator):
1982         (bmalloc::Allocator::log):
1983         (bmalloc::Allocator::processSmallAllocatorLog):
1984         (bmalloc::Allocator::processMediumAllocatorLog):
1985         (bmalloc::Allocator::allocateLarge):
1986         (bmalloc::Allocator::allocateXLarge):
1987         (bmalloc::Allocator::allocateMedium):
1988         (bmalloc::Allocator::allocateSlowCase):
1989         * bmalloc/Allocator.h: Added.
1990         (bmalloc::Allocator::smallAllocatorFor):
1991         (bmalloc::Allocator::allocateFastCase):
1992         (bmalloc::Allocator::allocate):
1993         * bmalloc/AsyncTask.cpp: Added.
1994         (bmalloc::AsyncTask<Function>::runSlowCase):
1995         (bmalloc::AsyncTask<Function>::pthreadEntryPoint):
1996         (bmalloc::AsyncTask<Function>::entryPoint):
1997         * bmalloc/AsyncTask.h: Added.
1998         (bmalloc::Function>::AsyncTask):
1999         (bmalloc::Function>::join):
2000         (bmalloc::Function>::run):
2001         (bmalloc::Function>::runSlowCase):
2002         (bmalloc::Function>::pthreadEntryPoint):
2003         (bmalloc::Function>::entryPoint):
2004         * bmalloc/BAssert.h: Added.
2005         * bmalloc/BeginTag.h: Added.
2006         (bmalloc::BeginTag::isInFreeList):
2007         * bmalloc/BoundaryTag.h: Added.
2008         (bmalloc::BoundaryTag::isXLarge):
2009         (bmalloc::BoundaryTag::setXLarge):
2010         (bmalloc::BoundaryTag::isFree):
2011         (bmalloc::BoundaryTag::setFree):
2012         (bmalloc::BoundaryTag::isEnd):
2013         (bmalloc::BoundaryTag::setEnd):
2014         (bmalloc::BoundaryTag::hasPhysicalPages):
2015         (bmalloc::BoundaryTag::setHasPhysicalPages):
2016         (bmalloc::BoundaryTag::isNull):
2017         (bmalloc::BoundaryTag::clear):
2018         (bmalloc::BoundaryTag::size):
2019         (bmalloc::BoundaryTag::setSize):
2020         (bmalloc::BoundaryTag::prev):
2021         (bmalloc::BoundaryTag::next):
2022         * bmalloc/BoundaryTagInlines.h: Added.
2023         (bmalloc::validate):
2024         (bmalloc::validatePrev):
2025         (bmalloc::validateNext):
2026         (bmalloc::BoundaryTag::init):
2027         (bmalloc::BoundaryTag::mergeLargeLeft):
2028         (bmalloc::BoundaryTag::mergeLargeRight):
2029         (bmalloc::BoundaryTag::mergeLarge):
2030         (bmalloc::BoundaryTag::deallocate):
2031         (bmalloc::BoundaryTag::splitLarge):
2032         (bmalloc::BoundaryTag::allocate):
2033         * bmalloc/Cache.cpp: Added.
2034         (bmalloc::Cache::operator new):
2035         (bmalloc::Cache::operator delete):
2036         (bmalloc::Cache::Cache):
2037         (bmalloc::Cache::allocateSlowCase):
2038         (bmalloc::Cache::allocateSlowCaseNullCache):
2039         (bmalloc::Cache::deallocateSlowCase):
2040         (bmalloc::Cache::deallocateSlowCaseNullCache):
2041         * bmalloc/Cache.h: Added.
2042         (bmalloc::Cache::allocator):
2043         (bmalloc::Cache::deallocator):
2044         (bmalloc::Cache::allocateFastCase):
2045         (bmalloc::Cache::deallocateFastCase):
2046         (bmalloc::Cache::allocate):
2047         (bmalloc::Cache::deallocate):
2048         * bmalloc/Chunk.h: Added.
2049         (bmalloc::Chunk::begin):
2050         (bmalloc::Chunk::end):
2051         (bmalloc::Chunk::lines):
2052         (bmalloc::Chunk::pages):
2053         * bmalloc/Deallocator.cpp: Added.
2054         (bmalloc::Deallocator::Deallocator):
2055         (bmalloc::Deallocator::~Deallocator):
2056         (bmalloc::Deallocator::deallocateLarge):
2057         (bmalloc::Deallocator::deallocateXLarge):
2058         (bmalloc::Deallocator::processObjectLog):
2059         (bmalloc::Deallocator::deallocateSlowCase):
2060         (bmalloc::Deallocator::deallocateSmallLine):
2061         (bmalloc::Deallocator::allocateSmallLine):
2062         (bmalloc::Deallocator::deallocateMediumLine):
2063         (bmalloc::Deallocator::allocateMediumLine):
2064         * bmalloc/Deallocator.h: Added.
2065         (bmalloc::Deallocator::deallocateFastCase):
2066         (bmalloc::Deallocator::deallocate):
2067         * bmalloc/EndTag.h: Added.
2068         (bmalloc::EndTag::operator=):
2069         * bmalloc/FixedVector.h: Added.
2070         (bmalloc::FixedVector::begin):
2071         (bmalloc::FixedVector::end):
2072         (bmalloc::FixedVector::size):
2073         (bmalloc::FixedVector::capacity):
2074         (bmalloc::FixedVector::clear):
2075         (bmalloc::FixedVector::isEmpty):
2076         (bmalloc::Capacity>::FixedVector):
2077         (bmalloc::Capacity>::operator):
2078         (bmalloc::Capacity>::push):
2079         (bmalloc::Capacity>::pop):
2080         (bmalloc::Capacity>::shrink):
2081         * bmalloc/Heap.cpp: Added.
2082         (bmalloc::sleep):
2083         (bmalloc::Heap::Heap):
2084         (bmalloc::Heap::concurrentScavenge):
2085         (bmalloc::Heap::scavengeSmallPages):
2086         (bmalloc::Heap::scavengeMediumPages):
2087         (bmalloc::Heap::scavengeLargeRanges):
2088         (bmalloc::Heap::allocateSmallLineSlowCase):
2089         (bmalloc::Heap::allocateMediumLineSlowCase):
2090         (bmalloc::Heap::allocateXLarge):
2091         (bmalloc::Heap::deallocateXLarge):
2092         (bmalloc::Heap::allocateLarge):
2093         (bmalloc::Heap::deallocateLarge):
2094         * bmalloc/Heap.h: Added.
2095         (bmalloc::Heap::deallocateSmallLine):
2096         (bmalloc::Heap::allocateSmallLine):
2097         (bmalloc::Heap::deallocateMediumLine):
2098         (bmalloc::Heap::allocateMediumLine):
2099         * bmalloc/Inline.h: Added.
2100         * bmalloc/LargeChunk.h: Added.
2101         (bmalloc::LargeChunk::begin):
2102         (bmalloc::LargeChunk::end):
2103         (bmalloc::LargeChunk::create):
2104         (bmalloc::LargeChunk::get):
2105         (bmalloc::LargeChunk::beginTag):
2106         (bmalloc::LargeChunk::endTag):
2107         * bmalloc/Line.h: Added.
2108         (bmalloc::Line<Traits>::begin):
2109         (bmalloc::Line<Traits>::end):
2110         (bmalloc::Line<Traits>::concurrentRef):
2111         (bmalloc::Line<Traits>::deref):
2112         * bmalloc/MediumAllocator.h: Added.
2113         (bmalloc::MediumAllocator::isNull):
2114         (bmalloc::MediumAllocator::MediumAllocator):
2115         (bmalloc::MediumAllocator::line):
2116         (bmalloc::MediumAllocator::allocate):
2117         (bmalloc::MediumAllocator::derefCount):
2118         (bmalloc::MediumAllocator::refill):
2119         * bmalloc/MediumChunk.h: Added.
2120         * bmalloc/MediumLine.h: Added.
2121         * bmalloc/MediumPage.h: Added.
2122         * bmalloc/MediumTraits.h: Added.
2123         * bmalloc/Mutex.cpp: Added.
2124         (bmalloc::Mutex::lockSlowCase):
2125         * bmalloc/Mutex.h: Added.
2126         (bmalloc::Mutex::Mutex):
2127         (bmalloc::Mutex::try_lock):
2128         (bmalloc::Mutex::lock):
2129         (bmalloc::Mutex::unlock):
2130         * bmalloc/ObjectType.cpp: Added.
2131         (bmalloc::objectType):
2132         * bmalloc/ObjectType.h: Added.
2133         (bmalloc::isSmallOrMedium):
2134         (bmalloc::isSmall):
2135         * bmalloc/Page.h: Added.
2136         (bmalloc::Page<Traits>::ref):
2137         (bmalloc::Page<Traits>::deref):
2138         (bmalloc::Page<Traits>::refCount):
2139         * bmalloc/PerProcess.h: Added.
2140         (bmalloc::PerProcess::mutex):
2141         (bmalloc::PerProcess<T>::getFastCase):
2142         (bmalloc::PerProcess<T>::get):
2143         (bmalloc::PerProcess<T>::getSlowCase):
2144         * bmalloc/PerThread.h: Added.
2145         (bmalloc::PerThreadStorage<Cache>::get):
2146         (bmalloc::PerThreadStorage<Cache>::init):
2147         (bmalloc::PerThreadStorage::get):
2148         (bmalloc::PerThreadStorage::init):
2149         (bmalloc::PerThread<T>::getFastCase):
2150         (bmalloc::PerThread<T>::get):
2151         (bmalloc::PerThread<T>::destructor):
2152         (bmalloc::PerThread<T>::getSlowCase):
2153         * bmalloc/Range.h: Added.
2154         (bmalloc::Range::Range):
2155         (bmalloc::Range::begin):
2156         (bmalloc::Range::end):
2157         (bmalloc::Range::size):
2158         (bmalloc::Range::operator!):
2159         (bmalloc::Range::operator<):
2160         * bmalloc/SegregatedFreeList.cpp: Added.
2161         (bmalloc::SegregatedFreeList::SegregatedFreeList):
2162         (bmalloc::SegregatedFreeList::insert):
2163         (bmalloc::SegregatedFreeList::takeGreedy):
2164         (bmalloc::SegregatedFreeList::take):
2165         * bmalloc/SegregatedFreeList.h: Added.
2166         * bmalloc/Sizes.h: Added.
2167         * bmalloc/SmallAllocator.h: Added.
2168         (bmalloc::SmallAllocator::isNull):
2169         (bmalloc::SmallAllocator::canAllocate):
2170         (bmalloc::SmallAllocator::SmallAllocator):
2171         (bmalloc::SmallAllocator::line):
2172         (bmalloc::SmallAllocator::allocate):
2173         (bmalloc::SmallAllocator::objectCount):
2174         (bmalloc::SmallAllocator::derefCount):
2175         (bmalloc::SmallAllocator::refill):
2176         * bmalloc/SmallChunk.h: Added.
2177         * bmalloc/SmallLine.h: Added.
2178         * bmalloc/SmallPage.h: Added.
2179         * bmalloc/SmallTraits.h: Added.
2180         * bmalloc/Syscall.h: Added.
2181         * bmalloc/VMAllocate.h: Added.
2182         (bmalloc::vmSize):
2183         (bmalloc::vmValidate):
2184         (bmalloc::vmAllocate):
2185         (bmalloc::vmDeallocate):
2186         (bmalloc::vmDeallocatePhysicalPages):
2187         (bmalloc::vmAllocatePhysicalPages):
2188         (bmalloc::vmDeallocatePhysicalPagesSloppy):
2189         (bmalloc::vmAllocatePhysicalPagesSloppy):
2190         * bmalloc/VMHeap.cpp: Added.
2191         (bmalloc::VMHeap::VMHeap):
2192         (bmalloc::VMHeap::allocateSmallChunk):
2193         (bmalloc::VMHeap::allocateMediumChunk):
2194         (bmalloc::VMHeap::allocateLargeChunk):
2195         * bmalloc/VMHeap.h: Added.
2196         (bmalloc::VMHeap::allocateSmallPage):
2197         (bmalloc::VMHeap::allocateMediumPage):
2198         (bmalloc::VMHeap::allocateLargeRange):
2199         (bmalloc::VMHeap::deallocateSmallPage):
2200         (bmalloc::VMHeap::deallocateMediumPage):
2201         (bmalloc::VMHeap::deallocateLargeRange):
2202         * bmalloc/Vector.h: Added.
2203         (bmalloc::Vector::begin):
2204         (bmalloc::Vector::end):
2205         (bmalloc::Vector::size):
2206         (bmalloc::Vector::capacity):
2207         (bmalloc::Vector::last):
2208         (bmalloc::Vector::pop):
2209         (bmalloc::Vector<T>::Vector):
2210         (bmalloc::Vector<T>::~Vector):
2211         (bmalloc::Vector<T>::operator):
2212         (bmalloc::Vector<T>::push):
2213         (bmalloc::Vector<T>::pop):
2214         (bmalloc::Vector<T>::shrink):
2215         (bmalloc::Vector<T>::reallocateBuffer):
2216         (bmalloc::Vector<T>::shrinkCapacity):
2217         (bmalloc::Vector<T>::growCapacity):
2218         * bmalloc/XLargeChunk.h: Added.
2219         (bmalloc::XLargeChunk::get):
2220         (bmalloc::XLargeChunk::begin):
2221         (bmalloc::XLargeChunk::XLargeChunk):
2222         (bmalloc::XLargeChunk::create):
2223         (bmalloc::XLargeChunk::destroy):
2224         (bmalloc::XLargeChunk::range):
2225         (bmalloc::XLargeChunk::size):
2226         * bmalloc/bmalloc.h: Added.
2227         (bmalloc::api::malloc):
2228         (bmalloc::api::free):
2229         (bmalloc::api::realloc):
2230         * bmalloc/mbmalloc.cpp: Added.
2231