41e8c5dea7f35971f11641f3d7d88a8c1e386749
[WebKit-https.git] / Source / bmalloc / ChangeLog
1 2015-02-19  Geoffrey Garen  <ggaren@apple.com>
2
3         bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
4         https://bugs.webkit.org/show_bug.cgi?id=141802
5
6         Reviewed by Andreas Kling.
7
8         This patch does the bare minimum to stop false positive leaks from
9         being reported by the Darwin leaks tool. We register each super chunk
10         as a single object, and then request that the leaks tool scan it.
11
12         * bmalloc.xcodeproj/project.pbxproj: Added an abstraction for the malloc
13         zone introspection API.
14
15         * bmalloc/Algorithm.h: Missing #include.
16
17         * bmalloc/VMHeap.cpp:
18         (bmalloc::VMHeap::grow):
19         * bmalloc/VMHeap.h: Adopt the new abstraction.
20
21         * bmalloc/Zone.cpp: Added.
22         (bmalloc::remoteRead): Helper for reading an object out of another process.
23         (bmalloc::Zone::enumerator):
24         (bmalloc::Zone::Zone): Register a malloc zone so that we will participate
25         in introspection.
26
27         * bmalloc/Zone.h: Added.
28         (bmalloc::Zone::superChunks):
29         (bmalloc::Zone::addSuperChunk): Use a non-dynamically-allocated vector
30         since our dynamic allocations will not be scanned by leaks since they
31         will have the malloc VM tag.
32
33 2015-02-18  Geoffrey Garen  <ggaren@apple.com>
34
35         bmalloc: VMHeap should keep a record of all of its VM ranges (for malloc introspection)
36         https://bugs.webkit.org/show_bug.cgi?id=141759
37
38         Reviewed by Andreas Kling.
39
40         * bmalloc.xcodeproj/project.pbxproj:
41         * bmalloc/SuperChunk.h: Added.
42         (bmalloc::SuperChunk::create):
43         (bmalloc::SuperChunk::SuperChunk):
44         (bmalloc::SuperChunk::smallChunk):
45         (bmalloc::SuperChunk::mediumChunk):
46         (bmalloc::SuperChunk::largeChunk): Factored out super chunk creation
47         into a separate class, for clarity and type safety.
48
49         * bmalloc/VMHeap.cpp:
50         (bmalloc::VMHeap::grow):
51         (bmalloc::VMHeap::allocateSuperChunk): Renamed "allocateSuperChunk" to
52         "grow" because Andreas found "allocateSuperChunk" to be unclear.
53
54         * bmalloc/VMHeap.h: Track all our VM ranges. We will use this information
55         for malloc introspection.
56
57         (bmalloc::VMHeap::allocateSmallPage):
58         (bmalloc::VMHeap::allocateMediumPage):
59         (bmalloc::VMHeap::allocateLargeRange): Updated for renames.
60
61 2015-02-18  Zan Dobersek  <zdobersek@igalia.com>
62
63         Build bmalloc through CMake as a static library. It's then linked either
64         into the WTF library (if built as a shared library) or into the JSC and
65         WebKit2 libraries. There's no need to build it as a standalone shared library.
66
67         Rubber-stamped by Carlos Garcia Campos.
68
69         * CMakeLists.txt:
70
71 2015-02-13  Gyuyoung Kim  <gyuyoung.kim@samsung.com>
72
73         [BMalloc] Add a FIXME comment for memory alignas
74         https://bugs.webkit.org/show_bug.cgi?id=141556
75
76         Reviewed by Csaba Osztrogonác.
77
78         * bmalloc/Chunk.h: Add a FIXME comment.
79         * bmalloc/LargeChunk.h: ditto.
80
81 2015-02-11  Csaba Osztrogonác  <ossy@webkit.org>
82
83         bmalloc buildfix on 32 bit Linux (x86/ARM)
84         https://bugs.webkit.org/show_bug.cgi?id=141472
85
86         Reviewed by Gyuyoung Kim.
87
88         * bmalloc/Algorithm.h:
89         (bmalloc::roundUpToMultipleOf):
90         * bmalloc/FixedVector.h:
91         (bmalloc::FixedVector::clear):
92         * bmalloc/Sizes.h:
93         (bmalloc::Sizes::sizeClass):
94
95 2015-02-11  Gyuyoung Kim  <gyuyoung.kim@samsung.com>
96
97         [EFL][GTK] Use bmalloc instead of tcmalloc
98         https://bugs.webkit.org/show_bug.cgi?id=140162
99
100         Reviewed by Carlos Garcia Campos.
101
102         Support to use bmalloc on EFL and GTK ports.
103
104         * CMakeLists.txt: Added.
105         * bmalloc/Allocator.cpp:
106         (bmalloc::Allocator::allocate):
107             Fix unused return value caused by posix_memalign().
108         * bmalloc/AsyncTask.h:
109         * bmalloc/BoundaryTag.h:
110         (bmalloc::BoundaryTag::clear):
111         * bmalloc/Chunk.h:
112             Change Traits::Page with Traits::PageType in order to fix
113             -fpermitive build error on EFL and GTK port. 
114         * bmalloc/EndTag.h:
115         (bmalloc::EndTag::operator=):
116         * bmalloc/Line.h: ditto.
117         * bmalloc/MediumTraits.h:
118         * bmalloc/Page.h: ditto.
119         * bmalloc/PerThread.h:
120             EFL port doesn't support __has_include definition yet.
121             Define HAVE_PTHREAD_MACHDEP_H according to check if __has_include is supported.
122         * bmalloc/SmallTraits.h: ditto.
123         * bmalloc/VMAllocate.h:
124         (bmalloc::vmDeallocatePhysicalPages):
125         (bmalloc::vmAllocatePhysicalPages):
126         * bmalloc/Vector.h:
127         (bmalloc::Vector<T>::push):
128         (bmalloc::Vector<T>::reallocateBuffer):
129
130 2015-01-31  Sam Weinig  <sam@webkit.org>
131
132         Remove even more Mountain Lion support
133         https://bugs.webkit.org/show_bug.cgi?id=141124
134
135         Reviewed by Alexey Proskuryakov.
136
137         * Configurations/Base.xcconfig:
138         * Configurations/DebugRelease.xcconfig:
139
140 2015-01-30  Geoffrey Garen  <ggaren@apple.com>
141
142         GC marking threads should clear malloc caches
143         https://bugs.webkit.org/show_bug.cgi?id=141097
144
145         Reviewed by Andreas Kling.
146
147         Split the scavenging API into per-thread vs global, so that you can
148         request to scavenge your own thread without scavenging the whole heap.
149
150         * bmalloc/Cache.cpp:
151         (bmalloc::Cache::scavenge):
152         * bmalloc/bmalloc.h:
153         (bmalloc::api::scavengeThisThread):
154         (bmalloc::api::scavenge):
155
156 2015-01-28  Dana Burkart  <dburkart@apple.com>
157
158         Move ASan flag settings from DebugRelease.xcconfig to Base.xcconfig
159         https://bugs.webkit.org/show_bug.cgi?id=136765
160
161         Reviewed by Alexey Proskuryakov.
162
163         * Configurations/Base.xcconfig:
164         * Configurations/DebugRelease.xcconfig:
165
166 2015-01-21  Geoffrey Garen  <ggaren@apple.com>
167
168         bmalloc: support aligned allocation
169         https://bugs.webkit.org/show_bug.cgi?id=140732
170
171         Reviewed by Andreas Kling.
172
173         * bmalloc/Allocator.cpp:
174         (bmalloc::Allocator::allocate): New function for aligned allocation.
175
176         Small and medium requests just allocate and free until they find an
177         aligned pointer. This is slightly inefficient in the worst case, but
178         still constant-time with little-to-no space overhead.
179
180         Large requests use a new API that requires the client to specify both
181         its ideal size and alignment, and the worst-case size you would have to
182         allocate in order to produce some interior pointer of the requested size
183         and alignment. We put the burden of this calculation on the client
184         because it simplifies things if we guarantee that allocation won't fail.
185
186         XLarge requests are easy: we just forward them to vmAllocate, which
187         already supported aligned requests.
188
189         * bmalloc/BoundaryTag.h:
190         * bmalloc/BoundaryTagInlines.h:
191         (bmalloc::BoundaryTag::mergeLeft):
192         (bmalloc::BoundaryTag::mergeRight):
193         (bmalloc::BoundaryTag::merge):
194         (bmalloc::BoundaryTag::deallocate):
195         (bmalloc::BoundaryTag::split):
196         (bmalloc::BoundaryTag::allocate): No behavior change here. I just
197         refactored the interface to remove some reference out parameters in
198         order to clarify what changes and what doesn't.
199
200         * bmalloc/Heap.cpp:
201         (bmalloc::Heap::allocateXLarge): Added an alignment API.
202
203         (bmalloc::Heap::allocateLarge):
204         * bmalloc/Heap.h: Added an alignment API. I split out allocateLarge into
205         a few variants, so aligned and unaligned allocation could share some code.
206
207         * bmalloc/SegregatedFreeList.cpp:
208         (bmalloc::SegregatedFreeList::take):
209         * bmalloc/SegregatedFreeList.h: Changed to use a separate, explicit API
210         for aligned allocation. It turns out that the aligned path is pretty
211         different, since it ends up searching for two potential ways to satisfy
212         an allocation: either large enough and aligned, or large enough to split
213         into something not aligned and something large enough and aligned.
214
215         * bmalloc/VMAllocate.h:
216         (bmalloc::vmAllocate): Switched alignment to come before size because
217         that's how the memalign API specifies it.
218
219         * bmalloc/VMHeap.h:
220         (bmalloc::VMHeap::allocateLargeRange): Added an alignment API.
221
222 2015-01-20  Geoffrey Garen  <ggaren@apple.com>
223
224         bmalloc: a little bit of cleanup
225         https://bugs.webkit.org/show_bug.cgi?id=140687
226
227         Reviewed by Anders Carlsson.
228
229         * bmalloc/Algorithm.h:
230         (bmalloc::isPowerOfTwo): Added a check for 0, since 0 would break a lot
231         of code.
232
233         * bmalloc/BoundaryTag.h:
234         * bmalloc/BoundaryTagInlines.h:
235         (bmalloc::BoundaryTag::mergeLeft):
236         (bmalloc::BoundaryTag::mergeRight):
237         (bmalloc::BoundaryTag::merge):
238         (bmalloc::BoundaryTag::deallocate):
239         (bmalloc::BoundaryTag::split):
240         (bmalloc::BoundaryTag::allocate):
241         (bmalloc::BoundaryTag::mergeLargeLeft): Deleted.
242         (bmalloc::BoundaryTag::mergeLargeRight): Deleted.
243         (bmalloc::BoundaryTag::mergeLarge): Deleted.
244         (bmalloc::BoundaryTag::splitLarge): Deleted. Removed the word "Large"
245         from all these functions, since boundary tags always pertain to large
246         objects, and putting the word "Large" everywhere wasn't helping to
247         explain that.
248
249         * bmalloc/Heap.cpp:
250         (bmalloc::Heap::allocateXLarge):
251         (bmalloc::Heap::findXLarge):
252         (bmalloc::Heap::deallocateXLarge):
253         * bmalloc/Heap.h:
254         * bmalloc/VMHeap.h:
255         (bmalloc::VMHeap::allocateXLarge): Deleted.
256         (bmalloc::VMHeap::findXLarge): Deleted.
257         (bmalloc::VMHeap::deallocateXLarge): Deleted. Moved XLarge allocation
258         from VMHeap to Heap. Since the purpose of the VMHeap is to cache VM
259         ranges, and the VMHeap never caches any XLarge ranges, it doesn't
260         really make sense for the VMHeap to be involved.
261
262 2015-01-16  Geoffrey Garen  <ggaren@apple.com>
263
264         bmalloc: refactored XLarge allocation for better alignment
265         https://bugs.webkit.org/show_bug.cgi?id=140582
266
267         Reviewed by Andreas Kling.
268
269         XLarge objects used to be Large objects with an extra bit of metadata
270         that said "actually, I'm not large -- I'm extra large".
271
272         The metadata header in an XLarge allocation made it impossible for the
273         XLarge object to honor a very large alignment request.
274
275         The solution is to stop using a metadata header for XLarge objects, and
276         instead to store explicit metadata on the side.
277
278         This is a bit less astonishing, which is also nice.
279
280         Finding XLarge metadata is now a linear search. That's probably OK, since
281         it was always so in TCMalloc, and the usual number of XLarge allocations
282         in a process is 0.
283
284         This design makes it possible for the heap to cache XLarge allocations
285         with and/or without physical pages. I haven't actually done that yet
286         because the tradeoffs are subtle, so I don't want to do anything without
287         a motivating test case.
288
289         * bmalloc.xcodeproj/project.pbxproj:
290         * bmalloc/Allocator.cpp:
291         (bmalloc::Allocator::reallocate): Removed the concept of an XLargeChunk,
292         since an XLarge allocation is now just a naked buffer without a header.
293
294         (bmalloc::Allocator::allocateXLarge): Added an explicit qualifier for
295         XLarge alignment, since XLargeChunk won't give this to us implicitly
296         anymore.
297
298         * bmalloc/BoundaryTag.h:
299         (bmalloc::BoundaryTag::setRange):
300         (bmalloc::BoundaryTag::isXLarge): Deleted.
301         (bmalloc::BoundaryTag::setXLarge): Deleted.
302         * bmalloc/BoundaryTagInlines.h:
303         (bmalloc::validate):
304         (bmalloc::BoundaryTag::deallocate): Removed the XLarge hacks from Large allocations.
305
306         * bmalloc/Deallocator.cpp:
307         (bmalloc::Deallocator::deallocateXLarge):
308         (bmalloc::Deallocator::deallocateSlowCase):
309         * bmalloc/Heap.cpp:
310         (bmalloc::Heap::findXLarge):
311         (bmalloc::Heap::allocateXLarge):
312         (bmalloc::Heap::deallocateXLarge):
313         * bmalloc/Heap.h: Updated for interface changes.
314
315         * bmalloc/ObjectType.cpp:
316         (bmalloc::objectType):
317         * bmalloc/ObjectType.h:
318         (bmalloc::isXLarge): We can now tell if a pointer is XLarge just by
319         examining its bit pattern -- just like we do for other kinds of
320         allocations -- which is nice.
321
322         * bmalloc/Sizes.h:
323         * bmalloc/VMHeap.h:
324         (bmalloc::VMHeap::allocateXLarge):
325         (bmalloc::VMHeap::findXLarge):
326         (bmalloc::VMHeap::deallocateXLarge): Keep an explicit vector of metadata
327         for XLarge allocations.
328
329         * bmalloc/XLargeChunk.h: Removed.
330
331 2015-01-16  Geoffrey Garen  <ggaren@apple.com>
332
333         bmalloc: added some infrastructure for aligned allocation
334         https://bugs.webkit.org/show_bug.cgi?id=140572
335
336         Reviewed by Andreas Kling.
337
338         * bmalloc/Algorithm.h:
339         (bmalloc::isPowerOfTwo):
340         (bmalloc::roundUpToMultipleOf):
341         (bmalloc::roundDownToMultipleOf): Refactored some duplicate code to use our
342         isPowerOfTwo helper function.
343
344         * bmalloc/Allocator.cpp:
345         (bmalloc::Allocator::allocate):
346         * bmalloc/Allocator.h: Stubbed out an implementation of aligned allocation.
347         Doesn't do anything yet, but does correctly forward to system malloc
348         when bmalloc is disabled.
349
350         * bmalloc/Cache.cpp:
351         (bmalloc::Cache::allocateSlowCaseNullCache):
352         * bmalloc/Cache.h:
353         (bmalloc::Cache::allocate):
354         * bmalloc/bmalloc.h:
355         (bmalloc::api::memalign):
356         * bmalloc/mbmalloc.cpp: Stubbed out an API for aligned allocation.
357
358 2015-01-13  Geoffrey Garen  <ggaren@apple.com>
359
360         Consider alignment when allocating from a SegregatedFreeList
361         https://bugs.webkit.org/show_bug.cgi?id=140408
362
363         Reviewed by Sam Weinig.
364
365         In preparation for supporting aligned allocation.
366
367         No performance change.
368
369         Since this is just one extra branch in an already expensive function,
370         I decided not to duplicate the function just to avoid the branch in
371         the un-aligned case.
372
373         * bmalloc/SegregatedFreeList.cpp:
374         (bmalloc::SegregatedFreeList::take):
375         * bmalloc/SegregatedFreeList.h:
376
377 2015-01-13  Geoffrey Garen  <ggaren@apple.com>
378
379         Renamed minimum to size in SegregatedFreeList
380         https://bugs.webkit.org/show_bug.cgi?id=140406
381
382         Reviewed by Sam Weinig.
383
384         In preparation for supporting aligned allocation.
385
386         * bmalloc/SegregatedFreeList.cpp:
387         (bmalloc::SegregatedFreeList::takeGreedy):
388         (bmalloc::SegregatedFreeList::take): Every size passed to malloc is
389         really just a minimum. Let's not imply that this value is special.
390
391 2015-01-11  Dan Bernstein  <mitz@apple.com>
392
393         Geoff is organized, but he is not an organization.
394
395         Rubber-stamped by Anders Carlsson.
396
397         * bmalloc.xcodeproj/project.pbxproj: Removed the ORGANIZATIONNAME project attribute.
398
399 2015-01-07  Geoffrey Garen  <ggaren@apple.com>
400
401         Make bmalloc work with ASan
402         https://bugs.webkit.org/show_bug.cgi?id=140194
403
404         Reviewed by Mark Lam.
405
406         * bmalloc/BPlatform.h: Added a way to detect Darwin OSes, since we need
407         an OS-specific API to test for loaded runtime libraries.
408
409         * bmalloc/Environment.cpp:
410         (bmalloc::isASanEnabled):
411         (bmalloc::Environment::computeIsBmallocEnabled): Disabled bmalloc if
412         ASan is enabled, since system malloc has the Asan hooks we need.
413
414         You could check for the ASan compile-time flag instead, but doing this
415         check at runtime prepares bmalloc for a world where it is a dynamic
416         library that might be loaded into projects it did not compile with.
417
418 2015-01-05  Geoffrey Garen  <ggaren@apple.com>
419
420         Fix up bmalloc's PerThread for use on Linux
421         https://bugs.webkit.org/show_bug.cgi?id=139804
422
423         Reviewed by Anders Carlsson.
424
425         The previous implementation was a bit slow.
426
427         * bmalloc/PerThread.h:
428         (bmalloc::PerThreadStorage<Cache>::get):
429         (bmalloc::PerThreadStorage::get):
430         (bmalloc::PerThreadStorage::init): Added a catch-all cross-platform Unix
431         way to do fast per-thread access without taking a lock every time. This
432         probably works on all the platforms we care about, and it matches other
433         techniques we use elsewhere in WebKit.
434
435         (bmalloc::PerThread<T>::getFastCase): Removed the conditional from
436         this class because PerThreadStorage now encapsulates everything that
437         needs to be conditional.
438
439         (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Deleted.
440
441 2014-12-26  Dan Bernstein  <mitz@apple.com>
442
443         <rdar://problem/19348208> REGRESSION (r177027): iOS builds use the wrong toolchain
444         https://bugs.webkit.org/show_bug.cgi?id=139950
445
446         Reviewed by David Kilzer.
447
448         * Configurations/Base.xcconfig: Only define TOOLCHAINS when building for OS X, doing so
449         in a manner that works with Xcode 5.1.1.
450
451 2014-12-15  Geoffrey Garen  <ggaren@apple.com>
452
453         Safari crashes when you set Malloc environment variables
454         https://bugs.webkit.org/show_bug.cgi?id=139656
455
456         Reviewed by Michael Saboff.
457
458         I forgot to cover the realloc() case. Whoops. (OoPS?)
459
460         This time around, I ran the full MallocBench test suite in Malloc=1
461         mode, and it passed.
462
463         * bmalloc/Allocator.cpp:
464         (bmalloc::Allocator::reallocate):
465         * bmalloc/Allocator.h: Pushed realloc() logic down into the allocator.
466         It needs to be down there so that we can do the short-circuiting check
467         for whether bmalloc is enabled first.
468
469         Also added the check.
470
471         * bmalloc/Cache.cpp:
472         (bmalloc::Cache::scavenge):
473         (bmalloc::Cache::Cache):
474         (bmalloc::Cache::reallocateSlowCaseNullCache):
475         * bmalloc/Cache.h:
476         (bmalloc::Cache::deallocator):
477         (bmalloc::Cache::reallocate): Ditto.
478
479         * bmalloc/bmalloc.h:
480         (bmalloc::api::free):
481         (bmalloc::api::realloc): Ditto.
482
483         (bmalloc::api::scavenge): Pushed this down into Cache to match the
484         surrounding functions.
485
486 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
487
488         bmalloc should support system memory analysis tools (part 2)
489         https://bugs.webkit.org/show_bug.cgi?id=139565
490
491         Reviewed by Mark Lam.
492
493         This patch actually queries the environment to see if memory analysis
494         tools have been enabled.
495
496         * bmalloc/Deallocator.cpp:
497         (bmalloc::Deallocator::scavenge): Don't process the object log if
498         we've disabled bmalloc because it will be full of invalid nullptrs.
499
500         * bmalloc/Environment.cpp:
501         (bmalloc::isMallocEnvironmentVariableSet): Test for the list of known
502         Malloc debugging flags. I also added a plain "Malloc" catch-all for
503         when you want to disable bmalloc without enabling any kind of funny
504         business.
505
506         It would be slightly nicer just to iterate the list of environment
507         variables and strstr them, but getenv is the more portable option,
508         and performance here doesn't really matter.
509
510         (bmalloc::isLibgmallocEnabled): Test for the libgmalloc insertion
511         environment variable.
512
513         (bmalloc::Environment::computeIsBmallocEnabled):
514
515 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
516
517         Try to fix the iOS simulator build.
518
519         #include the declaration of malloc / free.
520
521         * bmalloc/Allocator.cpp:
522         * bmalloc/Deallocator.cpp:
523
524 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
525
526         Try to fix the build.
527
528         * bmalloc.xcodeproj/project.pbxproj: Marked a header exported.
529
530 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
531
532         bmalloc should support system memory analysis tools (part 1)
533         https://bugs.webkit.org/show_bug.cgi?id=139559
534
535         Reviewed by Mark Lam.
536
537         This patch adds the hooks to disable bmalloc at runtime if certain
538         environment variables are set, but doesn't actually read from the
539         environment yet.
540
541         No performance change.
542
543         * bmalloc.xcodeproj/project.pbxproj: Added the Environment class, which
544         we'll use to read environment variables and see if memory analysis tools
545         have been enabled.
546
547         * bmalloc/Allocator.cpp:
548         (bmalloc::Allocator::Allocator):
549         (bmalloc::Allocator::allocateSlowCase): Added a hook to disable bmalloc
550         on the allocation path. We cache the setting to make the check fast.
551
552         * bmalloc/Allocator.h: Interface changes.
553
554         * bmalloc/Cache.cpp:
555         (bmalloc::Cache::Cache): Pass a heap pointer through to our allocator
556         and deallocator. This main purpose is to enable them to query the
557         environment for whether bmalloc is enabled; but this is also a slightly
558         cleaner way to guarantee to them that the Heap has been pre-initialized.
559
560         * bmalloc/Deallocator.cpp:
561         (bmalloc::Deallocator::Deallocator): If bmalloc is disable, artificially
562         fill the object log to force us to take the slow path on all deallocations.
563
564         (bmalloc::Deallocator::deallocateSlowCase): Do the disabled check.
565
566         * bmalloc/Deallocator.h: Interface changes.
567
568         * bmalloc/Environment.cpp: Added.
569         (bmalloc::Environment::Environment):
570         (bmalloc::Environment::computeIsBmallocEnabled):
571         * bmalloc/Environment.h: Added.
572         (bmalloc::Environment::isBmallocEnabled): This is the class that will
573         encapsulate looking for environment variables that turn on heap
574         analysis tools.
575
576         * bmalloc/Heap.h:
577         (bmalloc::Heap::environment):
578
579         * bmalloc/Mutex.h:
580         (bmalloc::Mutex::Mutex):
581         * bmalloc/StaticMutex.h: A little refactoring to clarify these comments,
582         since I got super confused about them while writing this patch.
583
584         * bmalloc/VMHeap.cpp: Fixed an #include.
585
586 2014-12-09  David Kilzer  <ddkilzer@apple.com>
587
588         Switch from using PLATFORM_NAME to SDK selectors in ANGLE, bmalloc, gtest, JavaScriptCore, WTF
589         <http://webkit.org/b/139212>
590
591         Reviewed by Joseph Pecoraro.
592
593         * Configurations/Base.xcconfig:
594         - Only set GCC_ENABLE_OBJC_GC, GCC_MODEL_TUNING and TOOLCHAINS
595           on OS X.
596         * Configurations/DebugRelease.xcconfig:
597         - Only set MACOSX_DEPLOYMENT_TARGET and SDKROOT on OS X.
598
599 2014-11-07  Geoffrey Garen  <ggaren@apple.com>
600
601         bmalloc uses 8X more virtual memory than necessary
602         https://bugs.webkit.org/show_bug.cgi?id=138495
603
604         Reviewed by Mark Lam.
605
606         iOS has a per-process virtual memory cap around 1GB, so there's some
607         value to not going totally ham with virtual memory.
608
609         We currently use about 8X the necessary amount:
610             - 2X to align our VM allocation
611             - 4X to reserve small / medium / (2) large chunk VM ranges per superchunk
612
613         We can cut that down:
614             - Return the unaligned portion of our VM allocation (-2X)
615             - Use all the chunks in a superchunk, instead of allocating one
616               chunk per superchunk (-4X)
617
618         * bmalloc/Algorithm.h:
619         (bmalloc::roundUpToMultipleOf): Added a non-constant version of this
620         function so we can call it with getpagesize() at runtime.
621
622         * bmalloc/Chunk.h:
623         * bmalloc/LargeChunk.h:
624         (bmalloc::LargeChunk::create): Deleted. Instead of each chunk allocating
625         its own VM, VMHeap allocates the superchunk and all the chunks in it at a time.
626
627         * bmalloc/VMAllocate.h:
628         (bmalloc::vmValidate):
629         (bmalloc::vmAllocate): ASSERT that mmap succeeds to make crashes clearer
630         if it does not succeed. Allocate precisely, and give back the extra.
631
632         * bmalloc/VMHeap.cpp:
633         (bmalloc::VMHeap::allocateSuperChunk):
634         (bmalloc::VMHeap::allocateSmallChunk): Deleted.
635         (bmalloc::VMHeap::allocateMediumChunk): Deleted.
636         (bmalloc::VMHeap::allocateLargeChunk): Deleted. Use all the chunks
637         in a superchunk, instead of just one.
638
639         * bmalloc/VMHeap.h:
640         (bmalloc::VMHeap::allocateSmallPage):
641         (bmalloc::VMHeap::allocateMediumPage):
642         (bmalloc::VMHeap::allocateLargeRange):
643         * bmalloc/XLargeChunk.h:
644         (bmalloc::XLargeChunk::create): Updated to match changes above.
645
646 2014-11-01  David Kilzer  <ddkilzer@apple.com>
647
648         JavaScriptCore is missing debug info for bmalloc because libbmalloc.a is stripped
649         <https://webkit.org/b/138286>
650         <rdar://problem/18847087>
651
652         Reviewed by Dan Bernstein.
653
654         * Configurations/bmalloc.xcconfig: Set STRIP_INSTALLED_PRODUCT
655         to NO for the target that produces libbmalloc.a so that the
656         debug symbols will be linked into JavaScriptCore and end up in
657         its dSYM file.
658
659 2014-10-30  Dana Burkart  <dburkart@apple.com>
660
661         <rdar://problem/18821260> Prepare for the mysterious future
662
663         Reviewed by Lucas Forschler.
664
665         * Configurations/Base.xcconfig:
666         * Configurations/DebugRelease.xcconfig:
667
668 2014-09-24  Geoffrey Garen  <ggaren@apple.com>
669
670         bmalloc: cleaned up fast path vs slow path
671         https://bugs.webkit.org/show_bug.cgi?id=137081
672
673         Reviewed by Sam Weinig.
674
675         Might be a 1% speedup on MallocBench. Also cleans up the code a bit.
676
677         * bmalloc/Allocator.cpp:
678         (bmalloc::Allocator::Allocator): Merged the small and medium range
679         caches, just like the small and medium allocators. Ranges are abstract
680         objects that don't really care whether they hold small or medium objects,
681         so they don't need to be segregated.
682
683         (bmalloc::Allocator::scavenge): Ditto.
684
685         (bmalloc::Allocator::allocateBumpRangeSlowCase):
686         (bmalloc::Allocator::allocateBumpRange): Same thing here, except that
687         we do care a tiny bit, because we need to specify small vs medium when
688         allocating new ranges from the heap, to ensure that the heap allocates
689         from the right segment of VM.
690
691         (bmalloc::Allocator::allocateLarge):
692         (bmalloc::Allocator::allocateXLarge): NO_INLINE because this was clouding
693         up the fast path. Large allocation performance is dominated by allocation
694         logic and initialization, so inlining it doesn't help.
695
696         (bmalloc::Allocator::allocateSlowCase): Slow path got a bit cleaner since
697         it doesn't need to distinguish small vs medium objects.
698
699         (bmalloc::Allocator::allocateSmallBumpRange): Deleted.
700         (bmalloc::Allocator::allocateMediumBumpRange): Deleted.
701
702         * bmalloc/Allocator.h:
703         * bmalloc/BumpRange.h:
704
705         * bmalloc/Cache.cpp:
706         (bmalloc::Cache::allocateSlowCase): Deleted.
707         (bmalloc::Cache::deallocateSlowCase): Deleted.
708         * bmalloc/Cache.h:
709         (bmalloc::Cache::allocate):
710         (bmalloc::Cache::deallocate):
711         (bmalloc::Cache::allocateFastCase): Deleted.
712         (bmalloc::Cache::deallocateFastCase): Deleted. Removed the Cache slow
713         paths. The downside to this change is that the fast path branches to two
714         distinct failure cases instead of one. The upside is that the slow path
715         doesn't need to re-read the segment register, which is not as cheap as a
716         normal register, and it doesn't need to do an extra level of function 
717         call. Seems to be worth it.
718
719         * bmalloc/Deallocator.h:
720         * bmalloc/Heap.cpp:
721         (bmalloc::Heap::refillSmallBumpRangeCache):
722         (bmalloc::Heap::refillMediumBumpRangeCache):
723         * bmalloc/Heap.h: Updated for interface changes.
724
725         * bmalloc/Sizes.h: The most ranges a cache will hold is the number of
726         small lines in a page / 2, since any other free lines will coalesce
727         with their neighbors.
728
729 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
730
731         Rolled out r173346.
732
733             bmalloc should honor the FastMalloc statistics API
734             https://bugs.webkit.org/show_bug.cgi?id=136592
735
736         This didn't really work. Because we allow ranges with and without
737         physical pages to merge, and we allow double-committing and
738         double-decommitting, we can't rely on commit actions to track memory
739         footprint.
740
741         * bmalloc/Heap.cpp:
742         (bmalloc::Heap::size): Deleted.
743         (bmalloc::Heap::capacity): Deleted.
744         * bmalloc/Heap.h:
745         * bmalloc/VMHeap.cpp:
746         (bmalloc::VMHeap::VMHeap):
747         (bmalloc::VMHeap::allocateSmallChunk):
748         (bmalloc::VMHeap::allocateMediumChunk):
749         (bmalloc::VMHeap::allocateLargeChunk):
750         * bmalloc/VMHeap.h:
751         (bmalloc::VMHeap::allocateSmallPage):
752         (bmalloc::VMHeap::allocateMediumPage):
753         (bmalloc::VMHeap::allocateLargeRange):
754         (bmalloc::VMHeap::deallocateSmallPage):
755         (bmalloc::VMHeap::deallocateMediumPage):
756         (bmalloc::VMHeap::deallocateLargeRange):
757         (bmalloc::VMHeap::size): Deleted.
758         (bmalloc::VMHeap::capacity): Deleted.
759         * bmalloc/bmalloc.h:
760         (bmalloc::api::heapSize): Deleted.
761         (bmalloc::api::heapCapacity): Deleted.
762
763 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
764
765         bmalloc: Allocation should be more precise
766         https://bugs.webkit.org/show_bug.cgi?id=136993
767
768         Reviewed by Gavin Barraclough.
769
770         13% reduction in heap size on the MallocBench *_memory_warning benchmarks.
771
772         This patch teaches the allocator to merge adjacent free lines into a
773         single allocatable range. This allows us to shrink the size of an
774         individual line without increasing fragmentation or the rate of allocator
775         slow paths.
776
777         We'll only take more slow paths when available memory is sparse, which
778         is exactly when it's worth it. When available memory is dense, we'll
779         take fewer slow paths.
780
781         * bmalloc.xcodeproj/project.pbxproj:
782         * bmalloc/Algorithm.h:
783         (bmalloc::divideRoundingUp):
784
785         * bmalloc/Allocator.cpp:
786         (bmalloc::Allocator::Allocator): Updated for interface changes.
787
788         (bmalloc::Allocator::scavenge): Scavenge by object instead of by line.
789         Now that we merge lines, it's not convenient to scavenge by line.
790
791         (bmalloc::Allocator::allocateSmallBumpRange):
792         (bmalloc::Allocator::allocateMediumBumpRange): Allocate whole ranges
793         instead of individual lines.
794
795         (bmalloc::Allocator::allocateSlowCase):
796         (bmalloc::Allocator::allocateSmallLine): Deleted.
797         (bmalloc::Allocator::allocateMediumLine): Deleted.
798         (bmalloc::Allocator::allocateMedium): Deleted.
799         * bmalloc/Allocator.h:
800         (bmalloc::Allocator::allocateFastCase): Folded medium allocations
801         into the standard fast path with small allocations. Since a BumpAllocator
802         just allocates out of an arbitrary range, it doesn't need to distinguish
803         between small and medium lines.
804
805         * bmalloc/BumpAllocator.h:
806         (bmalloc::BumpAllocator::size):
807         (bmalloc::BumpAllocator::BumpAllocator):
808         (bmalloc::BumpAllocator::init):
809         (bmalloc::BumpAllocator::refill):
810         (bmalloc::BumpAllocator::line): Deleted. No need to track line information
811         anymore: the heap just gives us a pointer and a pre-computed number of
812         objects, and we allocate them.
813
814         * bmalloc/Deallocator.cpp:
815         (bmalloc::Deallocator::processObjectLog): Updated for interface changes.
816
817         * bmalloc/Heap.cpp:
818         (bmalloc::Heap::Heap):
819         (bmalloc::Heap::initializeLineMetadata): Pre-compute precise metadata
820         detailing where all objects will lie in memory. After we merge two lines,
821         we might allocate an object that spans from one line to the next. This
822         metadata details which bits of memory overlap in that way, and how they
823         overlap.
824
825         (bmalloc::Heap::refillSmallBumpRangeCache):
826         (bmalloc::Heap::refillMediumBumpRangeCache): Scan a whole page at a time,
827         and merge adjacent free lines into BumpRanges.
828
829         (bmalloc::Heap::allocateSmallPage):
830         (bmalloc::Heap::allocateMediumPage):
831         (bmalloc::Heap::deallocateSmallLine):
832         (bmalloc::Heap::deallocateMediumLine): Track pages rather than lines,
833         since we scan for free memory a page at a time.
834
835         (bmalloc::Heap::allocateSmallLineSlowCase): Deleted.
836         (bmalloc::Heap::allocateMediumLineSlowCase): Deleted. Folded into the
837         fast path.
838
839         * bmalloc/Heap.h:
840         (bmalloc::Heap::derefSmallLine):
841         (bmalloc::Heap::derefMediumLine):
842         (bmalloc::Heap::deallocateSmallLine): Deleted.
843         (bmalloc::Heap::allocateSmallLine): Deleted.
844         (bmalloc::Heap::deallocateMediumLine): Deleted.
845         (bmalloc::Heap::allocateMediumLine): Deleted. Updated for interface changes.
846
847         * bmalloc/Line.h:
848         (bmalloc::Line<Traits>::ref):
849         (bmalloc::Line<Traits>::deref):
850         (bmalloc::Line<Traits>::concurrentRef): Deleted. We don't pass a derefCount
851         anymore, since we only ever deref by 1 now.
852
853         * bmalloc/MediumAllocator.h:
854         (bmalloc::MediumAllocator::isNull): Deleted.
855         (bmalloc::MediumAllocator::MediumAllocator): Deleted.
856         (bmalloc::MediumAllocator::line): Deleted.
857         (bmalloc::MediumAllocator::allocate): Deleted.
858         (bmalloc::MediumAllocator::derefCount): Deleted.
859         (bmalloc::MediumAllocator::refill): Deleted.
860         (bmalloc::MediumAllocator::clear): Deleted. Deleted some code that's
861         been dead for a while, since it doesn't build anymore with this patch.
862
863         * bmalloc/Page.h:
864         (bmalloc::Page::sizeClass):
865         (bmalloc::Page::setSizeClass):
866         (bmalloc::Page::smallSizeClass): Deleted.
867         (bmalloc::Page::setSmallSizeClass): Deleted. Renamed setSmallSizeClass
868         to sizeClass, since we use it for medium sizes too.
869
870         * bmalloc/Sizes.h:
871         (bmalloc::Sizes::sizeClass):
872         (bmalloc::Sizes::objectSize): Shrank line sizes to save memory.
873
874         (bmalloc::Sizes::smallSizeClassFor): Deleted.
875         (bmalloc::Sizes::mediumSizeClassFor): Deleted.
876
877         * bmalloc/bmalloc.h:
878         (bmalloc::api::realloc): Now that we have precise objects sizes, realloc
879         can be a bit more precise. It also has to be, since we can't guarantee
880         that an object ends at the end of a line anymore.
881
882 2014-09-19  Daniel Bates  <dabates@apple.com>
883
884         Always assume internal SDK when building configuration Production
885         https://bugs.webkit.org/show_bug.cgi?id=136925
886         <rdar://problem/18362399>
887
888         Reviewed by Dan Bernstein.
889
890         * Configurations/Base.xcconfig:
891
892 2014-09-16  Geoffrey Garen  <ggaren@apple.com>
893
894         bmalloc: moved line caches from the deallocator to the allocator
895         https://bugs.webkit.org/show_bug.cgi?id=136868
896
897         Reviewed by Gavin Barraclough.
898
899         I did this mostly as a simplification, to make it easier to change the
900         allocation strategy.
901
902         No throughput change on MallocBench. Saves about 50kB.
903
904         Since the deallocator needs to lock the heap when freeing lines anyway,
905         there isn't much benefit to giving the deallocator a local cache of
906         deallocated lines.
907
908         We still give the allocator a local cache of lines because that does
909         reduce the frequency at which it needs to lock the heap in order to
910         acquire more lines.
911
912         * bmalloc/Allocator.cpp:
913         (bmalloc::Allocator::scavenge):
914         (bmalloc::Allocator::allocateSmallLine):
915         (bmalloc::Allocator::allocateMediumLine):
916         (bmalloc::Allocator::allocateMedium):
917         (bmalloc::Allocator::allocateSlowCase):
918         * bmalloc/Allocator.h:
919         * bmalloc/Deallocator.cpp:
920         (bmalloc::Deallocator::Deallocator):
921         (bmalloc::Deallocator::scavenge):
922         (bmalloc::Deallocator::processObjectLog):
923         (bmalloc::Deallocator::deallocateSmallLine): Deleted.
924         (bmalloc::Deallocator::allocateSmallLine): Deleted.
925         (bmalloc::Deallocator::deallocateMediumLine): Deleted.
926         (bmalloc::Deallocator::allocateMediumLine): Deleted.
927         * bmalloc/Deallocator.h:
928
929         * bmalloc/Sizes.h:
930         * bmalloc/VMAllocate.h: Took the opportunity to make the line cache size
931         exactly one page in size. That's about what we were shooting for anyway,
932         and it may make it easier to switch to per-page allocation in future.
933
934 2014-09-15  Geoffrey Garen  <ggaren@apple.com>
935
936         bmalloc: allocate small and medium objects using the same bump pointer class
937         https://bugs.webkit.org/show_bug.cgi?id=136843
938
939         Reviewed by Gavin Barraclough.
940
941         4% speedup on MallocBench.
942
943         Now that medium-sized objects have dedicated per-size allocators, they
944         don't need to use an arbitrary bump pointer allocator. This means that
945         every allocator knows how many objects it will allocate from the start,
946         and we don't need a post-processing step to adjust refcounts based on
947         real allocation count.
948
949         * bmalloc.xcodeproj/project.pbxproj: Renamed SmallAllocator to BumpAllocator
950         since it's used for small and medium objects now.
951
952         * bmalloc/Allocator.cpp:
953         (bmalloc::Allocator::Allocator): Updated to use new interface.
954         (bmalloc::Allocator::scavenge): To "retire" an allocator, we just need
955         to make sure that we finish allocating all the objects in it.
956
957         (bmalloc::Allocator::allocateMedium):
958         (bmalloc::Allocator::allocateSlowCase):
959         (bmalloc::Allocator::retire): Deleted.
960         (bmalloc::Allocator::processSmallAllocatorLog): Deleted.
961         (bmalloc::Allocator::processMediumAllocatorLog): Deleted.
962         * bmalloc/Allocator.h:
963         (bmalloc::Allocator::allocateFastCase): Removed abstractions and data
964         used to post-process an allocator based on how many objects it allocated.
965
966         * bmalloc/BumpAllocator.h: Copied from Source/bmalloc/bmalloc/SmallAllocator.h.
967         (bmalloc::BumpAllocator::BumpAllocator):
968         (bmalloc::BumpAllocator::init):
969         (bmalloc::BumpAllocator::line):
970         (bmalloc::BumpAllocator::validate):
971         (bmalloc::BumpAllocator::allocate):
972         (bmalloc::BumpAllocator::refill):
973         (bmalloc::BumpAllocator::clear): Updated these functions to be agnostic
974         about the kinds of lines they allocate into. In some cases, the line
975         type must be provided as a template parameter by the caller.
976
977         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
978         (bmalloc::SmallAllocator::line): Deleted.
979         (bmalloc::SmallAllocator::allocate): Deleted.
980         (bmalloc::SmallAllocator::objectCount): Deleted.
981         (bmalloc::SmallAllocator::derefCount): Deleted.
982         (bmalloc::SmallAllocator::refill): Deleted.
983         (bmalloc::SmallAllocator::clear): Deleted.
984
985         * bmalloc/ObjectType.h:
986         (bmalloc::isMedium):
987
988         * bmalloc/SmallAllocator.h:
989         (bmalloc::SmallAllocator::isNull): Deleted.
990         (bmalloc::SmallAllocator::canAllocate): Deleted.
991         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
992         (bmalloc::SmallAllocator::line): Deleted.
993         (bmalloc::SmallAllocator::allocate): Deleted.
994         (bmalloc::SmallAllocator::objectCount): Deleted.
995         (bmalloc::SmallAllocator::derefCount): Deleted.
996         (bmalloc::SmallAllocator::refill): Deleted.
997         (bmalloc::SmallAllocator::clear): Deleted.
998
999 2014-09-12  Geoffrey Garen  <ggaren@apple.com>
1000
1001         Fixed a goof in bmalloc Vector sizing
1002         https://bugs.webkit.org/show_bug.cgi?id=136795
1003
1004         Reviewed by Gavin Barraclough and Sam Weinig.
1005
1006         We want our minimum vector to be page-sized since the OS will give us
1007         a page no matter what -- but we want that many bytes, and not enough
1008         bytes to store that many elements.
1009
1010         * bmalloc/Vector.h: Math is hard.
1011
1012 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
1013
1014         bmalloc should segregate medium-sized objects by line like it does for small-sized objects
1015         https://bugs.webkit.org/show_bug.cgi?id=136693
1016
1017         Reviewed by Gavin Barraclough.
1018
1019         4% reduction in heap size on the MallocBench *_memory_warning benchmarks.
1020
1021         No throughput change.
1022
1023         We keep an array of medium allocators, just like our array of small
1024         allocators.
1025
1026         In future, we can simplify the allocation fast path by merging the small
1027         and medium allocator arrays. For now, this is the simplest change that
1028         gets the win.
1029
1030         * bmalloc/Allocator.cpp:
1031         (bmalloc::Allocator::Allocator):
1032         (bmalloc::Allocator::scavenge):
1033         (bmalloc::Allocator::allocateMedium):
1034         * bmalloc/Allocator.h:
1035         * bmalloc/Sizes.h:
1036         (bmalloc::Sizes::mediumSizeClassFor):
1037
1038 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
1039
1040         Reviewed by Sam Weinig.
1041
1042         Renamed log => retire for clarity.
1043
1044         * bmalloc/Allocator.cpp:
1045         (bmalloc::Allocator::scavenge):
1046         (bmalloc::Allocator::retire):
1047         (bmalloc::Allocator::allocateMedium):
1048         (bmalloc::Allocator::allocateSlowCase):
1049         (bmalloc::Allocator::log): Deleted.
1050         * bmalloc/Allocator.h:
1051
1052 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
1053
1054         bmalloc: eager scavenge leaves behind a bogus allocator
1055         https://bugs.webkit.org/show_bug.cgi?id=136743
1056
1057         Reviewed by Sam Weinig.
1058
1059         Be sure to clear the allocator after logging it in the eager scavenge
1060         case, so that we don't later try to allocate out of the lines that we
1061         have thrown away.
1062
1063         We didn't need to do this previously because scavenge would only happen
1064         at thread exit time, after which no further allocation from the per-thread
1065         cache would take place.
1066
1067         * bmalloc/Allocator.cpp:
1068         (bmalloc::Allocator::scavenge):
1069         * bmalloc/MediumAllocator.h:
1070         (bmalloc::MediumAllocator::clear):
1071         * bmalloc/SmallAllocator.h:
1072         (bmalloc::SmallAllocator::clear):
1073
1074 2014-09-05  Geoffrey Garen  <ggaren@apple.com>
1075
1076         bmalloc should honor the FastMalloc statistics API
1077         https://bugs.webkit.org/show_bug.cgi?id=136592
1078
1079         Reviewed by Gavin Barraclough.
1080
1081         We do this by tracking "size" and "capacity" in the VM heap.
1082
1083         The VM heap's "capacity" is all the VM we ever allocated.
1084
1085         The VM heap's "size" the subset of VM currently held onto by the
1086         VM heap (and therefore not in use by the regular heap).
1087
1088         Somewhat ironically, reducing the process's memory footprint, increases
1089         the size of the VM heap, since the VM heap holds the pages that are
1090         purely virtual and not physical.
1091
1092         * bmalloc/Heap.cpp:
1093         (bmalloc::Heap::size):
1094         (bmalloc::Heap::capacity):
1095         * bmalloc/Heap.h:
1096         * bmalloc/VMHeap.cpp:
1097         (bmalloc::VMHeap::VMHeap):
1098         (bmalloc::VMHeap::allocateSmallChunk):
1099         (bmalloc::VMHeap::allocateMediumChunk):
1100         (bmalloc::VMHeap::allocateLargeChunk):
1101         * bmalloc/VMHeap.h:
1102         (bmalloc::VMHeap::size):
1103         (bmalloc::VMHeap::capacity):
1104         (bmalloc::VMHeap::allocateSmallPage):
1105         (bmalloc::VMHeap::allocateMediumPage):
1106         (bmalloc::VMHeap::allocateLargeRange):
1107         (bmalloc::VMHeap::deallocateSmallPage):
1108         (bmalloc::VMHeap::deallocateMediumPage):
1109         (bmalloc::VMHeap::deallocateLargeRange):
1110         * bmalloc/bmalloc.h:
1111         (bmalloc::api::heapSize):
1112         (bmalloc::api::heapCapacity):
1113
1114 2014-09-02  Geoffrey Garen  <ggaren@apple.com>
1115
1116         bmalloc crashes on the EWS bots (due to bad large object allocation)
1117         https://bugs.webkit.org/show_bug.cgi?id=136469
1118
1119         Reviewed by Andreas Kling.
1120
1121         It's possible to convince bmalloc to perform a bad large object allocation,
1122         through these steps:
1123
1124         (1) Insert object A into freelist F0.
1125
1126         (2) Split, merge and split again A's neighbors such that object B is
1127         inserted into freelist F0, with boundary tag and size equal to object A,
1128         but pointer not completely equal to object A. Put object B at the head of F0.
1129
1130         (3) Allocate some other object from F0, swapping its position in the
1131         freelist with object B, such that object A is now ahead of object B.
1132
1133         --> Now, the next allocation for size A/B will allocate object A, which
1134         has a slightly wrong idea about where the object actually begins.
1135         Immediately, you'll corrupt a little memory, and over time, you'll also
1136         corrupt boundary tag metadata.
1137
1138         The solution is to store the begin pointer in the boundary tag. Luckily,
1139         this doesn't make the tag any bigger, and it's not a noticeable slowdown
1140         on MallocBench.
1141
1142         * bmalloc/Algorithm.h:
1143         (bmalloc::rightShift):
1144         * bmalloc/BeginTag.h:
1145         (bmalloc::BeginTag::isInFreeList): This is the bug fix. Make sure to
1146         validate the start pointer when popping off the free list. Through a
1147         very uncommon set of steps, it is possible to have an item in the free
1148         list that is valid by all accounts except for its start pointer.
1149
1150         * bmalloc/BoundaryTag.h:
1151         (bmalloc::BoundaryTag::compactBegin):
1152         (bmalloc::BoundaryTag::setRange):
1153         (bmalloc::BoundaryTag::setSize): Deleted. Record a compact version of the
1154         start pointer. We don't need the whole pointer -- just the offset, in
1155         largeAlignment increments, into the relevant boundary tag bucket.
1156
1157         * bmalloc/BoundaryTagInlines.h:
1158         (bmalloc::validateNext):
1159         (bmalloc::BoundaryTag::init):
1160         (bmalloc::BoundaryTag::mergeLarge):
1161         (bmalloc::BoundaryTag::splitLarge):
1162         * bmalloc/SegregatedFreeList.cpp:
1163         (bmalloc::SegregatedFreeList::insert):
1164         (bmalloc::SegregatedFreeList::takeGreedy):
1165         (bmalloc::SegregatedFreeList::take): Provide the whole range instead of
1166         the size when establishing a boundary tag, as required by the new
1167         interface.
1168
1169         * bmalloc/Sizes.h:
1170
1171 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
1172
1173         Fixed a bmalloc crash seen on the EWS bot
1174         https://bugs.webkit.org/show_bug.cgi?id=135955
1175
1176         Reviewed by Andreas Kling.
1177
1178         * bmalloc/Syscall.h: Some CG APIs vm_copy their input buffers. If the
1179         input buffer is a malloc region, that region will get marked Copy-On-Write
1180         by the kernel. Calls to madvise() for COW regions fail and return EINVAL
1181         on older OS X's. In 10.10, they still fail, but they do not return
1182         EINVAL.
1183
1184         So, we can only ASSERT that our syscalls succeed starting with 10.10.
1185
1186 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
1187
1188         Fixed the bmalloc build
1189         https://bugs.webkit.org/show_bug.cgi?id=135953
1190
1191         Reviewed by Andreas Kling.
1192
1193         * bmalloc.xcodeproj/project.pbxproj: Marked a few headers as private.
1194         These headers are used, so they must be available outside the project.
1195
1196 2014-08-13  Daniel Bates  <dabates@apple.com>
1197
1198         Attempt to fix the build following <http://trac.webkit.org/changeset/172576>
1199         (https://bugs.webkit.org/show_bug.cgi?id=135895)
1200
1201         Substitute PerThreadStorage<T>::initSharedKeyIfNeeded() for initSharedKeyIfNeeded() in
1202         implementation of PerThread<T>::getFastCase().
1203
1204         * bmalloc/PerThread.h:
1205         (bmalloc::PerThread<T>::getFastCase):
1206
1207 2014-08-13  Daniel Bates  <dabates@apple.com>
1208
1209         Make bmalloc::PerThread work without C++ thread local storage
1210         https://bugs.webkit.org/show_bug.cgi?id=135895
1211
1212         Reviewed by Geoffrey Garen.
1213
1214         Implement support for building bmalloc without C++ thread local storage.
1215
1216         * bmalloc/BPlatform.h: Remove macro define BPLATFORM_IOS_SIMULATOR. Added macro function
1217         BCOMPILER_SUPPORTS() and macro define BCOMPILER_SUPPORTS_CXX_THREAD_LOCAL that can be used
1218         to determine whether the compiler supports C++ thread local storage.
1219         * bmalloc/PerThread.h:
1220         (bmalloc::PerThreadStorage::get): Modified to call pthread_getspecific() when building
1221         without C++ thread local storage.
1222         (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Added.
1223         (bmalloc::PerThreadStorage::init): Moved logic to initialize shared Pthread key from here to
1224         PerThreadStorage::initSharedKeyIfNeeded().
1225         (bmalloc::PerThread<T>::getFastCase): Modified to call PerThreadStorage::initSharedKeyIfNeeded()
1226         before querying PerThreadStorage::get() when building without C++ thread local storage so as to
1227         ensure that the shared key has been initialized.
1228         (_pthread_setspecific_direct): Deleted.
1229         (_pthread_getspecific_direct): Deleted.
1230
1231 2014-08-13  Daniel Bates  <dabates@apple.com>
1232
1233         [iOS] Make JavaScriptCore and bmalloc build with the public SDK
1234         https://bugs.webkit.org/show_bug.cgi?id=135848
1235
1236         Reviewed by Geoffrey Garen.
1237
1238         * bmalloc/BPlatform.h: Added macro BPLATFORM_IOS_SIMULATOR, which evaluates to true
1239         when building for the iOS Simulator.
1240         * bmalloc/PerThread.h: Use pthread_machdep.h code path when building for iOS Simulator
1241         using the public SDK.
1242         (_pthread_setspecific_direct): Added; only defined when building for the iOS Simulator
1243         using the public SDK.
1244         (_pthread_getspecific_direct): Added; only defined when building for the iOS Simulator
1245         using the public SDK.
1246
1247 2014-08-12  Daniel Bates  <dabates@apple.com>
1248
1249         BPLATFORM(IOS) always evaluates to false
1250         https://bugs.webkit.org/show_bug.cgi?id=135843
1251
1252         Reviewed by Geoffrey Garen.
1253
1254         Fix typo in definition of BPLATFORM() and include system header TargetConditionals.h
1255         (when building on an Apple platform) so that BPLATFORM(X) evaluates to true when
1256         building for platform X. In particular, so that BPLATFORM(IOS) evaluates to true when
1257         building for iOS.
1258
1259         As a side effect of this change, the change made in <http://trac.webkit.org/changeset/167289>
1260         will be honored and iOS will assume a VM page size of 16kB (again) instead of 4kB.
1261
1262         * bmalloc/BPlatform.h:
1263
1264 2014-08-11  Andy Estes  <aestes@apple.com>
1265
1266         [iOS] Get rid of iOS.xcconfig
1267         https://bugs.webkit.org/show_bug.cgi?id=135809
1268
1269         Reviewed by Joseph Pecoraro.
1270
1271         All iOS.xcconfig did was include AspenFamily.xcconfig, so there's no need for the indirection.
1272
1273         * Configurations/Base.xcconfig:
1274         * Configurations/iOS.xcconfig: Removed.
1275         * bmalloc.xcodeproj/project.pbxproj:
1276
1277 2014-05-01  Dan Bernstein  <mitz@apple.com>
1278
1279         Fixed production builds for the iOS Simulator.
1280         <rdar://problem/16792221>
1281
1282         * Configurations/bmalloc.xcconfig: Include INSTALL_PATH_PREFIX in
1283         PRIVATE_HEADERS_FOLDER_PATH when installing.
1284
1285 2014-04-20  Geoffrey Garen  <ggaren@apple.com>
1286
1287         bmalloc: Segregate pages by objects size
1288         https://bugs.webkit.org/show_bug.cgi?id=131909
1289
1290         Reviewed by Andreas Kling.
1291
1292         2% reduction in memory-at-end on the Membuster memory_warning benchmarks.
1293
1294         * bmalloc/Allocator.cpp:
1295         (bmalloc::Allocator::allocateSlowCase):
1296         * bmalloc/Allocator.h:
1297         (bmalloc::Allocator::allocateFastCase):
1298         (bmalloc::Allocator::smallAllocatorFor): Use the new shared helper
1299         function for size class calculation.
1300
1301         * bmalloc/Deallocator.cpp:
1302         (bmalloc::Deallocator::Deallocator):
1303         (bmalloc::Deallocator::scavenge):
1304         (bmalloc::Deallocator::deallocateSmallLine):
1305         (bmalloc::Deallocator::allocateSmallLine):
1306         * bmalloc/Deallocator.h: Keep a cache for every size class, since the
1307         cache can't be shared anymore.
1308
1309         * bmalloc/Heap.cpp:
1310         (bmalloc::Heap::allocateSmallLineSlowCase):
1311         * bmalloc/Heap.h:
1312         (bmalloc::Heap::deallocateSmallLine): Ditto.
1313
1314         (bmalloc::Heap::allocateSmallLine): Check size class in addition to
1315         page refcount when allocating a line because we might have deallocated
1316         the page and the recycled it for another size class.
1317
1318         (bmalloc::Heap::deallocateMediumLine):
1319         (bmalloc::Heap::allocateMediumLine):
1320         * bmalloc/Line.h:
1321         (bmalloc::Line::refCount):
1322         * bmalloc/Page.h:
1323         (bmalloc::Page::refCount):
1324         (bmalloc::Page::smallSizeClass):
1325         (bmalloc::Page::setSmallSizeClass):
1326         (bmalloc::Page<Traits>::refCount): Deleted.
1327         * bmalloc/Sizes.h:
1328         (bmalloc::Sizes::smallSizeClassFor): New shared API for computing
1329         an index into an array from a size.
1330
1331 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1332
1333         bmalloc: Improved alignment in LargeChunk
1334         https://bugs.webkit.org/show_bug.cgi?id=131895
1335
1336         Reviewed by Andreas Kling.
1337
1338         * bmalloc/Chunk.h:
1339         * bmalloc/LargeChunk.h: Align to vmPageSize just like Chunk does.
1340         Technically, the previous alignment was harmless, but I would prefer,
1341         dear reader, not to have to explain the interlocking set of
1342         circumstances that made it so.
1343
1344 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1345
1346         Rolled out r167502 because it caused a crash on the facebook benchmark.
1347
1348         Unreviewed.
1349
1350             bmalloc: Added an XSmall line size
1351             https://bugs.webkit.org/show_bug.cgi?id=131851
1352
1353             Reviewed by Sam Weinig.
1354
1355 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1356
1357         bmalloc: Mutex should be harder to use wrong
1358         https://bugs.webkit.org/show_bug.cgi?id=131879
1359
1360         Reviewed by Andreas Kling.
1361
1362         Mutex now has a proper constructor, so you can't deadlock by forgetting
1363         to initialize it.
1364
1365         * bmalloc.xcodeproj/project.pbxproj:
1366         * bmalloc/Allocator.cpp:
1367         (bmalloc::Allocator::processXSmallAllocatorLog):
1368         (bmalloc::Allocator::processSmallAllocatorLog):
1369         (bmalloc::Allocator::processMediumAllocatorLog):
1370         (bmalloc::Allocator::allocateLarge):
1371         (bmalloc::Allocator::allocateXLarge): Global replace Mutex => StaticMutex,
1372         since the Heap mutex is a static.
1373
1374         * bmalloc/AsyncTask.h:
1375         (bmalloc::Function>::AsyncTask): Use Mutex, since we're not static. No
1376         need for explicit initialization anymore.
1377
1378         * bmalloc/Deallocator.cpp:
1379         (bmalloc::Deallocator::scavenge):
1380         (bmalloc::Deallocator::deallocateLarge):
1381         (bmalloc::Deallocator::deallocateXLarge):
1382         (bmalloc::Deallocator::processObjectLog):
1383         (bmalloc::Deallocator::deallocateSmallLine):
1384         (bmalloc::Deallocator::deallocateXSmallLine):
1385         (bmalloc::Deallocator::allocateSmallLine):
1386         (bmalloc::Deallocator::allocateXSmallLine):
1387         (bmalloc::Deallocator::deallocateMediumLine):
1388         (bmalloc::Deallocator::allocateMediumLine):
1389         * bmalloc/Deallocator.h:
1390         * bmalloc/Heap.cpp:
1391         (bmalloc::sleep):
1392         (bmalloc::Heap::Heap):
1393         (bmalloc::Heap::concurrentScavenge):
1394         (bmalloc::Heap::scavenge):
1395         (bmalloc::Heap::scavengeSmallPages):
1396         (bmalloc::Heap::scavengeXSmallPages):
1397         (bmalloc::Heap::scavengeMediumPages):
1398         (bmalloc::Heap::scavengeLargeRanges):
1399         (bmalloc::Heap::allocateXSmallLineSlowCase):
1400         (bmalloc::Heap::allocateSmallLineSlowCase):
1401         (bmalloc::Heap::allocateMediumLineSlowCase):
1402         (bmalloc::Heap::allocateXLarge):
1403         (bmalloc::Heap::deallocateXLarge):
1404         (bmalloc::Heap::allocateLarge):
1405         (bmalloc::Heap::deallocateLarge):
1406         * bmalloc/Heap.h:
1407         (bmalloc::Heap::deallocateXSmallLine):
1408         (bmalloc::Heap::allocateXSmallLine):
1409         (bmalloc::Heap::deallocateSmallLine):
1410         (bmalloc::Heap::allocateSmallLine):
1411         (bmalloc::Heap::deallocateMediumLine):
1412         (bmalloc::Heap::allocateMediumLine):
1413         * bmalloc/Line.h:
1414         (bmalloc::Line<Traits>::deref):
1415         * bmalloc/Mutex.cpp: Removed.
1416         * bmalloc/Mutex.h:
1417         (bmalloc::Mutex::Mutex):
1418         (bmalloc::Mutex::init): Deleted.
1419         (bmalloc::Mutex::try_lock): Deleted.
1420         (bmalloc::Mutex::lock): Deleted.
1421         (bmalloc::Mutex::unlock): Deleted.
1422         * bmalloc/Page.h:
1423         (bmalloc::Page<Traits>::ref):
1424         (bmalloc::Page<Traits>::deref):
1425         (bmalloc::Page<Traits>::refCount):
1426         * bmalloc/PerProcess.h:
1427         (bmalloc::PerProcess::mutex):
1428         (bmalloc::PerProcess<T>::getSlowCase):
1429         * bmalloc/StaticMutex.cpp: Added.
1430         (bmalloc::StaticMutex::lockSlowCase):
1431         * bmalloc/StaticMutex.h: Added.
1432         (bmalloc::StaticMutex::init):
1433         (bmalloc::StaticMutex::try_lock):
1434         (bmalloc::StaticMutex::lock):
1435         (bmalloc::StaticMutex::unlock):
1436         * bmalloc/VMHeap.h:
1437         (bmalloc::VMHeap::deallocateXSmallPage):
1438         (bmalloc::VMHeap::deallocateSmallPage):
1439         (bmalloc::VMHeap::deallocateMediumPage):
1440         (bmalloc::VMHeap::deallocateLargeRange):
1441         * bmalloc/bmalloc.h:
1442         (bmalloc::api::scavenge): Global replace Mutex => StaticMutex,
1443         since the Heap mutex is a static.
1444
1445 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
1446
1447         bmalloc: AsyncTask should use Mutex instead of std::mutex
1448         https://bugs.webkit.org/show_bug.cgi?id=131865
1449
1450         Reviewed by Gavin Barraclough.
1451
1452         std::mutex is so slow that it makes parallelizing simple tasks through
1453         AsyncTask a net regression. Mutex fixes this.
1454
1455         * bmalloc/AsyncTask.h:
1456         (bmalloc::Function>::AsyncTask):
1457         (bmalloc::Function>::join):
1458         (bmalloc::Function>::runSlowCase):
1459         (bmalloc::Function>::entryPoint):
1460         * bmalloc/Mutex.h:
1461         (bmalloc::Mutex::init):
1462
1463 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
1464
1465         bmalloc: Added an XSmall line size
1466         https://bugs.webkit.org/show_bug.cgi?id=131851
1467
1468         Reviewed by Sam Weinig.
1469
1470         Reduces malloc footprint on Membuster recordings by 10%.
1471
1472         This is a throughput regression, but we're still way ahead of TCMalloc.
1473         I have some ideas for how to recover the regression -- but I wanted to
1474         get this win in first.
1475
1476         Full set of benchmark results:
1477
1478                 bmalloc> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks --measure-heap nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1479                                                                                                 
1480                                                                nopatch                      patch                                Δ
1481                 Peak Memory:
1482                     reddit_memory_warning                      7,896kB                    7,532kB                  ^ 1.05x smaller
1483                     flickr_memory_warning                     12,968kB                   12,324kB                  ^ 1.05x smaller
1484                     theverge_memory_warning                   16,672kB                   15,200kB                   ^ 1.1x smaller
1485
1486                     <geometric mean>                          11,952kB                   11,216kB                  ^ 1.07x smaller
1487                     <arithmetic mean>                         12,512kB                   11,685kB                  ^ 1.07x smaller
1488                     <harmonic mean>                           11,375kB                   10,726kB                  ^ 1.06x smaller
1489
1490                 Memory at End:
1491                     reddit_memory_warning                      7,320kB                    6,856kB                  ^ 1.07x smaller
1492                     flickr_memory_warning                     10,848kB                    9,692kB                  ^ 1.12x smaller
1493                     theverge_memory_warning                   16,380kB                   14,872kB                   ^ 1.1x smaller
1494
1495                     <geometric mean>                          10,916kB                    9,961kB                   ^ 1.1x smaller
1496                     <arithmetic mean>                         11,516kB                   10,473kB                   ^ 1.1x smaller
1497                     <harmonic mean>                           10,350kB                    9,485kB                  ^ 1.09x smaller
1498
1499                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1500                                                                                                 
1501                                                    nopatch                patch                         Δ
1502                 Execution Time:
1503                     churn                            127ms                151ms            ! 1.19x slower
1504                     list_allocate                    130ms                164ms            ! 1.26x slower
1505                     tree_allocate                    109ms                127ms            ! 1.17x slower
1506                     tree_churn                       115ms                120ms            ! 1.04x slower
1507                     facebook                         240ms                259ms            ! 1.08x slower
1508                     fragment                          91ms                131ms            ! 1.44x slower
1509                     fragment_iterate                 105ms                106ms            ! 1.01x slower
1510                     message_one                      260ms                259ms             ^ 1.0x faster
1511                     message_many                     149ms                154ms            ! 1.03x slower
1512                     medium                           194ms                248ms            ! 1.28x slower
1513                     big                              157ms                160ms            ! 1.02x slower
1514
1515                     <geometric mean>                 144ms                163ms            ! 1.13x slower
1516                     <arithmetic mean>                152ms                171ms            ! 1.12x slower
1517                     <harmonic mean>                  137ms                156ms            ! 1.14x slower
1518
1519                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1520                                                                                                 
1521                                                                        nopatch                          patch                                     Δ
1522                 Execution Time:
1523                     churn                                                126ms                          148ms                        ! 1.17x slower
1524                     churn --parallel                                      62ms                           76ms                        ! 1.23x slower
1525                     list_allocate                                        130ms                          164ms                        ! 1.26x slower
1526                     list_allocate --parallel                             120ms                          175ms                        ! 1.46x slower
1527                     tree_allocate                                        111ms                          127ms                        ! 1.14x slower
1528                     tree_allocate --parallel                              95ms                          135ms                        ! 1.42x slower
1529                     tree_churn                                           115ms                          124ms                        ! 1.08x slower
1530                     tree_churn --parallel                                107ms                          126ms                        ! 1.18x slower
1531                     facebook                                             240ms                          276ms                        ! 1.15x slower
1532                     facebook --parallel                                  802ms                        1,088ms                        ! 1.36x slower
1533                     fragment                                              92ms                          130ms                        ! 1.41x slower
1534                     fragment --parallel                                   66ms                          124ms                        ! 1.88x slower
1535                     fragment_iterate                                     109ms                          127ms                        ! 1.17x slower
1536                     fragment_iterate --parallel                           55ms                           64ms                        ! 1.16x slower
1537                     message_one                                          260ms                          260ms                                      
1538                     message_many                                         170ms                          238ms                         ! 1.4x slower
1539                     medium                                               185ms                          250ms                        ! 1.35x slower
1540                     medium --parallel                                    210ms                          334ms                        ! 1.59x slower
1541                     big                                                  150ms                          169ms                        ! 1.13x slower
1542                     big --parallel                                       138ms                          144ms                        ! 1.04x slower
1543
1544                     <geometric mean>                                     135ms                          170ms                        ! 1.26x slower
1545                     <arithmetic mean>                                    167ms                          214ms                        ! 1.28x slower
1546                     <harmonic mean>                                      117ms                          148ms                        ! 1.26x slower
1547
1548                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
1549
1550                                                                     TC                      patch                                Δ
1551                 Peak Memory:
1552                     reddit_memory_warning                     13,836kB                   13,436kB                  ^ 1.03x smaller
1553                     flickr_memory_warning                     24,868kB                   25,188kB                   ! 1.01x bigger
1554                     theverge_memory_warning                   24,504kB                   26,636kB                   ! 1.09x bigger
1555
1556                     <geometric mean>                          20,353kB                   20,812kB                   ! 1.02x bigger
1557                     <arithmetic mean>                         21,069kB                   21,753kB                   ! 1.03x bigger
1558                     <harmonic mean>                           19,570kB                   19,780kB                   ! 1.01x bigger
1559
1560                 Memory at End:
1561                     reddit_memory_warning                      8,656kB                   10,016kB                   ! 1.16x bigger
1562                     flickr_memory_warning                     11,844kB                   13,784kB                   ! 1.16x bigger
1563                     theverge_memory_warning                   18,516kB                   22,748kB                   ! 1.23x bigger
1564
1565                     <geometric mean>                          12,382kB                   14,644kB                   ! 1.18x bigger
1566                     <arithmetic mean>                         13,005kB                   15,516kB                   ! 1.19x bigger
1567                     <harmonic mean>                           11,813kB                   13,867kB                   ! 1.17x bigger
1568
1569                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
1570                                                                                                 
1571                                                         TC                patch                         Δ
1572                 Execution Time:
1573                     churn                            416ms                148ms            ^ 2.81x faster
1574                     list_allocate                    463ms                164ms            ^ 2.82x faster
1575                     tree_allocate                    292ms                127ms             ^ 2.3x faster
1576                     tree_churn                       157ms                120ms            ^ 1.31x faster
1577                     facebook                         327ms                276ms            ^ 1.18x faster
1578                     fragment                         335ms                129ms             ^ 2.6x faster
1579                     fragment_iterate                 344ms                108ms            ^ 3.19x faster
1580                     message_one                      386ms                258ms             ^ 1.5x faster
1581                     message_many                     410ms                154ms            ^ 2.66x faster
1582                     medium                           391ms                245ms             ^ 1.6x faster
1583                     big                              261ms                167ms            ^ 1.56x faster
1584
1585                     <geometric mean>                 332ms                164ms            ^ 2.02x faster
1586                     <arithmetic mean>                344ms                172ms            ^ 1.99x faster
1587                     <harmonic mean>                  317ms                157ms            ^ 2.02x faster
1588
1589         * bmalloc.xcodeproj/project.pbxproj:
1590         * bmalloc/Allocator.cpp:
1591         (bmalloc::Allocator::Allocator): Don't assume that each allocator's
1592         index corresponds with its size. Instead, use the size selection function
1593         explicitly. Now that we have XSmall, some small allocator entries are
1594         unused.
1595
1596         (bmalloc::Allocator::scavenge):
1597         (bmalloc::Allocator::log):
1598         (bmalloc::Allocator::processXSmallAllocatorLog):
1599         (bmalloc::Allocator::allocateSlowCase):
1600         * bmalloc/Allocator.h:
1601         (bmalloc::Allocator::xSmallAllocatorFor):
1602         (bmalloc::Allocator::allocateFastCase):
1603         * bmalloc/Chunk.h:
1604         * bmalloc/Deallocator.cpp:
1605         (bmalloc::Deallocator::scavenge):
1606         (bmalloc::Deallocator::processObjectLog):
1607         (bmalloc::Deallocator::deallocateSlowCase):
1608         (bmalloc::Deallocator::deallocateXSmallLine):
1609         (bmalloc::Deallocator::allocateXSmallLine):
1610         * bmalloc/Deallocator.h:
1611         (bmalloc::Deallocator::deallocateFastCase):
1612         * bmalloc/Heap.cpp:
1613         (bmalloc::Heap::scavenge):
1614         (bmalloc::Heap::scavengeXSmallPages):
1615         (bmalloc::Heap::allocateXSmallLineSlowCase):
1616         * bmalloc/Heap.h:
1617         (bmalloc::Heap::deallocateXSmallLine):
1618         (bmalloc::Heap::allocateXSmallLine):
1619         * bmalloc/LargeChunk.h:
1620         (bmalloc::LargeChunk::get):
1621         (bmalloc::LargeChunk::endTag):
1622         * bmalloc/Line.h:
1623         * bmalloc/MediumAllocator.h:
1624         (bmalloc::MediumAllocator::allocate):
1625         (bmalloc::MediumAllocator::refill):
1626         * bmalloc/ObjectType.cpp:
1627         (bmalloc::objectType):
1628         * bmalloc/ObjectType.h:
1629         (bmalloc::isXSmall):
1630         (bmalloc::isSmall):
1631         (bmalloc::isMedium):
1632         (bmalloc::isLarge):
1633         (bmalloc::isSmallOrMedium): Deleted.
1634         * bmalloc/SegregatedFreeList.h: I boiler-plate copied existing code for
1635         handling small objects. There's probably a reasonable way to share this
1636         code in the future -- I'll look into that once it's stopped changing.
1637
1638         * bmalloc/Sizes.h: Tweaked size classes to make Membuster happy. This
1639         is the main reason things got slower.
1640
1641         * bmalloc/SmallAllocator.h:
1642         (bmalloc::SmallAllocator::allocate):
1643         * bmalloc/SmallTraits.h:
1644         * bmalloc/VMHeap.cpp:
1645         (bmalloc::VMHeap::allocateXSmallChunk):
1646         * bmalloc/VMHeap.h:
1647         (bmalloc::VMHeap::allocateXSmallPage):
1648         (bmalloc::VMHeap::deallocateXSmallPage):
1649         * bmalloc/XSmallAllocator.h: Added.
1650         (bmalloc::XSmallAllocator::isNull):
1651         (bmalloc::XSmallAllocator::canAllocate):
1652         (bmalloc::XSmallAllocator::XSmallAllocator):
1653         (bmalloc::XSmallAllocator::line):
1654         (bmalloc::XSmallAllocator::allocate):
1655         (bmalloc::XSmallAllocator::objectCount):
1656         (bmalloc::XSmallAllocator::derefCount):
1657         (bmalloc::XSmallAllocator::refill):
1658         * bmalloc/XSmallChunk.h: Added.
1659         * bmalloc/XSmallLine.h: Added.
1660         * bmalloc/XSmallPage.h: Added.
1661         * bmalloc/XSmallTraits.h: Added.
1662         * bmalloc/bmalloc.h:
1663         (bmalloc::api::realloc): Boiler-plate copy, as above.
1664
1665 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
1666
1667         MallocBench should scavenge explicitly instead of waiting
1668         https://bugs.webkit.org/show_bug.cgi?id=131661
1669
1670         Reviewed by Andreas Kling.
1671
1672         Added explicit scavenge support to bmalloc. This isn't a memory win,
1673         since bmalloc's per-thread cache is so small. But it makes testing
1674         simpler.
1675
1676         * bmalloc/Allocator.cpp:
1677         (bmalloc::Allocator::~Allocator):
1678         (bmalloc::Allocator::scavenge):
1679         * bmalloc/Allocator.h:
1680         * bmalloc/Cache.cpp:
1681         (bmalloc::Cache::operator new):
1682         (bmalloc::Cache::operator delete):
1683         (bmalloc::Cache::Cache):
1684         (bmalloc::Cache::scavenge):
1685         * bmalloc/Cache.h:
1686         * bmalloc/Deallocator.cpp:
1687         (bmalloc::Deallocator::~Deallocator):
1688         (bmalloc::Deallocator::scavenge):
1689         * bmalloc/Deallocator.h: Factored existing scavenging code into helper
1690         functions, for reuse.
1691
1692         * bmalloc/Heap.cpp:
1693         (bmalloc::sleep):
1694         (bmalloc::Heap::concurrentScavenge):
1695         (bmalloc::Heap::scavenge):
1696         (bmalloc::Heap::scavengeSmallPages):
1697         (bmalloc::Heap::scavengeMediumPages):
1698         (bmalloc::Heap::scavengeLargeRanges):
1699         * bmalloc/Heap.h: Made scavenge sleep duration a parameter. Forced
1700         scavenging -- in response to a benchmark or a low memory warning --
1701         wants to complete as soon as possible, so its sleep duration is 0.
1702
1703         * bmalloc/bmalloc.h:
1704         (bmalloc::api::scavenge):
1705         * bmalloc/mbmalloc.cpp: Exported the scavenge API for MallocBench's use.
1706
1707 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
1708
1709         Use 4kB pages on Mac
1710         https://bugs.webkit.org/show_bug.cgi?id=131658
1711
1712         Reviewed by Sam Weinig.
1713
1714         This reduces memory use a lot on Membuster:
1715
1716                                                                   base                      patch                                Δ
1717                 Execution Time:
1718                     reddit_memory_warning                         18ms                       17ms                   ^ 1.06x faster
1719                     flickr_memory_warning                         34ms                       36ms                   ! 1.06x slower
1720                     theverge_memory_warning                       39ms                       41ms                   ! 1.05x slower
1721
1722                     <geometric mean>                              29ms                       29ms                   ! 1.02x slower
1723                     <arithmetic mean>                             30ms                       31ms                   ! 1.03x slower
1724                     <harmonic mean>                               27ms                       27ms                    ^ 1.0x faster
1725
1726                 Peak Memory:
1727                     reddit_memory_warning                     16,412kB                   16,436kB                    ! 1.0x bigger
1728                     flickr_memory_warning                     30,120kB                   30,184kB                    ! 1.0x bigger
1729                     theverge_memory_warning                   33,408kB                   33,420kB                    ! 1.0x bigger
1730
1731                     <geometric mean>                          25,466kB                   25,499kB                    ! 1.0x bigger
1732                     <arithmetic mean>                         26,647kB                   26,680kB                    ! 1.0x bigger
1733                     <harmonic mean>                           24,181kB                   24,214kB                    ! 1.0x bigger
1734
1735                 Memory at End:
1736                     reddit_memory_warning                      2,404kB                    1,920kB                  ^ 1.25x smaller
1737                     flickr_memory_warning                      3,764kB                    3,072kB                  ^ 1.23x smaller
1738                     theverge_memory_warning                    3,648kB                    3,132kB                  ^ 1.16x smaller
1739
1740                     <geometric mean>                           3,208kB                    2,644kB                  ^ 1.21x smaller
1741                     <arithmetic mean>                          3,272kB                    2,708kB                  ^ 1.21x smaller
1742                     <harmonic mean>                            3,139kB                    2,574kB                  ^ 1.22x smaller
1743
1744
1745         * bmalloc.xcodeproj/project.pbxproj:
1746         * bmalloc/BPlatform.h: Added.
1747         * bmalloc/VMAllocate.h: Only use 16kB pages on iOS because the page size
1748         is 4kB on Mac.
1749
1750 2014-04-14  Alexey Proskuryakov  <ap@apple.com>
1751
1752         Fixed svn:ignore on bmalloc.xcodeproj, it had erroneous leading spaces.
1753
1754         * bmalloc.xcodeproj: Modified property svn:ignore.
1755
1756 2014-04-13  Geoffrey Garen  <ggaren@apple.com>
1757
1758         Fixed some mbmalloc exports
1759         https://bugs.webkit.org/show_bug.cgi?id=131599
1760
1761         Reviewed by Ryosuke Niwa.
1762
1763         * bmalloc.xcodeproj/project.pbxproj: Made some headers a private part
1764         of the project, so we can call them from API.
1765
1766         * bmalloc/mbmalloc.cpp: Marked the mbmalloc functions with default
1767         visibility, so they show up as exported in the .dylib.
1768
1769 2014-04-09  Geoffrey Garen  <ggaren@apple.com>
1770
1771         Put bmalloc headers in the right place
1772         https://bugs.webkit.org/show_bug.cgi?id=131464
1773
1774         Reviewed by Mark Rowe.
1775
1776         * Configurations/bmalloc.xcconfig: Set PRIVATE_HEADERS_FOLDER_PATH to
1777         specify that we don't just want to dump all of our generically-named
1778         headers into /usr/local/include.
1779
1780 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
1781
1782         Made bmalloc more #include friendly
1783         https://bugs.webkit.org/show_bug.cgi?id=131386
1784
1785         Reviewed by Andreas Kling.
1786
1787         Marked a bunch of headers private so they can be used from client code
1788         that #includes bmalloc.h.
1789
1790         Renamed ASSERT macros to BASSERT. This matches their header, which already
1791         had to be renamed, and fixes conflicts with WTF's ASSERT macros.
1792
1793         * bmalloc.xcodeproj/project.pbxproj:
1794         * bmalloc/Allocator.cpp:
1795         (bmalloc::Allocator::allocateSlowCase):
1796         * bmalloc/AsyncTask.h:
1797         (bmalloc::Function>::runSlowCase):
1798         * bmalloc/BAssert.h:
1799         * bmalloc/BoundaryTag.h:
1800         (bmalloc::BoundaryTag::setSize):
1801         * bmalloc/BoundaryTagInlines.h:
1802         (bmalloc::validate):
1803         (bmalloc::BoundaryTag::init):
1804         (bmalloc::BoundaryTag::deallocate):
1805         (bmalloc::BoundaryTag::splitLarge):
1806         (bmalloc::BoundaryTag::allocate):
1807         * bmalloc/Chunk.h:
1808         * bmalloc/Deallocator.cpp:
1809         (bmalloc::Deallocator::processObjectLog):
1810         (bmalloc::Deallocator::deallocateSlowCase):
1811         * bmalloc/Deallocator.h:
1812         (bmalloc::Deallocator::deallocateFastCase):
1813         * bmalloc/FixedVector.h:
1814         (bmalloc::Capacity>::operator):
1815         (bmalloc::Capacity>::push):
1816         (bmalloc::Capacity>::pop):
1817         (bmalloc::Capacity>::shrink):
1818         * bmalloc/Heap.cpp:
1819         (bmalloc::Heap::allocateLarge):
1820         * bmalloc/LargeChunk.h:
1821         (bmalloc::LargeChunk::get):
1822         (bmalloc::LargeChunk::endTag):
1823         * bmalloc/Line.h:
1824         (bmalloc::Line<Traits>::concurrentRef):
1825         (bmalloc::Line<Traits>::deref):
1826         * bmalloc/MediumAllocator.h:
1827         (bmalloc::MediumAllocator::allocate):
1828         * bmalloc/ObjectType.h:
1829         (bmalloc::isSmall):
1830         * bmalloc/Page.h:
1831         (bmalloc::Page<Traits>::ref):
1832         (bmalloc::Page<Traits>::deref):
1833         * bmalloc/PerThread.h:
1834         (bmalloc::PerThread<T>::getSlowCase):
1835         * bmalloc/SegregatedFreeList.cpp:
1836         (bmalloc::SegregatedFreeList::SegregatedFreeList):
1837         (bmalloc::SegregatedFreeList::insert):
1838         * bmalloc/SmallAllocator.h:
1839         (bmalloc::SmallAllocator::allocate):
1840         (bmalloc::SmallAllocator::refill):
1841         * bmalloc/Syscall.h:
1842         * bmalloc/VMAllocate.h:
1843         (bmalloc::vmValidate):
1844         (bmalloc::vmAllocate):
1845         (bmalloc::vmDeallocatePhysicalPagesSloppy):
1846         * bmalloc/Vector.h:
1847         (bmalloc::Vector<T>::operator):
1848         (bmalloc::Vector<T>::pop):
1849         (bmalloc::Vector<T>::shrink):
1850         * bmalloc/XLargeChunk.h:
1851         (bmalloc::XLargeChunk::range):
1852         (bmalloc::XLargeChunk::size):
1853
1854 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
1855
1856         Removed an unused file.
1857
1858         Unreviewed.
1859
1860         * bmalloc/AsyncTask.cpp: Removed.
1861
1862 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1863
1864         Build bmalloc on Mac
1865         https://bugs.webkit.org/show_bug.cgi?id=131333
1866
1867         Reviewed by Mark Rowe.
1868
1869         * Makefile: Added. For make clients.
1870
1871         These files are required for building any project in WebKit. I copied
1872         them from WTF:
1873         * Configurations: Added.
1874         * Configurations/Base.xcconfig: Added.
1875         * Configurations/DebugRelease.xcconfig: Added.
1876         * Configurations/bmalloc.xcconfig: Added.
1877         * Configurations/iOS.xcconfig: Added.
1878         * Configurations/mbmalloc.xcconfig: Added.
1879
1880         * bmalloc.xcodeproj/project.pbxproj: I removed per-project-file stuff
1881         from here because everything is in .xcconfig files now.
1882
1883         I had to fix a bunch of minor warnings, since they're enabled in our
1884         .xcconfig files:
1885
1886         * bmalloc/AsyncTask.h:
1887         (bmalloc::Function>::AsyncTask):
1888         * bmalloc/BAssert.h:
1889         * bmalloc/BoundaryTagInlines.h:
1890         (bmalloc::validate):
1891         * bmalloc/Heap.cpp:
1892         (bmalloc::Heap::Heap):
1893         (bmalloc::Heap::allocateLarge):
1894         (bmalloc::Heap::deallocateLarge):
1895         * bmalloc/Mutex.h:
1896         (bmalloc::Mutex::Mutex): Deleted.
1897         * bmalloc/VMAllocate.h:
1898         (bmalloc::vmValidate):
1899         * bmalloc/mbmalloc.cpp:
1900
1901 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1902
1903         bmalloc: Fixed a leak in the per-thread cache
1904         https://bugs.webkit.org/show_bug.cgi?id=131330
1905
1906         Reviewed by Andreas Kling.
1907
1908         Remember to deallocate our line caches upon thread exit.
1909
1910         * bmalloc/Deallocator.cpp:
1911         (bmalloc::Deallocator::~Deallocator):
1912
1913 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1914
1915         bmalloc: rolled out the tryLock experiment
1916         https://bugs.webkit.org/show_bug.cgi?id=131328
1917
1918         Reviewed by Andreas Kling.
1919
1920         It wasn't a speedup.
1921
1922         * bmalloc.xcodeproj/project.pbxproj:
1923         * bmalloc/Allocator.cpp:
1924         (bmalloc::Allocator::processSmallAllocatorLog):
1925         (bmalloc::Allocator::processMediumAllocatorLog):
1926         * bmalloc/Deallocator.cpp:
1927         (bmalloc::Deallocator::processObjectLog):
1928         (bmalloc::Deallocator::deallocateSlowCase):
1929         (bmalloc::Deallocator::deallocateSmallLine):
1930         (bmalloc::Deallocator::deallocateMediumLine):
1931         * bmalloc/Deallocator.h:
1932         (bmalloc::Deallocator::deallocateFastCase):
1933         * bmalloc/Heap.h:
1934         (bmalloc::Heap::deallocateSmallLine):
1935         (bmalloc::Heap::deallocateMediumLine):
1936         * bmalloc/Line.h:
1937         (bmalloc::Line<Traits>::deref):
1938         * bmalloc/Page.h:
1939         (bmalloc::Page<Traits>::deref):
1940
1941 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1942
1943         bmalloc
1944         https://bugs.webkit.org/show_bug.cgi?id=131170
1945
1946         Reviewed by Andreas Kling.
1947
1948         Initial commit.
1949
1950         * bmalloc: Added.
1951         * bmalloc.xcodeproj: Added.
1952         * bmalloc.xcodeproj/project.pbxproj: Added.
1953         * bmalloc/Algorithm.h: Added.
1954         (bmalloc::max):
1955         (bmalloc::min):
1956         (bmalloc::mask):
1957         (bmalloc::test):
1958         (bmalloc::roundUpToMultipleOf):
1959         (bmalloc::roundDownToMultipleOf):
1960         (bmalloc::sizeOf):
1961         (bmalloc::bitCount):
1962         (bmalloc::isPowerOfTwo):
1963         * bmalloc/Allocator.cpp: Added.
1964         (bmalloc::Allocator::Allocator):
1965         (bmalloc::Allocator::~Allocator):
1966         (bmalloc::Allocator::log):
1967         (bmalloc::Allocator::processSmallAllocatorLog):
1968         (bmalloc::Allocator::processMediumAllocatorLog):
1969         (bmalloc::Allocator::allocateLarge):
1970         (bmalloc::Allocator::allocateXLarge):
1971         (bmalloc::Allocator::allocateMedium):
1972         (bmalloc::Allocator::allocateSlowCase):
1973         * bmalloc/Allocator.h: Added.
1974         (bmalloc::Allocator::smallAllocatorFor):
1975         (bmalloc::Allocator::allocateFastCase):
1976         (bmalloc::Allocator::allocate):
1977         * bmalloc/AsyncTask.cpp: Added.
1978         (bmalloc::AsyncTask<Function>::runSlowCase):
1979         (bmalloc::AsyncTask<Function>::pthreadEntryPoint):
1980         (bmalloc::AsyncTask<Function>::entryPoint):
1981         * bmalloc/AsyncTask.h: Added.
1982         (bmalloc::Function>::AsyncTask):
1983         (bmalloc::Function>::join):
1984         (bmalloc::Function>::run):
1985         (bmalloc::Function>::runSlowCase):
1986         (bmalloc::Function>::pthreadEntryPoint):
1987         (bmalloc::Function>::entryPoint):
1988         * bmalloc/BAssert.h: Added.
1989         * bmalloc/BeginTag.h: Added.
1990         (bmalloc::BeginTag::isInFreeList):
1991         * bmalloc/BoundaryTag.h: Added.
1992         (bmalloc::BoundaryTag::isXLarge):
1993         (bmalloc::BoundaryTag::setXLarge):
1994         (bmalloc::BoundaryTag::isFree):
1995         (bmalloc::BoundaryTag::setFree):
1996         (bmalloc::BoundaryTag::isEnd):
1997         (bmalloc::BoundaryTag::setEnd):
1998         (bmalloc::BoundaryTag::hasPhysicalPages):
1999         (bmalloc::BoundaryTag::setHasPhysicalPages):
2000         (bmalloc::BoundaryTag::isNull):
2001         (bmalloc::BoundaryTag::clear):
2002         (bmalloc::BoundaryTag::size):
2003         (bmalloc::BoundaryTag::setSize):
2004         (bmalloc::BoundaryTag::prev):
2005         (bmalloc::BoundaryTag::next):
2006         * bmalloc/BoundaryTagInlines.h: Added.
2007         (bmalloc::validate):
2008         (bmalloc::validatePrev):
2009         (bmalloc::validateNext):
2010         (bmalloc::BoundaryTag::init):
2011         (bmalloc::BoundaryTag::mergeLargeLeft):
2012         (bmalloc::BoundaryTag::mergeLargeRight):
2013         (bmalloc::BoundaryTag::mergeLarge):
2014         (bmalloc::BoundaryTag::deallocate):
2015         (bmalloc::BoundaryTag::splitLarge):
2016         (bmalloc::BoundaryTag::allocate):
2017         * bmalloc/Cache.cpp: Added.
2018         (bmalloc::Cache::operator new):
2019         (bmalloc::Cache::operator delete):
2020         (bmalloc::Cache::Cache):
2021         (bmalloc::Cache::allocateSlowCase):
2022         (bmalloc::Cache::allocateSlowCaseNullCache):
2023         (bmalloc::Cache::deallocateSlowCase):
2024         (bmalloc::Cache::deallocateSlowCaseNullCache):
2025         * bmalloc/Cache.h: Added.
2026         (bmalloc::Cache::allocator):
2027         (bmalloc::Cache::deallocator):
2028         (bmalloc::Cache::allocateFastCase):
2029         (bmalloc::Cache::deallocateFastCase):
2030         (bmalloc::Cache::allocate):
2031         (bmalloc::Cache::deallocate):
2032         * bmalloc/Chunk.h: Added.
2033         (bmalloc::Chunk::begin):
2034         (bmalloc::Chunk::end):
2035         (bmalloc::Chunk::lines):
2036         (bmalloc::Chunk::pages):
2037         * bmalloc/Deallocator.cpp: Added.
2038         (bmalloc::Deallocator::Deallocator):
2039         (bmalloc::Deallocator::~Deallocator):
2040         (bmalloc::Deallocator::deallocateLarge):
2041         (bmalloc::Deallocator::deallocateXLarge):
2042         (bmalloc::Deallocator::processObjectLog):
2043         (bmalloc::Deallocator::deallocateSlowCase):
2044         (bmalloc::Deallocator::deallocateSmallLine):
2045         (bmalloc::Deallocator::allocateSmallLine):
2046         (bmalloc::Deallocator::deallocateMediumLine):
2047         (bmalloc::Deallocator::allocateMediumLine):
2048         * bmalloc/Deallocator.h: Added.
2049         (bmalloc::Deallocator::deallocateFastCase):
2050         (bmalloc::Deallocator::deallocate):
2051         * bmalloc/EndTag.h: Added.
2052         (bmalloc::EndTag::operator=):
2053         * bmalloc/FixedVector.h: Added.
2054         (bmalloc::FixedVector::begin):
2055         (bmalloc::FixedVector::end):
2056         (bmalloc::FixedVector::size):
2057         (bmalloc::FixedVector::capacity):
2058         (bmalloc::FixedVector::clear):
2059         (bmalloc::FixedVector::isEmpty):
2060         (bmalloc::Capacity>::FixedVector):
2061         (bmalloc::Capacity>::operator):
2062         (bmalloc::Capacity>::push):
2063         (bmalloc::Capacity>::pop):
2064         (bmalloc::Capacity>::shrink):
2065         * bmalloc/Heap.cpp: Added.
2066         (bmalloc::sleep):
2067         (bmalloc::Heap::Heap):
2068         (bmalloc::Heap::concurrentScavenge):
2069         (bmalloc::Heap::scavengeSmallPages):
2070         (bmalloc::Heap::scavengeMediumPages):
2071         (bmalloc::Heap::scavengeLargeRanges):
2072         (bmalloc::Heap::allocateSmallLineSlowCase):
2073         (bmalloc::Heap::allocateMediumLineSlowCase):
2074         (bmalloc::Heap::allocateXLarge):
2075         (bmalloc::Heap::deallocateXLarge):
2076         (bmalloc::Heap::allocateLarge):
2077         (bmalloc::Heap::deallocateLarge):
2078         * bmalloc/Heap.h: Added.
2079         (bmalloc::Heap::deallocateSmallLine):
2080         (bmalloc::Heap::allocateSmallLine):
2081         (bmalloc::Heap::deallocateMediumLine):
2082         (bmalloc::Heap::allocateMediumLine):
2083         * bmalloc/Inline.h: Added.
2084         * bmalloc/LargeChunk.h: Added.
2085         (bmalloc::LargeChunk::begin):
2086         (bmalloc::LargeChunk::end):
2087         (bmalloc::LargeChunk::create):
2088         (bmalloc::LargeChunk::get):
2089         (bmalloc::LargeChunk::beginTag):
2090         (bmalloc::LargeChunk::endTag):
2091         * bmalloc/Line.h: Added.
2092         (bmalloc::Line<Traits>::begin):
2093         (bmalloc::Line<Traits>::end):
2094         (bmalloc::Line<Traits>::concurrentRef):
2095         (bmalloc::Line<Traits>::deref):
2096         * bmalloc/MediumAllocator.h: Added.
2097         (bmalloc::MediumAllocator::isNull):
2098         (bmalloc::MediumAllocator::MediumAllocator):
2099         (bmalloc::MediumAllocator::line):
2100         (bmalloc::MediumAllocator::allocate):
2101         (bmalloc::MediumAllocator::derefCount):
2102         (bmalloc::MediumAllocator::refill):
2103         * bmalloc/MediumChunk.h: Added.
2104         * bmalloc/MediumLine.h: Added.
2105         * bmalloc/MediumPage.h: Added.
2106         * bmalloc/MediumTraits.h: Added.
2107         * bmalloc/Mutex.cpp: Added.
2108         (bmalloc::Mutex::lockSlowCase):
2109         * bmalloc/Mutex.h: Added.
2110         (bmalloc::Mutex::Mutex):
2111         (bmalloc::Mutex::try_lock):
2112         (bmalloc::Mutex::lock):
2113         (bmalloc::Mutex::unlock):
2114         * bmalloc/ObjectType.cpp: Added.
2115         (bmalloc::objectType):
2116         * bmalloc/ObjectType.h: Added.
2117         (bmalloc::isSmallOrMedium):
2118         (bmalloc::isSmall):
2119         * bmalloc/Page.h: Added.
2120         (bmalloc::Page<Traits>::ref):
2121         (bmalloc::Page<Traits>::deref):
2122         (bmalloc::Page<Traits>::refCount):
2123         * bmalloc/PerProcess.h: Added.
2124         (bmalloc::PerProcess::mutex):
2125         (bmalloc::PerProcess<T>::getFastCase):
2126         (bmalloc::PerProcess<T>::get):
2127         (bmalloc::PerProcess<T>::getSlowCase):
2128         * bmalloc/PerThread.h: Added.
2129         (bmalloc::PerThreadStorage<Cache>::get):
2130         (bmalloc::PerThreadStorage<Cache>::init):
2131         (bmalloc::PerThreadStorage::get):
2132         (bmalloc::PerThreadStorage::init):
2133         (bmalloc::PerThread<T>::getFastCase):
2134         (bmalloc::PerThread<T>::get):
2135         (bmalloc::PerThread<T>::destructor):
2136         (bmalloc::PerThread<T>::getSlowCase):
2137         * bmalloc/Range.h: Added.
2138         (bmalloc::Range::Range):
2139         (bmalloc::Range::begin):
2140         (bmalloc::Range::end):
2141         (bmalloc::Range::size):
2142         (bmalloc::Range::operator!):
2143         (bmalloc::Range::operator<):
2144         * bmalloc/SegregatedFreeList.cpp: Added.
2145         (bmalloc::SegregatedFreeList::SegregatedFreeList):
2146         (bmalloc::SegregatedFreeList::insert):
2147         (bmalloc::SegregatedFreeList::takeGreedy):
2148         (bmalloc::SegregatedFreeList::take):
2149         * bmalloc/SegregatedFreeList.h: Added.
2150         * bmalloc/Sizes.h: Added.
2151         * bmalloc/SmallAllocator.h: Added.
2152         (bmalloc::SmallAllocator::isNull):
2153         (bmalloc::SmallAllocator::canAllocate):
2154         (bmalloc::SmallAllocator::SmallAllocator):
2155         (bmalloc::SmallAllocator::line):
2156         (bmalloc::SmallAllocator::allocate):
2157         (bmalloc::SmallAllocator::objectCount):
2158         (bmalloc::SmallAllocator::derefCount):
2159         (bmalloc::SmallAllocator::refill):
2160         * bmalloc/SmallChunk.h: Added.
2161         * bmalloc/SmallLine.h: Added.
2162         * bmalloc/SmallPage.h: Added.
2163         * bmalloc/SmallTraits.h: Added.
2164         * bmalloc/Syscall.h: Added.
2165         * bmalloc/VMAllocate.h: Added.
2166         (bmalloc::vmSize):
2167         (bmalloc::vmValidate):
2168         (bmalloc::vmAllocate):
2169         (bmalloc::vmDeallocate):
2170         (bmalloc::vmDeallocatePhysicalPages):
2171         (bmalloc::vmAllocatePhysicalPages):
2172         (bmalloc::vmDeallocatePhysicalPagesSloppy):
2173         (bmalloc::vmAllocatePhysicalPagesSloppy):
2174         * bmalloc/VMHeap.cpp: Added.
2175         (bmalloc::VMHeap::VMHeap):
2176         (bmalloc::VMHeap::allocateSmallChunk):
2177         (bmalloc::VMHeap::allocateMediumChunk):
2178         (bmalloc::VMHeap::allocateLargeChunk):
2179         * bmalloc/VMHeap.h: Added.
2180         (bmalloc::VMHeap::allocateSmallPage):
2181         (bmalloc::VMHeap::allocateMediumPage):
2182         (bmalloc::VMHeap::allocateLargeRange):
2183         (bmalloc::VMHeap::deallocateSmallPage):
2184         (bmalloc::VMHeap::deallocateMediumPage):
2185         (bmalloc::VMHeap::deallocateLargeRange):
2186         * bmalloc/Vector.h: Added.
2187         (bmalloc::Vector::begin):
2188         (bmalloc::Vector::end):
2189         (bmalloc::Vector::size):
2190         (bmalloc::Vector::capacity):
2191         (bmalloc::Vector::last):
2192         (bmalloc::Vector::pop):
2193         (bmalloc::Vector<T>::Vector):
2194         (bmalloc::Vector<T>::~Vector):
2195         (bmalloc::Vector<T>::operator):
2196         (bmalloc::Vector<T>::push):
2197         (bmalloc::Vector<T>::pop):
2198         (bmalloc::Vector<T>::shrink):
2199         (bmalloc::Vector<T>::reallocateBuffer):
2200         (bmalloc::Vector<T>::shrinkCapacity):
2201         (bmalloc::Vector<T>::growCapacity):
2202         * bmalloc/XLargeChunk.h: Added.
2203         (bmalloc::XLargeChunk::get):
2204         (bmalloc::XLargeChunk::begin):
2205         (bmalloc::XLargeChunk::XLargeChunk):
2206         (bmalloc::XLargeChunk::create):
2207         (bmalloc::XLargeChunk::destroy):
2208         (bmalloc::XLargeChunk::range):
2209         (bmalloc::XLargeChunk::size):
2210         * bmalloc/bmalloc.h: Added.
2211         (bmalloc::api::malloc):
2212         (bmalloc::api::free):
2213         (bmalloc::api::realloc):
2214         * bmalloc/mbmalloc.cpp: Added.
2215