FontCache should only deal with SimpleFontData
[WebKit-https.git] / Source / bmalloc / ChangeLog
1 2015-01-07  Geoffrey Garen  <ggaren@apple.com>
2
3         Make bmalloc work with ASan
4         https://bugs.webkit.org/show_bug.cgi?id=140194
5
6         Reviewed by Mark Lam.
7
8         * bmalloc/BPlatform.h: Added a way to detect Darwin OSes, since we need
9         an OS-specific API to test for loaded runtime libraries.
10
11         * bmalloc/Environment.cpp:
12         (bmalloc::isASanEnabled):
13         (bmalloc::Environment::computeIsBmallocEnabled): Disabled bmalloc if
14         ASan is enabled, since system malloc has the Asan hooks we need.
15
16         You could check for the ASan compile-time flag instead, but doing this
17         check at runtime prepares bmalloc for a world where it is a dynamic
18         library that might be loaded into projects it did not compile with.
19
20 2015-01-05  Geoffrey Garen  <ggaren@apple.com>
21
22         Fix up bmalloc's PerThread for use on Linux
23         https://bugs.webkit.org/show_bug.cgi?id=139804
24
25         Reviewed by Anders Carlsson.
26
27         The previous implementation was a bit slow.
28
29         * bmalloc/PerThread.h:
30         (bmalloc::PerThreadStorage<Cache>::get):
31         (bmalloc::PerThreadStorage::get):
32         (bmalloc::PerThreadStorage::init): Added a catch-all cross-platform Unix
33         way to do fast per-thread access without taking a lock every time. This
34         probably works on all the platforms we care about, and it matches other
35         techniques we use elsewhere in WebKit.
36
37         (bmalloc::PerThread<T>::getFastCase): Removed the conditional from
38         this class because PerThreadStorage now encapsulates everything that
39         needs to be conditional.
40
41         (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Deleted.
42
43 2014-12-26  Dan Bernstein  <mitz@apple.com>
44
45         <rdar://problem/19348208> REGRESSION (r177027): iOS builds use the wrong toolchain
46         https://bugs.webkit.org/show_bug.cgi?id=139950
47
48         Reviewed by David Kilzer.
49
50         * Configurations/Base.xcconfig: Only define TOOLCHAINS when building for OS X, doing so
51         in a manner that works with Xcode 5.1.1.
52
53 2014-12-15  Geoffrey Garen  <ggaren@apple.com>
54
55         Safari crashes when you set Malloc environment variables
56         https://bugs.webkit.org/show_bug.cgi?id=139656
57
58         Reviewed by Michael Saboff.
59
60         I forgot to cover the realloc() case. Whoops. (OoPS?)
61
62         This time around, I ran the full MallocBench test suite in Malloc=1
63         mode, and it passed.
64
65         * bmalloc/Allocator.cpp:
66         (bmalloc::Allocator::reallocate):
67         * bmalloc/Allocator.h: Pushed realloc() logic down into the allocator.
68         It needs to be down there so that we can do the short-circuiting check
69         for whether bmalloc is enabled first.
70
71         Also added the check.
72
73         * bmalloc/Cache.cpp:
74         (bmalloc::Cache::scavenge):
75         (bmalloc::Cache::Cache):
76         (bmalloc::Cache::reallocateSlowCaseNullCache):
77         * bmalloc/Cache.h:
78         (bmalloc::Cache::deallocator):
79         (bmalloc::Cache::reallocate): Ditto.
80
81         * bmalloc/bmalloc.h:
82         (bmalloc::api::free):
83         (bmalloc::api::realloc): Ditto.
84
85         (bmalloc::api::scavenge): Pushed this down into Cache to match the
86         surrounding functions.
87
88 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
89
90         bmalloc should support system memory analysis tools (part 2)
91         https://bugs.webkit.org/show_bug.cgi?id=139565
92
93         Reviewed by Mark Lam.
94
95         This patch actually queries the environment to see if memory analysis
96         tools have been enabled.
97
98         * bmalloc/Deallocator.cpp:
99         (bmalloc::Deallocator::scavenge): Don't process the object log if
100         we've disabled bmalloc because it will be full of invalid nullptrs.
101
102         * bmalloc/Environment.cpp:
103         (bmalloc::isMallocEnvironmentVariableSet): Test for the list of known
104         Malloc debugging flags. I also added a plain "Malloc" catch-all for
105         when you want to disable bmalloc without enabling any kind of funny
106         business.
107
108         It would be slightly nicer just to iterate the list of environment
109         variables and strstr them, but getenv is the more portable option,
110         and performance here doesn't really matter.
111
112         (bmalloc::isLibgmallocEnabled): Test for the libgmalloc insertion
113         environment variable.
114
115         (bmalloc::Environment::computeIsBmallocEnabled):
116
117 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
118
119         Try to fix the iOS simulator build.
120
121         #include the declaration of malloc / free.
122
123         * bmalloc/Allocator.cpp:
124         * bmalloc/Deallocator.cpp:
125
126 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
127
128         Try to fix the build.
129
130         * bmalloc.xcodeproj/project.pbxproj: Marked a header exported.
131
132 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
133
134         bmalloc should support system memory analysis tools (part 1)
135         https://bugs.webkit.org/show_bug.cgi?id=139559
136
137         Reviewed by Mark Lam.
138
139         This patch adds the hooks to disable bmalloc at runtime if certain
140         environment variables are set, but doesn't actually read from the
141         environment yet.
142
143         No performance change.
144
145         * bmalloc.xcodeproj/project.pbxproj: Added the Environment class, which
146         we'll use to read environment variables and see if memory analysis tools
147         have been enabled.
148
149         * bmalloc/Allocator.cpp:
150         (bmalloc::Allocator::Allocator):
151         (bmalloc::Allocator::allocateSlowCase): Added a hook to disable bmalloc
152         on the allocation path. We cache the setting to make the check fast.
153
154         * bmalloc/Allocator.h: Interface changes.
155
156         * bmalloc/Cache.cpp:
157         (bmalloc::Cache::Cache): Pass a heap pointer through to our allocator
158         and deallocator. This main purpose is to enable them to query the
159         environment for whether bmalloc is enabled; but this is also a slightly
160         cleaner way to guarantee to them that the Heap has been pre-initialized.
161
162         * bmalloc/Deallocator.cpp:
163         (bmalloc::Deallocator::Deallocator): If bmalloc is disable, artificially
164         fill the object log to force us to take the slow path on all deallocations.
165
166         (bmalloc::Deallocator::deallocateSlowCase): Do the disabled check.
167
168         * bmalloc/Deallocator.h: Interface changes.
169
170         * bmalloc/Environment.cpp: Added.
171         (bmalloc::Environment::Environment):
172         (bmalloc::Environment::computeIsBmallocEnabled):
173         * bmalloc/Environment.h: Added.
174         (bmalloc::Environment::isBmallocEnabled): This is the class that will
175         encapsulate looking for environment variables that turn on heap
176         analysis tools.
177
178         * bmalloc/Heap.h:
179         (bmalloc::Heap::environment):
180
181         * bmalloc/Mutex.h:
182         (bmalloc::Mutex::Mutex):
183         * bmalloc/StaticMutex.h: A little refactoring to clarify these comments,
184         since I got super confused about them while writing this patch.
185
186         * bmalloc/VMHeap.cpp: Fixed an #include.
187
188 2014-12-09  David Kilzer  <ddkilzer@apple.com>
189
190         Switch from using PLATFORM_NAME to SDK selectors in ANGLE, bmalloc, gtest, JavaScriptCore, WTF
191         <http://webkit.org/b/139212>
192
193         Reviewed by Joseph Pecoraro.
194
195         * Configurations/Base.xcconfig:
196         - Only set GCC_ENABLE_OBJC_GC, GCC_MODEL_TUNING and TOOLCHAINS
197           on OS X.
198         * Configurations/DebugRelease.xcconfig:
199         - Only set MACOSX_DEPLOYMENT_TARGET and SDKROOT on OS X.
200
201 2014-11-07  Geoffrey Garen  <ggaren@apple.com>
202
203         bmalloc uses 8X more virtual memory than necessary
204         https://bugs.webkit.org/show_bug.cgi?id=138495
205
206         Reviewed by Mark Lam.
207
208         iOS has a per-process virtual memory cap around 1GB, so there's some
209         value to not going totally ham with virtual memory.
210
211         We currently use about 8X the necessary amount:
212             - 2X to align our VM allocation
213             - 4X to reserve small / medium / (2) large chunk VM ranges per superchunk
214
215         We can cut that down:
216             - Return the unaligned portion of our VM allocation (-2X)
217             - Use all the chunks in a superchunk, instead of allocating one
218               chunk per superchunk (-4X)
219
220         * bmalloc/Algorithm.h:
221         (bmalloc::roundUpToMultipleOf): Added a non-constant version of this
222         function so we can call it with getpagesize() at runtime.
223
224         * bmalloc/Chunk.h:
225         * bmalloc/LargeChunk.h:
226         (bmalloc::LargeChunk::create): Deleted. Instead of each chunk allocating
227         its own VM, VMHeap allocates the superchunk and all the chunks in it at a time.
228
229         * bmalloc/VMAllocate.h:
230         (bmalloc::vmValidate):
231         (bmalloc::vmAllocate): ASSERT that mmap succeeds to make crashes clearer
232         if it does not succeed. Allocate precisely, and give back the extra.
233
234         * bmalloc/VMHeap.cpp:
235         (bmalloc::VMHeap::allocateSuperChunk):
236         (bmalloc::VMHeap::allocateSmallChunk): Deleted.
237         (bmalloc::VMHeap::allocateMediumChunk): Deleted.
238         (bmalloc::VMHeap::allocateLargeChunk): Deleted. Use all the chunks
239         in a superchunk, instead of just one.
240
241         * bmalloc/VMHeap.h:
242         (bmalloc::VMHeap::allocateSmallPage):
243         (bmalloc::VMHeap::allocateMediumPage):
244         (bmalloc::VMHeap::allocateLargeRange):
245         * bmalloc/XLargeChunk.h:
246         (bmalloc::XLargeChunk::create): Updated to match changes above.
247
248 2014-11-01  David Kilzer  <ddkilzer@apple.com>
249
250         JavaScriptCore is missing debug info for bmalloc because libbmalloc.a is stripped
251         <https://webkit.org/b/138286>
252         <rdar://problem/18847087>
253
254         Reviewed by Dan Bernstein.
255
256         * Configurations/bmalloc.xcconfig: Set STRIP_INSTALLED_PRODUCT
257         to NO for the target that produces libbmalloc.a so that the
258         debug symbols will be linked into JavaScriptCore and end up in
259         its dSYM file.
260
261 2014-10-30  Dana Burkart  <dburkart@apple.com>
262
263         <rdar://problem/18821260> Prepare for the mysterious future
264
265         Reviewed by Lucas Forschler.
266
267         * Configurations/Base.xcconfig:
268         * Configurations/DebugRelease.xcconfig:
269
270 2014-09-24  Geoffrey Garen  <ggaren@apple.com>
271
272         bmalloc: cleaned up fast path vs slow path
273         https://bugs.webkit.org/show_bug.cgi?id=137081
274
275         Reviewed by Sam Weinig.
276
277         Might be a 1% speedup on MallocBench. Also cleans up the code a bit.
278
279         * bmalloc/Allocator.cpp:
280         (bmalloc::Allocator::Allocator): Merged the small and medium range
281         caches, just like the small and medium allocators. Ranges are abstract
282         objects that don't really care whether they hold small or medium objects,
283         so they don't need to be segregated.
284
285         (bmalloc::Allocator::scavenge): Ditto.
286
287         (bmalloc::Allocator::allocateBumpRangeSlowCase):
288         (bmalloc::Allocator::allocateBumpRange): Same thing here, except that
289         we do care a tiny bit, because we need to specify small vs medium when
290         allocating new ranges from the heap, to ensure that the heap allocates
291         from the right segment of VM.
292
293         (bmalloc::Allocator::allocateLarge):
294         (bmalloc::Allocator::allocateXLarge): NO_INLINE because this was clouding
295         up the fast path. Large allocation performance is dominated by allocation
296         logic and initialization, so inlining it doesn't help.
297
298         (bmalloc::Allocator::allocateSlowCase): Slow path got a bit cleaner since
299         it doesn't need to distinguish small vs medium objects.
300
301         (bmalloc::Allocator::allocateSmallBumpRange): Deleted.
302         (bmalloc::Allocator::allocateMediumBumpRange): Deleted.
303
304         * bmalloc/Allocator.h:
305         * bmalloc/BumpRange.h:
306
307         * bmalloc/Cache.cpp:
308         (bmalloc::Cache::allocateSlowCase): Deleted.
309         (bmalloc::Cache::deallocateSlowCase): Deleted.
310         * bmalloc/Cache.h:
311         (bmalloc::Cache::allocate):
312         (bmalloc::Cache::deallocate):
313         (bmalloc::Cache::allocateFastCase): Deleted.
314         (bmalloc::Cache::deallocateFastCase): Deleted. Removed the Cache slow
315         paths. The downside to this change is that the fast path branches to two
316         distinct failure cases instead of one. The upside is that the slow path
317         doesn't need to re-read the segment register, which is not as cheap as a
318         normal register, and it doesn't need to do an extra level of function 
319         call. Seems to be worth it.
320
321         * bmalloc/Deallocator.h:
322         * bmalloc/Heap.cpp:
323         (bmalloc::Heap::refillSmallBumpRangeCache):
324         (bmalloc::Heap::refillMediumBumpRangeCache):
325         * bmalloc/Heap.h: Updated for interface changes.
326
327         * bmalloc/Sizes.h: The most ranges a cache will hold is the number of
328         small lines in a page / 2, since any other free lines will coalesce
329         with their neighbors.
330
331 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
332
333         Rolled out r173346.
334
335             bmalloc should honor the FastMalloc statistics API
336             https://bugs.webkit.org/show_bug.cgi?id=136592
337
338         This didn't really work. Because we allow ranges with and without
339         physical pages to merge, and we allow double-committing and
340         double-decommitting, we can't rely on commit actions to track memory
341         footprint.
342
343         * bmalloc/Heap.cpp:
344         (bmalloc::Heap::size): Deleted.
345         (bmalloc::Heap::capacity): Deleted.
346         * bmalloc/Heap.h:
347         * bmalloc/VMHeap.cpp:
348         (bmalloc::VMHeap::VMHeap):
349         (bmalloc::VMHeap::allocateSmallChunk):
350         (bmalloc::VMHeap::allocateMediumChunk):
351         (bmalloc::VMHeap::allocateLargeChunk):
352         * bmalloc/VMHeap.h:
353         (bmalloc::VMHeap::allocateSmallPage):
354         (bmalloc::VMHeap::allocateMediumPage):
355         (bmalloc::VMHeap::allocateLargeRange):
356         (bmalloc::VMHeap::deallocateSmallPage):
357         (bmalloc::VMHeap::deallocateMediumPage):
358         (bmalloc::VMHeap::deallocateLargeRange):
359         (bmalloc::VMHeap::size): Deleted.
360         (bmalloc::VMHeap::capacity): Deleted.
361         * bmalloc/bmalloc.h:
362         (bmalloc::api::heapSize): Deleted.
363         (bmalloc::api::heapCapacity): Deleted.
364
365 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
366
367         bmalloc: Allocation should be more precise
368         https://bugs.webkit.org/show_bug.cgi?id=136993
369
370         Reviewed by Gavin Barraclough.
371
372         13% reduction in heap size on the MallocBench *_memory_warning benchmarks.
373
374         This patch teaches the allocator to merge adjacent free lines into a
375         single allocatable range. This allows us to shrink the size of an
376         individual line without increasing fragmentation or the rate of allocator
377         slow paths.
378
379         We'll only take more slow paths when available memory is sparse, which
380         is exactly when it's worth it. When available memory is dense, we'll
381         take fewer slow paths.
382
383         * bmalloc.xcodeproj/project.pbxproj:
384         * bmalloc/Algorithm.h:
385         (bmalloc::divideRoundingUp):
386
387         * bmalloc/Allocator.cpp:
388         (bmalloc::Allocator::Allocator): Updated for interface changes.
389
390         (bmalloc::Allocator::scavenge): Scavenge by object instead of by line.
391         Now that we merge lines, it's not convenient to scavenge by line.
392
393         (bmalloc::Allocator::allocateSmallBumpRange):
394         (bmalloc::Allocator::allocateMediumBumpRange): Allocate whole ranges
395         instead of individual lines.
396
397         (bmalloc::Allocator::allocateSlowCase):
398         (bmalloc::Allocator::allocateSmallLine): Deleted.
399         (bmalloc::Allocator::allocateMediumLine): Deleted.
400         (bmalloc::Allocator::allocateMedium): Deleted.
401         * bmalloc/Allocator.h:
402         (bmalloc::Allocator::allocateFastCase): Folded medium allocations
403         into the standard fast path with small allocations. Since a BumpAllocator
404         just allocates out of an arbitrary range, it doesn't need to distinguish
405         between small and medium lines.
406
407         * bmalloc/BumpAllocator.h:
408         (bmalloc::BumpAllocator::size):
409         (bmalloc::BumpAllocator::BumpAllocator):
410         (bmalloc::BumpAllocator::init):
411         (bmalloc::BumpAllocator::refill):
412         (bmalloc::BumpAllocator::line): Deleted. No need to track line information
413         anymore: the heap just gives us a pointer and a pre-computed number of
414         objects, and we allocate them.
415
416         * bmalloc/Deallocator.cpp:
417         (bmalloc::Deallocator::processObjectLog): Updated for interface changes.
418
419         * bmalloc/Heap.cpp:
420         (bmalloc::Heap::Heap):
421         (bmalloc::Heap::initializeLineMetadata): Pre-compute precise metadata
422         detailing where all objects will lie in memory. After we merge two lines,
423         we might allocate an object that spans from one line to the next. This
424         metadata details which bits of memory overlap in that way, and how they
425         overlap.
426
427         (bmalloc::Heap::refillSmallBumpRangeCache):
428         (bmalloc::Heap::refillMediumBumpRangeCache): Scan a whole page at a time,
429         and merge adjacent free lines into BumpRanges.
430
431         (bmalloc::Heap::allocateSmallPage):
432         (bmalloc::Heap::allocateMediumPage):
433         (bmalloc::Heap::deallocateSmallLine):
434         (bmalloc::Heap::deallocateMediumLine): Track pages rather than lines,
435         since we scan for free memory a page at a time.
436
437         (bmalloc::Heap::allocateSmallLineSlowCase): Deleted.
438         (bmalloc::Heap::allocateMediumLineSlowCase): Deleted. Folded into the
439         fast path.
440
441         * bmalloc/Heap.h:
442         (bmalloc::Heap::derefSmallLine):
443         (bmalloc::Heap::derefMediumLine):
444         (bmalloc::Heap::deallocateSmallLine): Deleted.
445         (bmalloc::Heap::allocateSmallLine): Deleted.
446         (bmalloc::Heap::deallocateMediumLine): Deleted.
447         (bmalloc::Heap::allocateMediumLine): Deleted. Updated for interface changes.
448
449         * bmalloc/Line.h:
450         (bmalloc::Line<Traits>::ref):
451         (bmalloc::Line<Traits>::deref):
452         (bmalloc::Line<Traits>::concurrentRef): Deleted. We don't pass a derefCount
453         anymore, since we only ever deref by 1 now.
454
455         * bmalloc/MediumAllocator.h:
456         (bmalloc::MediumAllocator::isNull): Deleted.
457         (bmalloc::MediumAllocator::MediumAllocator): Deleted.
458         (bmalloc::MediumAllocator::line): Deleted.
459         (bmalloc::MediumAllocator::allocate): Deleted.
460         (bmalloc::MediumAllocator::derefCount): Deleted.
461         (bmalloc::MediumAllocator::refill): Deleted.
462         (bmalloc::MediumAllocator::clear): Deleted. Deleted some code that's
463         been dead for a while, since it doesn't build anymore with this patch.
464
465         * bmalloc/Page.h:
466         (bmalloc::Page::sizeClass):
467         (bmalloc::Page::setSizeClass):
468         (bmalloc::Page::smallSizeClass): Deleted.
469         (bmalloc::Page::setSmallSizeClass): Deleted. Renamed setSmallSizeClass
470         to sizeClass, since we use it for medium sizes too.
471
472         * bmalloc/Sizes.h:
473         (bmalloc::Sizes::sizeClass):
474         (bmalloc::Sizes::objectSize): Shrank line sizes to save memory.
475
476         (bmalloc::Sizes::smallSizeClassFor): Deleted.
477         (bmalloc::Sizes::mediumSizeClassFor): Deleted.
478
479         * bmalloc/bmalloc.h:
480         (bmalloc::api::realloc): Now that we have precise objects sizes, realloc
481         can be a bit more precise. It also has to be, since we can't guarantee
482         that an object ends at the end of a line anymore.
483
484 2014-09-19  Daniel Bates  <dabates@apple.com>
485
486         Always assume internal SDK when building configuration Production
487         https://bugs.webkit.org/show_bug.cgi?id=136925
488         <rdar://problem/18362399>
489
490         Reviewed by Dan Bernstein.
491
492         * Configurations/Base.xcconfig:
493
494 2014-09-16  Geoffrey Garen  <ggaren@apple.com>
495
496         bmalloc: moved line caches from the deallocator to the allocator
497         https://bugs.webkit.org/show_bug.cgi?id=136868
498
499         Reviewed by Gavin Barraclough.
500
501         I did this mostly as a simplification, to make it easier to change the
502         allocation strategy.
503
504         No throughput change on MallocBench. Saves about 50kB.
505
506         Since the deallocator needs to lock the heap when freeing lines anyway,
507         there isn't much benefit to giving the deallocator a local cache of
508         deallocated lines.
509
510         We still give the allocator a local cache of lines because that does
511         reduce the frequency at which it needs to lock the heap in order to
512         acquire more lines.
513
514         * bmalloc/Allocator.cpp:
515         (bmalloc::Allocator::scavenge):
516         (bmalloc::Allocator::allocateSmallLine):
517         (bmalloc::Allocator::allocateMediumLine):
518         (bmalloc::Allocator::allocateMedium):
519         (bmalloc::Allocator::allocateSlowCase):
520         * bmalloc/Allocator.h:
521         * bmalloc/Deallocator.cpp:
522         (bmalloc::Deallocator::Deallocator):
523         (bmalloc::Deallocator::scavenge):
524         (bmalloc::Deallocator::processObjectLog):
525         (bmalloc::Deallocator::deallocateSmallLine): Deleted.
526         (bmalloc::Deallocator::allocateSmallLine): Deleted.
527         (bmalloc::Deallocator::deallocateMediumLine): Deleted.
528         (bmalloc::Deallocator::allocateMediumLine): Deleted.
529         * bmalloc/Deallocator.h:
530
531         * bmalloc/Sizes.h:
532         * bmalloc/VMAllocate.h: Took the opportunity to make the line cache size
533         exactly one page in size. That's about what we were shooting for anyway,
534         and it may make it easier to switch to per-page allocation in future.
535
536 2014-09-15  Geoffrey Garen  <ggaren@apple.com>
537
538         bmalloc: allocate small and medium objects using the same bump pointer class
539         https://bugs.webkit.org/show_bug.cgi?id=136843
540
541         Reviewed by Gavin Barraclough.
542
543         4% speedup on MallocBench.
544
545         Now that medium-sized objects have dedicated per-size allocators, they
546         don't need to use an arbitrary bump pointer allocator. This means that
547         every allocator knows how many objects it will allocate from the start,
548         and we don't need a post-processing step to adjust refcounts based on
549         real allocation count.
550
551         * bmalloc.xcodeproj/project.pbxproj: Renamed SmallAllocator to BumpAllocator
552         since it's used for small and medium objects now.
553
554         * bmalloc/Allocator.cpp:
555         (bmalloc::Allocator::Allocator): Updated to use new interface.
556         (bmalloc::Allocator::scavenge): To "retire" an allocator, we just need
557         to make sure that we finish allocating all the objects in it.
558
559         (bmalloc::Allocator::allocateMedium):
560         (bmalloc::Allocator::allocateSlowCase):
561         (bmalloc::Allocator::retire): Deleted.
562         (bmalloc::Allocator::processSmallAllocatorLog): Deleted.
563         (bmalloc::Allocator::processMediumAllocatorLog): Deleted.
564         * bmalloc/Allocator.h:
565         (bmalloc::Allocator::allocateFastCase): Removed abstractions and data
566         used to post-process an allocator based on how many objects it allocated.
567
568         * bmalloc/BumpAllocator.h: Copied from Source/bmalloc/bmalloc/SmallAllocator.h.
569         (bmalloc::BumpAllocator::BumpAllocator):
570         (bmalloc::BumpAllocator::init):
571         (bmalloc::BumpAllocator::line):
572         (bmalloc::BumpAllocator::validate):
573         (bmalloc::BumpAllocator::allocate):
574         (bmalloc::BumpAllocator::refill):
575         (bmalloc::BumpAllocator::clear): Updated these functions to be agnostic
576         about the kinds of lines they allocate into. In some cases, the line
577         type must be provided as a template parameter by the caller.
578
579         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
580         (bmalloc::SmallAllocator::line): Deleted.
581         (bmalloc::SmallAllocator::allocate): Deleted.
582         (bmalloc::SmallAllocator::objectCount): Deleted.
583         (bmalloc::SmallAllocator::derefCount): Deleted.
584         (bmalloc::SmallAllocator::refill): Deleted.
585         (bmalloc::SmallAllocator::clear): Deleted.
586
587         * bmalloc/ObjectType.h:
588         (bmalloc::isMedium):
589
590         * bmalloc/SmallAllocator.h:
591         (bmalloc::SmallAllocator::isNull): Deleted.
592         (bmalloc::SmallAllocator::canAllocate): Deleted.
593         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
594         (bmalloc::SmallAllocator::line): Deleted.
595         (bmalloc::SmallAllocator::allocate): Deleted.
596         (bmalloc::SmallAllocator::objectCount): Deleted.
597         (bmalloc::SmallAllocator::derefCount): Deleted.
598         (bmalloc::SmallAllocator::refill): Deleted.
599         (bmalloc::SmallAllocator::clear): Deleted.
600
601 2014-09-12  Geoffrey Garen  <ggaren@apple.com>
602
603         Fixed a goof in bmalloc Vector sizing
604         https://bugs.webkit.org/show_bug.cgi?id=136795
605
606         Reviewed by Gavin Barraclough and Sam Weinig.
607
608         We want our minimum vector to be page-sized since the OS will give us
609         a page no matter what -- but we want that many bytes, and not enough
610         bytes to store that many elements.
611
612         * bmalloc/Vector.h: Math is hard.
613
614 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
615
616         bmalloc should segregate medium-sized objects by line like it does for small-sized objects
617         https://bugs.webkit.org/show_bug.cgi?id=136693
618
619         Reviewed by Gavin Barraclough.
620
621         4% reduction in heap size on the MallocBench *_memory_warning benchmarks.
622
623         No throughput change.
624
625         We keep an array of medium allocators, just like our array of small
626         allocators.
627
628         In future, we can simplify the allocation fast path by merging the small
629         and medium allocator arrays. For now, this is the simplest change that
630         gets the win.
631
632         * bmalloc/Allocator.cpp:
633         (bmalloc::Allocator::Allocator):
634         (bmalloc::Allocator::scavenge):
635         (bmalloc::Allocator::allocateMedium):
636         * bmalloc/Allocator.h:
637         * bmalloc/Sizes.h:
638         (bmalloc::Sizes::mediumSizeClassFor):
639
640 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
641
642         Reviewed by Sam Weinig.
643
644         Renamed log => retire for clarity.
645
646         * bmalloc/Allocator.cpp:
647         (bmalloc::Allocator::scavenge):
648         (bmalloc::Allocator::retire):
649         (bmalloc::Allocator::allocateMedium):
650         (bmalloc::Allocator::allocateSlowCase):
651         (bmalloc::Allocator::log): Deleted.
652         * bmalloc/Allocator.h:
653
654 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
655
656         bmalloc: eager scavenge leaves behind a bogus allocator
657         https://bugs.webkit.org/show_bug.cgi?id=136743
658
659         Reviewed by Sam Weinig.
660
661         Be sure to clear the allocator after logging it in the eager scavenge
662         case, so that we don't later try to allocate out of the lines that we
663         have thrown away.
664
665         We didn't need to do this previously because scavenge would only happen
666         at thread exit time, after which no further allocation from the per-thread
667         cache would take place.
668
669         * bmalloc/Allocator.cpp:
670         (bmalloc::Allocator::scavenge):
671         * bmalloc/MediumAllocator.h:
672         (bmalloc::MediumAllocator::clear):
673         * bmalloc/SmallAllocator.h:
674         (bmalloc::SmallAllocator::clear):
675
676 2014-09-05  Geoffrey Garen  <ggaren@apple.com>
677
678         bmalloc should honor the FastMalloc statistics API
679         https://bugs.webkit.org/show_bug.cgi?id=136592
680
681         Reviewed by Gavin Barraclough.
682
683         We do this by tracking "size" and "capacity" in the VM heap.
684
685         The VM heap's "capacity" is all the VM we ever allocated.
686
687         The VM heap's "size" the subset of VM currently held onto by the
688         VM heap (and therefore not in use by the regular heap).
689
690         Somewhat ironically, reducing the process's memory footprint, increases
691         the size of the VM heap, since the VM heap holds the pages that are
692         purely virtual and not physical.
693
694         * bmalloc/Heap.cpp:
695         (bmalloc::Heap::size):
696         (bmalloc::Heap::capacity):
697         * bmalloc/Heap.h:
698         * bmalloc/VMHeap.cpp:
699         (bmalloc::VMHeap::VMHeap):
700         (bmalloc::VMHeap::allocateSmallChunk):
701         (bmalloc::VMHeap::allocateMediumChunk):
702         (bmalloc::VMHeap::allocateLargeChunk):
703         * bmalloc/VMHeap.h:
704         (bmalloc::VMHeap::size):
705         (bmalloc::VMHeap::capacity):
706         (bmalloc::VMHeap::allocateSmallPage):
707         (bmalloc::VMHeap::allocateMediumPage):
708         (bmalloc::VMHeap::allocateLargeRange):
709         (bmalloc::VMHeap::deallocateSmallPage):
710         (bmalloc::VMHeap::deallocateMediumPage):
711         (bmalloc::VMHeap::deallocateLargeRange):
712         * bmalloc/bmalloc.h:
713         (bmalloc::api::heapSize):
714         (bmalloc::api::heapCapacity):
715
716 2014-09-02  Geoffrey Garen  <ggaren@apple.com>
717
718         bmalloc crashes on the EWS bots (due to bad large object allocation)
719         https://bugs.webkit.org/show_bug.cgi?id=136469
720
721         Reviewed by Andreas Kling.
722
723         It's possible to convince bmalloc to perform a bad large object allocation,
724         through these steps:
725
726         (1) Insert object A into freelist F0.
727
728         (2) Split, merge and split again A's neighbors such that object B is
729         inserted into freelist F0, with boundary tag and size equal to object A,
730         but pointer not completely equal to object A. Put object B at the head of F0.
731
732         (3) Allocate some other object from F0, swapping its position in the
733         freelist with object B, such that object A is now ahead of object B.
734
735         --> Now, the next allocation for size A/B will allocate object A, which
736         has a slightly wrong idea about where the object actually begins.
737         Immediately, you'll corrupt a little memory, and over time, you'll also
738         corrupt boundary tag metadata.
739
740         The solution is to store the begin pointer in the boundary tag. Luckily,
741         this doesn't make the tag any bigger, and it's not a noticeable slowdown
742         on MallocBench.
743
744         * bmalloc/Algorithm.h:
745         (bmalloc::rightShift):
746         * bmalloc/BeginTag.h:
747         (bmalloc::BeginTag::isInFreeList): This is the bug fix. Make sure to
748         validate the start pointer when popping off the free list. Through a
749         very uncommon set of steps, it is possible to have an item in the free
750         list that is valid by all accounts except for its start pointer.
751
752         * bmalloc/BoundaryTag.h:
753         (bmalloc::BoundaryTag::compactBegin):
754         (bmalloc::BoundaryTag::setRange):
755         (bmalloc::BoundaryTag::setSize): Deleted. Record a compact version of the
756         start pointer. We don't need the whole pointer -- just the offset, in
757         largeAlignment increments, into the relevant boundary tag bucket.
758
759         * bmalloc/BoundaryTagInlines.h:
760         (bmalloc::validateNext):
761         (bmalloc::BoundaryTag::init):
762         (bmalloc::BoundaryTag::mergeLarge):
763         (bmalloc::BoundaryTag::splitLarge):
764         * bmalloc/SegregatedFreeList.cpp:
765         (bmalloc::SegregatedFreeList::insert):
766         (bmalloc::SegregatedFreeList::takeGreedy):
767         (bmalloc::SegregatedFreeList::take): Provide the whole range instead of
768         the size when establishing a boundary tag, as required by the new
769         interface.
770
771         * bmalloc/Sizes.h:
772
773 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
774
775         Fixed a bmalloc crash seen on the EWS bot
776         https://bugs.webkit.org/show_bug.cgi?id=135955
777
778         Reviewed by Andreas Kling.
779
780         * bmalloc/Syscall.h: Some CG APIs vm_copy their input buffers. If the
781         input buffer is a malloc region, that region will get marked Copy-On-Write
782         by the kernel. Calls to madvise() for COW regions fail and return EINVAL
783         on older OS X's. In 10.10, they still fail, but they do not return
784         EINVAL.
785
786         So, we can only ASSERT that our syscalls succeed starting with 10.10.
787
788 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
789
790         Fixed the bmalloc build
791         https://bugs.webkit.org/show_bug.cgi?id=135953
792
793         Reviewed by Andreas Kling.
794
795         * bmalloc.xcodeproj/project.pbxproj: Marked a few headers as private.
796         These headers are used, so they must be available outside the project.
797
798 2014-08-13  Daniel Bates  <dabates@apple.com>
799
800         Attempt to fix the build following <http://trac.webkit.org/changeset/172576>
801         (https://bugs.webkit.org/show_bug.cgi?id=135895)
802
803         Substitute PerThreadStorage<T>::initSharedKeyIfNeeded() for initSharedKeyIfNeeded() in
804         implementation of PerThread<T>::getFastCase().
805
806         * bmalloc/PerThread.h:
807         (bmalloc::PerThread<T>::getFastCase):
808
809 2014-08-13  Daniel Bates  <dabates@apple.com>
810
811         Make bmalloc::PerThread work without C++ thread local storage
812         https://bugs.webkit.org/show_bug.cgi?id=135895
813
814         Reviewed by Geoffrey Garen.
815
816         Implement support for building bmalloc without C++ thread local storage.
817
818         * bmalloc/BPlatform.h: Remove macro define BPLATFORM_IOS_SIMULATOR. Added macro function
819         BCOMPILER_SUPPORTS() and macro define BCOMPILER_SUPPORTS_CXX_THREAD_LOCAL that can be used
820         to determine whether the compiler supports C++ thread local storage.
821         * bmalloc/PerThread.h:
822         (bmalloc::PerThreadStorage::get): Modified to call pthread_getspecific() when building
823         without C++ thread local storage.
824         (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Added.
825         (bmalloc::PerThreadStorage::init): Moved logic to initialize shared Pthread key from here to
826         PerThreadStorage::initSharedKeyIfNeeded().
827         (bmalloc::PerThread<T>::getFastCase): Modified to call PerThreadStorage::initSharedKeyIfNeeded()
828         before querying PerThreadStorage::get() when building without C++ thread local storage so as to
829         ensure that the shared key has been initialized.
830         (_pthread_setspecific_direct): Deleted.
831         (_pthread_getspecific_direct): Deleted.
832
833 2014-08-13  Daniel Bates  <dabates@apple.com>
834
835         [iOS] Make JavaScriptCore and bmalloc build with the public SDK
836         https://bugs.webkit.org/show_bug.cgi?id=135848
837
838         Reviewed by Geoffrey Garen.
839
840         * bmalloc/BPlatform.h: Added macro BPLATFORM_IOS_SIMULATOR, which evaluates to true
841         when building for the iOS Simulator.
842         * bmalloc/PerThread.h: Use pthread_machdep.h code path when building for iOS Simulator
843         using the public SDK.
844         (_pthread_setspecific_direct): Added; only defined when building for the iOS Simulator
845         using the public SDK.
846         (_pthread_getspecific_direct): Added; only defined when building for the iOS Simulator
847         using the public SDK.
848
849 2014-08-12  Daniel Bates  <dabates@apple.com>
850
851         BPLATFORM(IOS) always evaluates to false
852         https://bugs.webkit.org/show_bug.cgi?id=135843
853
854         Reviewed by Geoffrey Garen.
855
856         Fix typo in definition of BPLATFORM() and include system header TargetConditionals.h
857         (when building on an Apple platform) so that BPLATFORM(X) evaluates to true when
858         building for platform X. In particular, so that BPLATFORM(IOS) evaluates to true when
859         building for iOS.
860
861         As a side effect of this change, the change made in <http://trac.webkit.org/changeset/167289>
862         will be honored and iOS will assume a VM page size of 16kB (again) instead of 4kB.
863
864         * bmalloc/BPlatform.h:
865
866 2014-08-11  Andy Estes  <aestes@apple.com>
867
868         [iOS] Get rid of iOS.xcconfig
869         https://bugs.webkit.org/show_bug.cgi?id=135809
870
871         Reviewed by Joseph Pecoraro.
872
873         All iOS.xcconfig did was include AspenFamily.xcconfig, so there's no need for the indirection.
874
875         * Configurations/Base.xcconfig:
876         * Configurations/iOS.xcconfig: Removed.
877         * bmalloc.xcodeproj/project.pbxproj:
878
879 2014-05-01  Dan Bernstein  <mitz@apple.com>
880
881         Fixed production builds for the iOS Simulator.
882         <rdar://problem/16792221>
883
884         * Configurations/bmalloc.xcconfig: Include INSTALL_PATH_PREFIX in
885         PRIVATE_HEADERS_FOLDER_PATH when installing.
886
887 2014-04-20  Geoffrey Garen  <ggaren@apple.com>
888
889         bmalloc: Segregate pages by objects size
890         https://bugs.webkit.org/show_bug.cgi?id=131909
891
892         Reviewed by Andreas Kling.
893
894         2% reduction in memory-at-end on the Membuster memory_warning benchmarks.
895
896         * bmalloc/Allocator.cpp:
897         (bmalloc::Allocator::allocateSlowCase):
898         * bmalloc/Allocator.h:
899         (bmalloc::Allocator::allocateFastCase):
900         (bmalloc::Allocator::smallAllocatorFor): Use the new shared helper
901         function for size class calculation.
902
903         * bmalloc/Deallocator.cpp:
904         (bmalloc::Deallocator::Deallocator):
905         (bmalloc::Deallocator::scavenge):
906         (bmalloc::Deallocator::deallocateSmallLine):
907         (bmalloc::Deallocator::allocateSmallLine):
908         * bmalloc/Deallocator.h: Keep a cache for every size class, since the
909         cache can't be shared anymore.
910
911         * bmalloc/Heap.cpp:
912         (bmalloc::Heap::allocateSmallLineSlowCase):
913         * bmalloc/Heap.h:
914         (bmalloc::Heap::deallocateSmallLine): Ditto.
915
916         (bmalloc::Heap::allocateSmallLine): Check size class in addition to
917         page refcount when allocating a line because we might have deallocated
918         the page and the recycled it for another size class.
919
920         (bmalloc::Heap::deallocateMediumLine):
921         (bmalloc::Heap::allocateMediumLine):
922         * bmalloc/Line.h:
923         (bmalloc::Line::refCount):
924         * bmalloc/Page.h:
925         (bmalloc::Page::refCount):
926         (bmalloc::Page::smallSizeClass):
927         (bmalloc::Page::setSmallSizeClass):
928         (bmalloc::Page<Traits>::refCount): Deleted.
929         * bmalloc/Sizes.h:
930         (bmalloc::Sizes::smallSizeClassFor): New shared API for computing
931         an index into an array from a size.
932
933 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
934
935         bmalloc: Improved alignment in LargeChunk
936         https://bugs.webkit.org/show_bug.cgi?id=131895
937
938         Reviewed by Andreas Kling.
939
940         * bmalloc/Chunk.h:
941         * bmalloc/LargeChunk.h: Align to vmPageSize just like Chunk does.
942         Technically, the previous alignment was harmless, but I would prefer,
943         dear reader, not to have to explain the interlocking set of
944         circumstances that made it so.
945
946 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
947
948         Rolled out r167502 because it caused a crash on the facebook benchmark.
949
950         Unreviewed.
951
952             bmalloc: Added an XSmall line size
953             https://bugs.webkit.org/show_bug.cgi?id=131851
954
955             Reviewed by Sam Weinig.
956
957 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
958
959         bmalloc: Mutex should be harder to use wrong
960         https://bugs.webkit.org/show_bug.cgi?id=131879
961
962         Reviewed by Andreas Kling.
963
964         Mutex now has a proper constructor, so you can't deadlock by forgetting
965         to initialize it.
966
967         * bmalloc.xcodeproj/project.pbxproj:
968         * bmalloc/Allocator.cpp:
969         (bmalloc::Allocator::processXSmallAllocatorLog):
970         (bmalloc::Allocator::processSmallAllocatorLog):
971         (bmalloc::Allocator::processMediumAllocatorLog):
972         (bmalloc::Allocator::allocateLarge):
973         (bmalloc::Allocator::allocateXLarge): Global replace Mutex => StaticMutex,
974         since the Heap mutex is a static.
975
976         * bmalloc/AsyncTask.h:
977         (bmalloc::Function>::AsyncTask): Use Mutex, since we're not static. No
978         need for explicit initialization anymore.
979
980         * bmalloc/Deallocator.cpp:
981         (bmalloc::Deallocator::scavenge):
982         (bmalloc::Deallocator::deallocateLarge):
983         (bmalloc::Deallocator::deallocateXLarge):
984         (bmalloc::Deallocator::processObjectLog):
985         (bmalloc::Deallocator::deallocateSmallLine):
986         (bmalloc::Deallocator::deallocateXSmallLine):
987         (bmalloc::Deallocator::allocateSmallLine):
988         (bmalloc::Deallocator::allocateXSmallLine):
989         (bmalloc::Deallocator::deallocateMediumLine):
990         (bmalloc::Deallocator::allocateMediumLine):
991         * bmalloc/Deallocator.h:
992         * bmalloc/Heap.cpp:
993         (bmalloc::sleep):
994         (bmalloc::Heap::Heap):
995         (bmalloc::Heap::concurrentScavenge):
996         (bmalloc::Heap::scavenge):
997         (bmalloc::Heap::scavengeSmallPages):
998         (bmalloc::Heap::scavengeXSmallPages):
999         (bmalloc::Heap::scavengeMediumPages):
1000         (bmalloc::Heap::scavengeLargeRanges):
1001         (bmalloc::Heap::allocateXSmallLineSlowCase):
1002         (bmalloc::Heap::allocateSmallLineSlowCase):
1003         (bmalloc::Heap::allocateMediumLineSlowCase):
1004         (bmalloc::Heap::allocateXLarge):
1005         (bmalloc::Heap::deallocateXLarge):
1006         (bmalloc::Heap::allocateLarge):
1007         (bmalloc::Heap::deallocateLarge):
1008         * bmalloc/Heap.h:
1009         (bmalloc::Heap::deallocateXSmallLine):
1010         (bmalloc::Heap::allocateXSmallLine):
1011         (bmalloc::Heap::deallocateSmallLine):
1012         (bmalloc::Heap::allocateSmallLine):
1013         (bmalloc::Heap::deallocateMediumLine):
1014         (bmalloc::Heap::allocateMediumLine):
1015         * bmalloc/Line.h:
1016         (bmalloc::Line<Traits>::deref):
1017         * bmalloc/Mutex.cpp: Removed.
1018         * bmalloc/Mutex.h:
1019         (bmalloc::Mutex::Mutex):
1020         (bmalloc::Mutex::init): Deleted.
1021         (bmalloc::Mutex::try_lock): Deleted.
1022         (bmalloc::Mutex::lock): Deleted.
1023         (bmalloc::Mutex::unlock): Deleted.
1024         * bmalloc/Page.h:
1025         (bmalloc::Page<Traits>::ref):
1026         (bmalloc::Page<Traits>::deref):
1027         (bmalloc::Page<Traits>::refCount):
1028         * bmalloc/PerProcess.h:
1029         (bmalloc::PerProcess::mutex):
1030         (bmalloc::PerProcess<T>::getSlowCase):
1031         * bmalloc/StaticMutex.cpp: Added.
1032         (bmalloc::StaticMutex::lockSlowCase):
1033         * bmalloc/StaticMutex.h: Added.
1034         (bmalloc::StaticMutex::init):
1035         (bmalloc::StaticMutex::try_lock):
1036         (bmalloc::StaticMutex::lock):
1037         (bmalloc::StaticMutex::unlock):
1038         * bmalloc/VMHeap.h:
1039         (bmalloc::VMHeap::deallocateXSmallPage):
1040         (bmalloc::VMHeap::deallocateSmallPage):
1041         (bmalloc::VMHeap::deallocateMediumPage):
1042         (bmalloc::VMHeap::deallocateLargeRange):
1043         * bmalloc/bmalloc.h:
1044         (bmalloc::api::scavenge): Global replace Mutex => StaticMutex,
1045         since the Heap mutex is a static.
1046
1047 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
1048
1049         bmalloc: AsyncTask should use Mutex instead of std::mutex
1050         https://bugs.webkit.org/show_bug.cgi?id=131865
1051
1052         Reviewed by Gavin Barraclough.
1053
1054         std::mutex is so slow that it makes parallelizing simple tasks through
1055         AsyncTask a net regression. Mutex fixes this.
1056
1057         * bmalloc/AsyncTask.h:
1058         (bmalloc::Function>::AsyncTask):
1059         (bmalloc::Function>::join):
1060         (bmalloc::Function>::runSlowCase):
1061         (bmalloc::Function>::entryPoint):
1062         * bmalloc/Mutex.h:
1063         (bmalloc::Mutex::init):
1064
1065 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
1066
1067         bmalloc: Added an XSmall line size
1068         https://bugs.webkit.org/show_bug.cgi?id=131851
1069
1070         Reviewed by Sam Weinig.
1071
1072         Reduces malloc footprint on Membuster recordings by 10%.
1073
1074         This is a throughput regression, but we're still way ahead of TCMalloc.
1075         I have some ideas for how to recover the regression -- but I wanted to
1076         get this win in first.
1077
1078         Full set of benchmark results:
1079
1080                 bmalloc> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks --measure-heap nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1081                                                                                                 
1082                                                                nopatch                      patch                                Δ
1083                 Peak Memory:
1084                     reddit_memory_warning                      7,896kB                    7,532kB                  ^ 1.05x smaller
1085                     flickr_memory_warning                     12,968kB                   12,324kB                  ^ 1.05x smaller
1086                     theverge_memory_warning                   16,672kB                   15,200kB                   ^ 1.1x smaller
1087
1088                     <geometric mean>                          11,952kB                   11,216kB                  ^ 1.07x smaller
1089                     <arithmetic mean>                         12,512kB                   11,685kB                  ^ 1.07x smaller
1090                     <harmonic mean>                           11,375kB                   10,726kB                  ^ 1.06x smaller
1091
1092                 Memory at End:
1093                     reddit_memory_warning                      7,320kB                    6,856kB                  ^ 1.07x smaller
1094                     flickr_memory_warning                     10,848kB                    9,692kB                  ^ 1.12x smaller
1095                     theverge_memory_warning                   16,380kB                   14,872kB                   ^ 1.1x smaller
1096
1097                     <geometric mean>                          10,916kB                    9,961kB                   ^ 1.1x smaller
1098                     <arithmetic mean>                         11,516kB                   10,473kB                   ^ 1.1x smaller
1099                     <harmonic mean>                           10,350kB                    9,485kB                  ^ 1.09x smaller
1100
1101                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1102                                                                                                 
1103                                                    nopatch                patch                         Δ
1104                 Execution Time:
1105                     churn                            127ms                151ms            ! 1.19x slower
1106                     list_allocate                    130ms                164ms            ! 1.26x slower
1107                     tree_allocate                    109ms                127ms            ! 1.17x slower
1108                     tree_churn                       115ms                120ms            ! 1.04x slower
1109                     facebook                         240ms                259ms            ! 1.08x slower
1110                     fragment                          91ms                131ms            ! 1.44x slower
1111                     fragment_iterate                 105ms                106ms            ! 1.01x slower
1112                     message_one                      260ms                259ms             ^ 1.0x faster
1113                     message_many                     149ms                154ms            ! 1.03x slower
1114                     medium                           194ms                248ms            ! 1.28x slower
1115                     big                              157ms                160ms            ! 1.02x slower
1116
1117                     <geometric mean>                 144ms                163ms            ! 1.13x slower
1118                     <arithmetic mean>                152ms                171ms            ! 1.12x slower
1119                     <harmonic mean>                  137ms                156ms            ! 1.14x slower
1120
1121                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1122                                                                                                 
1123                                                                        nopatch                          patch                                     Δ
1124                 Execution Time:
1125                     churn                                                126ms                          148ms                        ! 1.17x slower
1126                     churn --parallel                                      62ms                           76ms                        ! 1.23x slower
1127                     list_allocate                                        130ms                          164ms                        ! 1.26x slower
1128                     list_allocate --parallel                             120ms                          175ms                        ! 1.46x slower
1129                     tree_allocate                                        111ms                          127ms                        ! 1.14x slower
1130                     tree_allocate --parallel                              95ms                          135ms                        ! 1.42x slower
1131                     tree_churn                                           115ms                          124ms                        ! 1.08x slower
1132                     tree_churn --parallel                                107ms                          126ms                        ! 1.18x slower
1133                     facebook                                             240ms                          276ms                        ! 1.15x slower
1134                     facebook --parallel                                  802ms                        1,088ms                        ! 1.36x slower
1135                     fragment                                              92ms                          130ms                        ! 1.41x slower
1136                     fragment --parallel                                   66ms                          124ms                        ! 1.88x slower
1137                     fragment_iterate                                     109ms                          127ms                        ! 1.17x slower
1138                     fragment_iterate --parallel                           55ms                           64ms                        ! 1.16x slower
1139                     message_one                                          260ms                          260ms                                      
1140                     message_many                                         170ms                          238ms                         ! 1.4x slower
1141                     medium                                               185ms                          250ms                        ! 1.35x slower
1142                     medium --parallel                                    210ms                          334ms                        ! 1.59x slower
1143                     big                                                  150ms                          169ms                        ! 1.13x slower
1144                     big --parallel                                       138ms                          144ms                        ! 1.04x slower
1145
1146                     <geometric mean>                                     135ms                          170ms                        ! 1.26x slower
1147                     <arithmetic mean>                                    167ms                          214ms                        ! 1.28x slower
1148                     <harmonic mean>                                      117ms                          148ms                        ! 1.26x slower
1149
1150                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
1151
1152                                                                     TC                      patch                                Δ
1153                 Peak Memory:
1154                     reddit_memory_warning                     13,836kB                   13,436kB                  ^ 1.03x smaller
1155                     flickr_memory_warning                     24,868kB                   25,188kB                   ! 1.01x bigger
1156                     theverge_memory_warning                   24,504kB                   26,636kB                   ! 1.09x bigger
1157
1158                     <geometric mean>                          20,353kB                   20,812kB                   ! 1.02x bigger
1159                     <arithmetic mean>                         21,069kB                   21,753kB                   ! 1.03x bigger
1160                     <harmonic mean>                           19,570kB                   19,780kB                   ! 1.01x bigger
1161
1162                 Memory at End:
1163                     reddit_memory_warning                      8,656kB                   10,016kB                   ! 1.16x bigger
1164                     flickr_memory_warning                     11,844kB                   13,784kB                   ! 1.16x bigger
1165                     theverge_memory_warning                   18,516kB                   22,748kB                   ! 1.23x bigger
1166
1167                     <geometric mean>                          12,382kB                   14,644kB                   ! 1.18x bigger
1168                     <arithmetic mean>                         13,005kB                   15,516kB                   ! 1.19x bigger
1169                     <harmonic mean>                           11,813kB                   13,867kB                   ! 1.17x bigger
1170
1171                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
1172                                                                                                 
1173                                                         TC                patch                         Δ
1174                 Execution Time:
1175                     churn                            416ms                148ms            ^ 2.81x faster
1176                     list_allocate                    463ms                164ms            ^ 2.82x faster
1177                     tree_allocate                    292ms                127ms             ^ 2.3x faster
1178                     tree_churn                       157ms                120ms            ^ 1.31x faster
1179                     facebook                         327ms                276ms            ^ 1.18x faster
1180                     fragment                         335ms                129ms             ^ 2.6x faster
1181                     fragment_iterate                 344ms                108ms            ^ 3.19x faster
1182                     message_one                      386ms                258ms             ^ 1.5x faster
1183                     message_many                     410ms                154ms            ^ 2.66x faster
1184                     medium                           391ms                245ms             ^ 1.6x faster
1185                     big                              261ms                167ms            ^ 1.56x faster
1186
1187                     <geometric mean>                 332ms                164ms            ^ 2.02x faster
1188                     <arithmetic mean>                344ms                172ms            ^ 1.99x faster
1189                     <harmonic mean>                  317ms                157ms            ^ 2.02x faster
1190
1191         * bmalloc.xcodeproj/project.pbxproj:
1192         * bmalloc/Allocator.cpp:
1193         (bmalloc::Allocator::Allocator): Don't assume that each allocator's
1194         index corresponds with its size. Instead, use the size selection function
1195         explicitly. Now that we have XSmall, some small allocator entries are
1196         unused.
1197
1198         (bmalloc::Allocator::scavenge):
1199         (bmalloc::Allocator::log):
1200         (bmalloc::Allocator::processXSmallAllocatorLog):
1201         (bmalloc::Allocator::allocateSlowCase):
1202         * bmalloc/Allocator.h:
1203         (bmalloc::Allocator::xSmallAllocatorFor):
1204         (bmalloc::Allocator::allocateFastCase):
1205         * bmalloc/Chunk.h:
1206         * bmalloc/Deallocator.cpp:
1207         (bmalloc::Deallocator::scavenge):
1208         (bmalloc::Deallocator::processObjectLog):
1209         (bmalloc::Deallocator::deallocateSlowCase):
1210         (bmalloc::Deallocator::deallocateXSmallLine):
1211         (bmalloc::Deallocator::allocateXSmallLine):
1212         * bmalloc/Deallocator.h:
1213         (bmalloc::Deallocator::deallocateFastCase):
1214         * bmalloc/Heap.cpp:
1215         (bmalloc::Heap::scavenge):
1216         (bmalloc::Heap::scavengeXSmallPages):
1217         (bmalloc::Heap::allocateXSmallLineSlowCase):
1218         * bmalloc/Heap.h:
1219         (bmalloc::Heap::deallocateXSmallLine):
1220         (bmalloc::Heap::allocateXSmallLine):
1221         * bmalloc/LargeChunk.h:
1222         (bmalloc::LargeChunk::get):
1223         (bmalloc::LargeChunk::endTag):
1224         * bmalloc/Line.h:
1225         * bmalloc/MediumAllocator.h:
1226         (bmalloc::MediumAllocator::allocate):
1227         (bmalloc::MediumAllocator::refill):
1228         * bmalloc/ObjectType.cpp:
1229         (bmalloc::objectType):
1230         * bmalloc/ObjectType.h:
1231         (bmalloc::isXSmall):
1232         (bmalloc::isSmall):
1233         (bmalloc::isMedium):
1234         (bmalloc::isLarge):
1235         (bmalloc::isSmallOrMedium): Deleted.
1236         * bmalloc/SegregatedFreeList.h: I boiler-plate copied existing code for
1237         handling small objects. There's probably a reasonable way to share this
1238         code in the future -- I'll look into that once it's stopped changing.
1239
1240         * bmalloc/Sizes.h: Tweaked size classes to make Membuster happy. This
1241         is the main reason things got slower.
1242
1243         * bmalloc/SmallAllocator.h:
1244         (bmalloc::SmallAllocator::allocate):
1245         * bmalloc/SmallTraits.h:
1246         * bmalloc/VMHeap.cpp:
1247         (bmalloc::VMHeap::allocateXSmallChunk):
1248         * bmalloc/VMHeap.h:
1249         (bmalloc::VMHeap::allocateXSmallPage):
1250         (bmalloc::VMHeap::deallocateXSmallPage):
1251         * bmalloc/XSmallAllocator.h: Added.
1252         (bmalloc::XSmallAllocator::isNull):
1253         (bmalloc::XSmallAllocator::canAllocate):
1254         (bmalloc::XSmallAllocator::XSmallAllocator):
1255         (bmalloc::XSmallAllocator::line):
1256         (bmalloc::XSmallAllocator::allocate):
1257         (bmalloc::XSmallAllocator::objectCount):
1258         (bmalloc::XSmallAllocator::derefCount):
1259         (bmalloc::XSmallAllocator::refill):
1260         * bmalloc/XSmallChunk.h: Added.
1261         * bmalloc/XSmallLine.h: Added.
1262         * bmalloc/XSmallPage.h: Added.
1263         * bmalloc/XSmallTraits.h: Added.
1264         * bmalloc/bmalloc.h:
1265         (bmalloc::api::realloc): Boiler-plate copy, as above.
1266
1267 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
1268
1269         MallocBench should scavenge explicitly instead of waiting
1270         https://bugs.webkit.org/show_bug.cgi?id=131661
1271
1272         Reviewed by Andreas Kling.
1273
1274         Added explicit scavenge support to bmalloc. This isn't a memory win,
1275         since bmalloc's per-thread cache is so small. But it makes testing
1276         simpler.
1277
1278         * bmalloc/Allocator.cpp:
1279         (bmalloc::Allocator::~Allocator):
1280         (bmalloc::Allocator::scavenge):
1281         * bmalloc/Allocator.h:
1282         * bmalloc/Cache.cpp:
1283         (bmalloc::Cache::operator new):
1284         (bmalloc::Cache::operator delete):
1285         (bmalloc::Cache::Cache):
1286         (bmalloc::Cache::scavenge):
1287         * bmalloc/Cache.h:
1288         * bmalloc/Deallocator.cpp:
1289         (bmalloc::Deallocator::~Deallocator):
1290         (bmalloc::Deallocator::scavenge):
1291         * bmalloc/Deallocator.h: Factored existing scavenging code into helper
1292         functions, for reuse.
1293
1294         * bmalloc/Heap.cpp:
1295         (bmalloc::sleep):
1296         (bmalloc::Heap::concurrentScavenge):
1297         (bmalloc::Heap::scavenge):
1298         (bmalloc::Heap::scavengeSmallPages):
1299         (bmalloc::Heap::scavengeMediumPages):
1300         (bmalloc::Heap::scavengeLargeRanges):
1301         * bmalloc/Heap.h: Made scavenge sleep duration a parameter. Forced
1302         scavenging -- in response to a benchmark or a low memory warning --
1303         wants to complete as soon as possible, so its sleep duration is 0.
1304
1305         * bmalloc/bmalloc.h:
1306         (bmalloc::api::scavenge):
1307         * bmalloc/mbmalloc.cpp: Exported the scavenge API for MallocBench's use.
1308
1309 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
1310
1311         Use 4kB pages on Mac
1312         https://bugs.webkit.org/show_bug.cgi?id=131658
1313
1314         Reviewed by Sam Weinig.
1315
1316         This reduces memory use a lot on Membuster:
1317
1318                                                                   base                      patch                                Δ
1319                 Execution Time:
1320                     reddit_memory_warning                         18ms                       17ms                   ^ 1.06x faster
1321                     flickr_memory_warning                         34ms                       36ms                   ! 1.06x slower
1322                     theverge_memory_warning                       39ms                       41ms                   ! 1.05x slower
1323
1324                     <geometric mean>                              29ms                       29ms                   ! 1.02x slower
1325                     <arithmetic mean>                             30ms                       31ms                   ! 1.03x slower
1326                     <harmonic mean>                               27ms                       27ms                    ^ 1.0x faster
1327
1328                 Peak Memory:
1329                     reddit_memory_warning                     16,412kB                   16,436kB                    ! 1.0x bigger
1330                     flickr_memory_warning                     30,120kB                   30,184kB                    ! 1.0x bigger
1331                     theverge_memory_warning                   33,408kB                   33,420kB                    ! 1.0x bigger
1332
1333                     <geometric mean>                          25,466kB                   25,499kB                    ! 1.0x bigger
1334                     <arithmetic mean>                         26,647kB                   26,680kB                    ! 1.0x bigger
1335                     <harmonic mean>                           24,181kB                   24,214kB                    ! 1.0x bigger
1336
1337                 Memory at End:
1338                     reddit_memory_warning                      2,404kB                    1,920kB                  ^ 1.25x smaller
1339                     flickr_memory_warning                      3,764kB                    3,072kB                  ^ 1.23x smaller
1340                     theverge_memory_warning                    3,648kB                    3,132kB                  ^ 1.16x smaller
1341
1342                     <geometric mean>                           3,208kB                    2,644kB                  ^ 1.21x smaller
1343                     <arithmetic mean>                          3,272kB                    2,708kB                  ^ 1.21x smaller
1344                     <harmonic mean>                            3,139kB                    2,574kB                  ^ 1.22x smaller
1345
1346
1347         * bmalloc.xcodeproj/project.pbxproj:
1348         * bmalloc/BPlatform.h: Added.
1349         * bmalloc/VMAllocate.h: Only use 16kB pages on iOS because the page size
1350         is 4kB on Mac.
1351
1352 2014-04-14  Alexey Proskuryakov  <ap@apple.com>
1353
1354         Fixed svn:ignore on bmalloc.xcodeproj, it had erroneous leading spaces.
1355
1356         * bmalloc.xcodeproj: Modified property svn:ignore.
1357
1358 2014-04-13  Geoffrey Garen  <ggaren@apple.com>
1359
1360         Fixed some mbmalloc exports
1361         https://bugs.webkit.org/show_bug.cgi?id=131599
1362
1363         Reviewed by Ryosuke Niwa.
1364
1365         * bmalloc.xcodeproj/project.pbxproj: Made some headers a private part
1366         of the project, so we can call them from API.
1367
1368         * bmalloc/mbmalloc.cpp: Marked the mbmalloc functions with default
1369         visibility, so they show up as exported in the .dylib.
1370
1371 2014-04-09  Geoffrey Garen  <ggaren@apple.com>
1372
1373         Put bmalloc headers in the right place
1374         https://bugs.webkit.org/show_bug.cgi?id=131464
1375
1376         Reviewed by Mark Rowe.
1377
1378         * Configurations/bmalloc.xcconfig: Set PRIVATE_HEADERS_FOLDER_PATH to
1379         specify that we don't just want to dump all of our generically-named
1380         headers into /usr/local/include.
1381
1382 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
1383
1384         Made bmalloc more #include friendly
1385         https://bugs.webkit.org/show_bug.cgi?id=131386
1386
1387         Reviewed by Andreas Kling.
1388
1389         Marked a bunch of headers private so they can be used from client code
1390         that #includes bmalloc.h.
1391
1392         Renamed ASSERT macros to BASSERT. This matches their header, which already
1393         had to be renamed, and fixes conflicts with WTF's ASSERT macros.
1394
1395         * bmalloc.xcodeproj/project.pbxproj:
1396         * bmalloc/Allocator.cpp:
1397         (bmalloc::Allocator::allocateSlowCase):
1398         * bmalloc/AsyncTask.h:
1399         (bmalloc::Function>::runSlowCase):
1400         * bmalloc/BAssert.h:
1401         * bmalloc/BoundaryTag.h:
1402         (bmalloc::BoundaryTag::setSize):
1403         * bmalloc/BoundaryTagInlines.h:
1404         (bmalloc::validate):
1405         (bmalloc::BoundaryTag::init):
1406         (bmalloc::BoundaryTag::deallocate):
1407         (bmalloc::BoundaryTag::splitLarge):
1408         (bmalloc::BoundaryTag::allocate):
1409         * bmalloc/Chunk.h:
1410         * bmalloc/Deallocator.cpp:
1411         (bmalloc::Deallocator::processObjectLog):
1412         (bmalloc::Deallocator::deallocateSlowCase):
1413         * bmalloc/Deallocator.h:
1414         (bmalloc::Deallocator::deallocateFastCase):
1415         * bmalloc/FixedVector.h:
1416         (bmalloc::Capacity>::operator):
1417         (bmalloc::Capacity>::push):
1418         (bmalloc::Capacity>::pop):
1419         (bmalloc::Capacity>::shrink):
1420         * bmalloc/Heap.cpp:
1421         (bmalloc::Heap::allocateLarge):
1422         * bmalloc/LargeChunk.h:
1423         (bmalloc::LargeChunk::get):
1424         (bmalloc::LargeChunk::endTag):
1425         * bmalloc/Line.h:
1426         (bmalloc::Line<Traits>::concurrentRef):
1427         (bmalloc::Line<Traits>::deref):
1428         * bmalloc/MediumAllocator.h:
1429         (bmalloc::MediumAllocator::allocate):
1430         * bmalloc/ObjectType.h:
1431         (bmalloc::isSmall):
1432         * bmalloc/Page.h:
1433         (bmalloc::Page<Traits>::ref):
1434         (bmalloc::Page<Traits>::deref):
1435         * bmalloc/PerThread.h:
1436         (bmalloc::PerThread<T>::getSlowCase):
1437         * bmalloc/SegregatedFreeList.cpp:
1438         (bmalloc::SegregatedFreeList::SegregatedFreeList):
1439         (bmalloc::SegregatedFreeList::insert):
1440         * bmalloc/SmallAllocator.h:
1441         (bmalloc::SmallAllocator::allocate):
1442         (bmalloc::SmallAllocator::refill):
1443         * bmalloc/Syscall.h:
1444         * bmalloc/VMAllocate.h:
1445         (bmalloc::vmValidate):
1446         (bmalloc::vmAllocate):
1447         (bmalloc::vmDeallocatePhysicalPagesSloppy):
1448         * bmalloc/Vector.h:
1449         (bmalloc::Vector<T>::operator):
1450         (bmalloc::Vector<T>::pop):
1451         (bmalloc::Vector<T>::shrink):
1452         * bmalloc/XLargeChunk.h:
1453         (bmalloc::XLargeChunk::range):
1454         (bmalloc::XLargeChunk::size):
1455
1456 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
1457
1458         Removed an unused file.
1459
1460         Unreviewed.
1461
1462         * bmalloc/AsyncTask.cpp: Removed.
1463
1464 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1465
1466         Build bmalloc on Mac
1467         https://bugs.webkit.org/show_bug.cgi?id=131333
1468
1469         Reviewed by Mark Rowe.
1470
1471         * Makefile: Added. For make clients.
1472
1473         These files are required for building any project in WebKit. I copied
1474         them from WTF:
1475         * Configurations: Added.
1476         * Configurations/Base.xcconfig: Added.
1477         * Configurations/DebugRelease.xcconfig: Added.
1478         * Configurations/bmalloc.xcconfig: Added.
1479         * Configurations/iOS.xcconfig: Added.
1480         * Configurations/mbmalloc.xcconfig: Added.
1481
1482         * bmalloc.xcodeproj/project.pbxproj: I removed per-project-file stuff
1483         from here because everything is in .xcconfig files now.
1484
1485         I had to fix a bunch of minor warnings, since they're enabled in our
1486         .xcconfig files:
1487
1488         * bmalloc/AsyncTask.h:
1489         (bmalloc::Function>::AsyncTask):
1490         * bmalloc/BAssert.h:
1491         * bmalloc/BoundaryTagInlines.h:
1492         (bmalloc::validate):
1493         * bmalloc/Heap.cpp:
1494         (bmalloc::Heap::Heap):
1495         (bmalloc::Heap::allocateLarge):
1496         (bmalloc::Heap::deallocateLarge):
1497         * bmalloc/Mutex.h:
1498         (bmalloc::Mutex::Mutex): Deleted.
1499         * bmalloc/VMAllocate.h:
1500         (bmalloc::vmValidate):
1501         * bmalloc/mbmalloc.cpp:
1502
1503 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1504
1505         bmalloc: Fixed a leak in the per-thread cache
1506         https://bugs.webkit.org/show_bug.cgi?id=131330
1507
1508         Reviewed by Andreas Kling.
1509
1510         Remember to deallocate our line caches upon thread exit.
1511
1512         * bmalloc/Deallocator.cpp:
1513         (bmalloc::Deallocator::~Deallocator):
1514
1515 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1516
1517         bmalloc: rolled out the tryLock experiment
1518         https://bugs.webkit.org/show_bug.cgi?id=131328
1519
1520         Reviewed by Andreas Kling.
1521
1522         It wasn't a speedup.
1523
1524         * bmalloc.xcodeproj/project.pbxproj:
1525         * bmalloc/Allocator.cpp:
1526         (bmalloc::Allocator::processSmallAllocatorLog):
1527         (bmalloc::Allocator::processMediumAllocatorLog):
1528         * bmalloc/Deallocator.cpp:
1529         (bmalloc::Deallocator::processObjectLog):
1530         (bmalloc::Deallocator::deallocateSlowCase):
1531         (bmalloc::Deallocator::deallocateSmallLine):
1532         (bmalloc::Deallocator::deallocateMediumLine):
1533         * bmalloc/Deallocator.h:
1534         (bmalloc::Deallocator::deallocateFastCase):
1535         * bmalloc/Heap.h:
1536         (bmalloc::Heap::deallocateSmallLine):
1537         (bmalloc::Heap::deallocateMediumLine):
1538         * bmalloc/Line.h:
1539         (bmalloc::Line<Traits>::deref):
1540         * bmalloc/Page.h:
1541         (bmalloc::Page<Traits>::deref):
1542
1543 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1544
1545         bmalloc
1546         https://bugs.webkit.org/show_bug.cgi?id=131170
1547
1548         Reviewed by Andreas Kling.
1549
1550         Initial commit.
1551
1552         * bmalloc: Added.
1553         * bmalloc.xcodeproj: Added.
1554         * bmalloc.xcodeproj/project.pbxproj: Added.
1555         * bmalloc/Algorithm.h: Added.
1556         (bmalloc::max):
1557         (bmalloc::min):
1558         (bmalloc::mask):
1559         (bmalloc::test):
1560         (bmalloc::roundUpToMultipleOf):
1561         (bmalloc::roundDownToMultipleOf):
1562         (bmalloc::sizeOf):
1563         (bmalloc::bitCount):
1564         (bmalloc::isPowerOfTwo):
1565         * bmalloc/Allocator.cpp: Added.
1566         (bmalloc::Allocator::Allocator):
1567         (bmalloc::Allocator::~Allocator):
1568         (bmalloc::Allocator::log):
1569         (bmalloc::Allocator::processSmallAllocatorLog):
1570         (bmalloc::Allocator::processMediumAllocatorLog):
1571         (bmalloc::Allocator::allocateLarge):
1572         (bmalloc::Allocator::allocateXLarge):
1573         (bmalloc::Allocator::allocateMedium):
1574         (bmalloc::Allocator::allocateSlowCase):
1575         * bmalloc/Allocator.h: Added.
1576         (bmalloc::Allocator::smallAllocatorFor):
1577         (bmalloc::Allocator::allocateFastCase):
1578         (bmalloc::Allocator::allocate):
1579         * bmalloc/AsyncTask.cpp: Added.
1580         (bmalloc::AsyncTask<Function>::runSlowCase):
1581         (bmalloc::AsyncTask<Function>::pthreadEntryPoint):
1582         (bmalloc::AsyncTask<Function>::entryPoint):
1583         * bmalloc/AsyncTask.h: Added.
1584         (bmalloc::Function>::AsyncTask):
1585         (bmalloc::Function>::join):
1586         (bmalloc::Function>::run):
1587         (bmalloc::Function>::runSlowCase):
1588         (bmalloc::Function>::pthreadEntryPoint):
1589         (bmalloc::Function>::entryPoint):
1590         * bmalloc/BAssert.h: Added.
1591         * bmalloc/BeginTag.h: Added.
1592         (bmalloc::BeginTag::isInFreeList):
1593         * bmalloc/BoundaryTag.h: Added.
1594         (bmalloc::BoundaryTag::isXLarge):
1595         (bmalloc::BoundaryTag::setXLarge):
1596         (bmalloc::BoundaryTag::isFree):
1597         (bmalloc::BoundaryTag::setFree):
1598         (bmalloc::BoundaryTag::isEnd):
1599         (bmalloc::BoundaryTag::setEnd):
1600         (bmalloc::BoundaryTag::hasPhysicalPages):
1601         (bmalloc::BoundaryTag::setHasPhysicalPages):
1602         (bmalloc::BoundaryTag::isNull):
1603         (bmalloc::BoundaryTag::clear):
1604         (bmalloc::BoundaryTag::size):
1605         (bmalloc::BoundaryTag::setSize):
1606         (bmalloc::BoundaryTag::prev):
1607         (bmalloc::BoundaryTag::next):
1608         * bmalloc/BoundaryTagInlines.h: Added.
1609         (bmalloc::validate):
1610         (bmalloc::validatePrev):
1611         (bmalloc::validateNext):
1612         (bmalloc::BoundaryTag::init):
1613         (bmalloc::BoundaryTag::mergeLargeLeft):
1614         (bmalloc::BoundaryTag::mergeLargeRight):
1615         (bmalloc::BoundaryTag::mergeLarge):
1616         (bmalloc::BoundaryTag::deallocate):
1617         (bmalloc::BoundaryTag::splitLarge):
1618         (bmalloc::BoundaryTag::allocate):
1619         * bmalloc/Cache.cpp: Added.
1620         (bmalloc::Cache::operator new):
1621         (bmalloc::Cache::operator delete):
1622         (bmalloc::Cache::Cache):
1623         (bmalloc::Cache::allocateSlowCase):
1624         (bmalloc::Cache::allocateSlowCaseNullCache):
1625         (bmalloc::Cache::deallocateSlowCase):
1626         (bmalloc::Cache::deallocateSlowCaseNullCache):
1627         * bmalloc/Cache.h: Added.
1628         (bmalloc::Cache::allocator):
1629         (bmalloc::Cache::deallocator):
1630         (bmalloc::Cache::allocateFastCase):
1631         (bmalloc::Cache::deallocateFastCase):
1632         (bmalloc::Cache::allocate):
1633         (bmalloc::Cache::deallocate):
1634         * bmalloc/Chunk.h: Added.
1635         (bmalloc::Chunk::begin):
1636         (bmalloc::Chunk::end):
1637         (bmalloc::Chunk::lines):
1638         (bmalloc::Chunk::pages):
1639         * bmalloc/Deallocator.cpp: Added.
1640         (bmalloc::Deallocator::Deallocator):
1641         (bmalloc::Deallocator::~Deallocator):
1642         (bmalloc::Deallocator::deallocateLarge):
1643         (bmalloc::Deallocator::deallocateXLarge):
1644         (bmalloc::Deallocator::processObjectLog):
1645         (bmalloc::Deallocator::deallocateSlowCase):
1646         (bmalloc::Deallocator::deallocateSmallLine):
1647         (bmalloc::Deallocator::allocateSmallLine):
1648         (bmalloc::Deallocator::deallocateMediumLine):
1649         (bmalloc::Deallocator::allocateMediumLine):
1650         * bmalloc/Deallocator.h: Added.
1651         (bmalloc::Deallocator::deallocateFastCase):
1652         (bmalloc::Deallocator::deallocate):
1653         * bmalloc/EndTag.h: Added.
1654         (bmalloc::EndTag::operator=):
1655         * bmalloc/FixedVector.h: Added.
1656         (bmalloc::FixedVector::begin):
1657         (bmalloc::FixedVector::end):
1658         (bmalloc::FixedVector::size):
1659         (bmalloc::FixedVector::capacity):
1660         (bmalloc::FixedVector::clear):
1661         (bmalloc::FixedVector::isEmpty):
1662         (bmalloc::Capacity>::FixedVector):
1663         (bmalloc::Capacity>::operator):
1664         (bmalloc::Capacity>::push):
1665         (bmalloc::Capacity>::pop):
1666         (bmalloc::Capacity>::shrink):
1667         * bmalloc/Heap.cpp: Added.
1668         (bmalloc::sleep):
1669         (bmalloc::Heap::Heap):
1670         (bmalloc::Heap::concurrentScavenge):
1671         (bmalloc::Heap::scavengeSmallPages):
1672         (bmalloc::Heap::scavengeMediumPages):
1673         (bmalloc::Heap::scavengeLargeRanges):
1674         (bmalloc::Heap::allocateSmallLineSlowCase):
1675         (bmalloc::Heap::allocateMediumLineSlowCase):
1676         (bmalloc::Heap::allocateXLarge):
1677         (bmalloc::Heap::deallocateXLarge):
1678         (bmalloc::Heap::allocateLarge):
1679         (bmalloc::Heap::deallocateLarge):
1680         * bmalloc/Heap.h: Added.
1681         (bmalloc::Heap::deallocateSmallLine):
1682         (bmalloc::Heap::allocateSmallLine):
1683         (bmalloc::Heap::deallocateMediumLine):
1684         (bmalloc::Heap::allocateMediumLine):
1685         * bmalloc/Inline.h: Added.
1686         * bmalloc/LargeChunk.h: Added.
1687         (bmalloc::LargeChunk::begin):
1688         (bmalloc::LargeChunk::end):
1689         (bmalloc::LargeChunk::create):
1690         (bmalloc::LargeChunk::get):
1691         (bmalloc::LargeChunk::beginTag):
1692         (bmalloc::LargeChunk::endTag):
1693         * bmalloc/Line.h: Added.
1694         (bmalloc::Line<Traits>::begin):
1695         (bmalloc::Line<Traits>::end):
1696         (bmalloc::Line<Traits>::concurrentRef):
1697         (bmalloc::Line<Traits>::deref):
1698         * bmalloc/MediumAllocator.h: Added.
1699         (bmalloc::MediumAllocator::isNull):
1700         (bmalloc::MediumAllocator::MediumAllocator):
1701         (bmalloc::MediumAllocator::line):
1702         (bmalloc::MediumAllocator::allocate):
1703         (bmalloc::MediumAllocator::derefCount):
1704         (bmalloc::MediumAllocator::refill):
1705         * bmalloc/MediumChunk.h: Added.
1706         * bmalloc/MediumLine.h: Added.
1707         * bmalloc/MediumPage.h: Added.
1708         * bmalloc/MediumTraits.h: Added.
1709         * bmalloc/Mutex.cpp: Added.
1710         (bmalloc::Mutex::lockSlowCase):
1711         * bmalloc/Mutex.h: Added.
1712         (bmalloc::Mutex::Mutex):
1713         (bmalloc::Mutex::try_lock):
1714         (bmalloc::Mutex::lock):
1715         (bmalloc::Mutex::unlock):
1716         * bmalloc/ObjectType.cpp: Added.
1717         (bmalloc::objectType):
1718         * bmalloc/ObjectType.h: Added.
1719         (bmalloc::isSmallOrMedium):
1720         (bmalloc::isSmall):
1721         * bmalloc/Page.h: Added.
1722         (bmalloc::Page<Traits>::ref):
1723         (bmalloc::Page<Traits>::deref):
1724         (bmalloc::Page<Traits>::refCount):
1725         * bmalloc/PerProcess.h: Added.
1726         (bmalloc::PerProcess::mutex):
1727         (bmalloc::PerProcess<T>::getFastCase):
1728         (bmalloc::PerProcess<T>::get):
1729         (bmalloc::PerProcess<T>::getSlowCase):
1730         * bmalloc/PerThread.h: Added.
1731         (bmalloc::PerThreadStorage<Cache>::get):
1732         (bmalloc::PerThreadStorage<Cache>::init):
1733         (bmalloc::PerThreadStorage::get):
1734         (bmalloc::PerThreadStorage::init):
1735         (bmalloc::PerThread<T>::getFastCase):
1736         (bmalloc::PerThread<T>::get):
1737         (bmalloc::PerThread<T>::destructor):
1738         (bmalloc::PerThread<T>::getSlowCase):
1739         * bmalloc/Range.h: Added.
1740         (bmalloc::Range::Range):
1741         (bmalloc::Range::begin):
1742         (bmalloc::Range::end):
1743         (bmalloc::Range::size):
1744         (bmalloc::Range::operator!):
1745         (bmalloc::Range::operator<):
1746         * bmalloc/SegregatedFreeList.cpp: Added.
1747         (bmalloc::SegregatedFreeList::SegregatedFreeList):
1748         (bmalloc::SegregatedFreeList::insert):
1749         (bmalloc::SegregatedFreeList::takeGreedy):
1750         (bmalloc::SegregatedFreeList::take):
1751         * bmalloc/SegregatedFreeList.h: Added.
1752         * bmalloc/Sizes.h: Added.
1753         * bmalloc/SmallAllocator.h: Added.
1754         (bmalloc::SmallAllocator::isNull):
1755         (bmalloc::SmallAllocator::canAllocate):
1756         (bmalloc::SmallAllocator::SmallAllocator):
1757         (bmalloc::SmallAllocator::line):
1758         (bmalloc::SmallAllocator::allocate):
1759         (bmalloc::SmallAllocator::objectCount):
1760         (bmalloc::SmallAllocator::derefCount):
1761         (bmalloc::SmallAllocator::refill):
1762         * bmalloc/SmallChunk.h: Added.
1763         * bmalloc/SmallLine.h: Added.
1764         * bmalloc/SmallPage.h: Added.
1765         * bmalloc/SmallTraits.h: Added.
1766         * bmalloc/Syscall.h: Added.
1767         * bmalloc/VMAllocate.h: Added.
1768         (bmalloc::vmSize):
1769         (bmalloc::vmValidate):
1770         (bmalloc::vmAllocate):
1771         (bmalloc::vmDeallocate):
1772         (bmalloc::vmDeallocatePhysicalPages):
1773         (bmalloc::vmAllocatePhysicalPages):
1774         (bmalloc::vmDeallocatePhysicalPagesSloppy):
1775         (bmalloc::vmAllocatePhysicalPagesSloppy):
1776         * bmalloc/VMHeap.cpp: Added.
1777         (bmalloc::VMHeap::VMHeap):
1778         (bmalloc::VMHeap::allocateSmallChunk):
1779         (bmalloc::VMHeap::allocateMediumChunk):
1780         (bmalloc::VMHeap::allocateLargeChunk):
1781         * bmalloc/VMHeap.h: Added.
1782         (bmalloc::VMHeap::allocateSmallPage):
1783         (bmalloc::VMHeap::allocateMediumPage):
1784         (bmalloc::VMHeap::allocateLargeRange):
1785         (bmalloc::VMHeap::deallocateSmallPage):
1786         (bmalloc::VMHeap::deallocateMediumPage):
1787         (bmalloc::VMHeap::deallocateLargeRange):
1788         * bmalloc/Vector.h: Added.
1789         (bmalloc::Vector::begin):
1790         (bmalloc::Vector::end):
1791         (bmalloc::Vector::size):
1792         (bmalloc::Vector::capacity):
1793         (bmalloc::Vector::last):
1794         (bmalloc::Vector::pop):
1795         (bmalloc::Vector<T>::Vector):
1796         (bmalloc::Vector<T>::~Vector):
1797         (bmalloc::Vector<T>::operator):
1798         (bmalloc::Vector<T>::push):
1799         (bmalloc::Vector<T>::pop):
1800         (bmalloc::Vector<T>::shrink):
1801         (bmalloc::Vector<T>::reallocateBuffer):
1802         (bmalloc::Vector<T>::shrinkCapacity):
1803         (bmalloc::Vector<T>::growCapacity):
1804         * bmalloc/XLargeChunk.h: Added.
1805         (bmalloc::XLargeChunk::get):
1806         (bmalloc::XLargeChunk::begin):
1807         (bmalloc::XLargeChunk::XLargeChunk):
1808         (bmalloc::XLargeChunk::create):
1809         (bmalloc::XLargeChunk::destroy):
1810         (bmalloc::XLargeChunk::range):
1811         (bmalloc::XLargeChunk::size):
1812         * bmalloc/bmalloc.h: Added.
1813         (bmalloc::api::malloc):
1814         (bmalloc::api::free):
1815         (bmalloc::api::realloc):
1816         * bmalloc/mbmalloc.cpp: Added.
1817