bmalloc: added some infrastructure for aligned allocation
[WebKit-https.git] / Source / bmalloc / ChangeLog
1 2015-01-16  Geoffrey Garen  <ggaren@apple.com>
2
3         bmalloc: added some infrastructure for aligned allocation
4         https://bugs.webkit.org/show_bug.cgi?id=140572
5
6         Reviewed by Andreas Kling.
7
8         * bmalloc/Algorithm.h:
9         (bmalloc::isPowerOfTwo):
10         (bmalloc::roundUpToMultipleOf):
11         (bmalloc::roundDownToMultipleOf): Refactored some duplicate code to use our
12         isPowerOfTwo helper function.
13
14         * bmalloc/Allocator.cpp:
15         (bmalloc::Allocator::allocate):
16         * bmalloc/Allocator.h: Stubbed out an implementation of aligned allocation.
17         Doesn't do anything yet, but does correctly forward to system malloc
18         when bmalloc is disabled.
19
20         * bmalloc/Cache.cpp:
21         (bmalloc::Cache::allocateSlowCaseNullCache):
22         * bmalloc/Cache.h:
23         (bmalloc::Cache::allocate):
24         * bmalloc/bmalloc.h:
25         (bmalloc::api::memalign):
26         * bmalloc/mbmalloc.cpp: Stubbed out an API for aligned allocation.
27
28 2015-01-13  Geoffrey Garen  <ggaren@apple.com>
29
30         Consider alignment when allocating from a SegregatedFreeList
31         https://bugs.webkit.org/show_bug.cgi?id=140408
32
33         Reviewed by Sam Weinig.
34
35         In preparation for supporting aligned allocation.
36
37         No performance change.
38
39         Since this is just one extra branch in an already expensive function,
40         I decided not to duplicate the function just to avoid the branch in
41         the un-aligned case.
42
43         * bmalloc/SegregatedFreeList.cpp:
44         (bmalloc::SegregatedFreeList::take):
45         * bmalloc/SegregatedFreeList.h:
46
47 2015-01-13  Geoffrey Garen  <ggaren@apple.com>
48
49         Renamed minimum to size in SegregatedFreeList
50         https://bugs.webkit.org/show_bug.cgi?id=140406
51
52         Reviewed by Sam Weinig.
53
54         In preparation for supporting aligned allocation.
55
56         * bmalloc/SegregatedFreeList.cpp:
57         (bmalloc::SegregatedFreeList::takeGreedy):
58         (bmalloc::SegregatedFreeList::take): Every size passed to malloc is
59         really just a minimum. Let's not imply that this value is special.
60
61 2015-01-11  Dan Bernstein  <mitz@apple.com>
62
63         Geoff is organized, but he is not an organization.
64
65         Rubber-stamped by Anders Carlsson.
66
67         * bmalloc.xcodeproj/project.pbxproj: Removed the ORGANIZATIONNAME project attribute.
68
69 2015-01-07  Geoffrey Garen  <ggaren@apple.com>
70
71         Make bmalloc work with ASan
72         https://bugs.webkit.org/show_bug.cgi?id=140194
73
74         Reviewed by Mark Lam.
75
76         * bmalloc/BPlatform.h: Added a way to detect Darwin OSes, since we need
77         an OS-specific API to test for loaded runtime libraries.
78
79         * bmalloc/Environment.cpp:
80         (bmalloc::isASanEnabled):
81         (bmalloc::Environment::computeIsBmallocEnabled): Disabled bmalloc if
82         ASan is enabled, since system malloc has the Asan hooks we need.
83
84         You could check for the ASan compile-time flag instead, but doing this
85         check at runtime prepares bmalloc for a world where it is a dynamic
86         library that might be loaded into projects it did not compile with.
87
88 2015-01-05  Geoffrey Garen  <ggaren@apple.com>
89
90         Fix up bmalloc's PerThread for use on Linux
91         https://bugs.webkit.org/show_bug.cgi?id=139804
92
93         Reviewed by Anders Carlsson.
94
95         The previous implementation was a bit slow.
96
97         * bmalloc/PerThread.h:
98         (bmalloc::PerThreadStorage<Cache>::get):
99         (bmalloc::PerThreadStorage::get):
100         (bmalloc::PerThreadStorage::init): Added a catch-all cross-platform Unix
101         way to do fast per-thread access without taking a lock every time. This
102         probably works on all the platforms we care about, and it matches other
103         techniques we use elsewhere in WebKit.
104
105         (bmalloc::PerThread<T>::getFastCase): Removed the conditional from
106         this class because PerThreadStorage now encapsulates everything that
107         needs to be conditional.
108
109         (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Deleted.
110
111 2014-12-26  Dan Bernstein  <mitz@apple.com>
112
113         <rdar://problem/19348208> REGRESSION (r177027): iOS builds use the wrong toolchain
114         https://bugs.webkit.org/show_bug.cgi?id=139950
115
116         Reviewed by David Kilzer.
117
118         * Configurations/Base.xcconfig: Only define TOOLCHAINS when building for OS X, doing so
119         in a manner that works with Xcode 5.1.1.
120
121 2014-12-15  Geoffrey Garen  <ggaren@apple.com>
122
123         Safari crashes when you set Malloc environment variables
124         https://bugs.webkit.org/show_bug.cgi?id=139656
125
126         Reviewed by Michael Saboff.
127
128         I forgot to cover the realloc() case. Whoops. (OoPS?)
129
130         This time around, I ran the full MallocBench test suite in Malloc=1
131         mode, and it passed.
132
133         * bmalloc/Allocator.cpp:
134         (bmalloc::Allocator::reallocate):
135         * bmalloc/Allocator.h: Pushed realloc() logic down into the allocator.
136         It needs to be down there so that we can do the short-circuiting check
137         for whether bmalloc is enabled first.
138
139         Also added the check.
140
141         * bmalloc/Cache.cpp:
142         (bmalloc::Cache::scavenge):
143         (bmalloc::Cache::Cache):
144         (bmalloc::Cache::reallocateSlowCaseNullCache):
145         * bmalloc/Cache.h:
146         (bmalloc::Cache::deallocator):
147         (bmalloc::Cache::reallocate): Ditto.
148
149         * bmalloc/bmalloc.h:
150         (bmalloc::api::free):
151         (bmalloc::api::realloc): Ditto.
152
153         (bmalloc::api::scavenge): Pushed this down into Cache to match the
154         surrounding functions.
155
156 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
157
158         bmalloc should support system memory analysis tools (part 2)
159         https://bugs.webkit.org/show_bug.cgi?id=139565
160
161         Reviewed by Mark Lam.
162
163         This patch actually queries the environment to see if memory analysis
164         tools have been enabled.
165
166         * bmalloc/Deallocator.cpp:
167         (bmalloc::Deallocator::scavenge): Don't process the object log if
168         we've disabled bmalloc because it will be full of invalid nullptrs.
169
170         * bmalloc/Environment.cpp:
171         (bmalloc::isMallocEnvironmentVariableSet): Test for the list of known
172         Malloc debugging flags. I also added a plain "Malloc" catch-all for
173         when you want to disable bmalloc without enabling any kind of funny
174         business.
175
176         It would be slightly nicer just to iterate the list of environment
177         variables and strstr them, but getenv is the more portable option,
178         and performance here doesn't really matter.
179
180         (bmalloc::isLibgmallocEnabled): Test for the libgmalloc insertion
181         environment variable.
182
183         (bmalloc::Environment::computeIsBmallocEnabled):
184
185 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
186
187         Try to fix the iOS simulator build.
188
189         #include the declaration of malloc / free.
190
191         * bmalloc/Allocator.cpp:
192         * bmalloc/Deallocator.cpp:
193
194 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
195
196         Try to fix the build.
197
198         * bmalloc.xcodeproj/project.pbxproj: Marked a header exported.
199
200 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
201
202         bmalloc should support system memory analysis tools (part 1)
203         https://bugs.webkit.org/show_bug.cgi?id=139559
204
205         Reviewed by Mark Lam.
206
207         This patch adds the hooks to disable bmalloc at runtime if certain
208         environment variables are set, but doesn't actually read from the
209         environment yet.
210
211         No performance change.
212
213         * bmalloc.xcodeproj/project.pbxproj: Added the Environment class, which
214         we'll use to read environment variables and see if memory analysis tools
215         have been enabled.
216
217         * bmalloc/Allocator.cpp:
218         (bmalloc::Allocator::Allocator):
219         (bmalloc::Allocator::allocateSlowCase): Added a hook to disable bmalloc
220         on the allocation path. We cache the setting to make the check fast.
221
222         * bmalloc/Allocator.h: Interface changes.
223
224         * bmalloc/Cache.cpp:
225         (bmalloc::Cache::Cache): Pass a heap pointer through to our allocator
226         and deallocator. This main purpose is to enable them to query the
227         environment for whether bmalloc is enabled; but this is also a slightly
228         cleaner way to guarantee to them that the Heap has been pre-initialized.
229
230         * bmalloc/Deallocator.cpp:
231         (bmalloc::Deallocator::Deallocator): If bmalloc is disable, artificially
232         fill the object log to force us to take the slow path on all deallocations.
233
234         (bmalloc::Deallocator::deallocateSlowCase): Do the disabled check.
235
236         * bmalloc/Deallocator.h: Interface changes.
237
238         * bmalloc/Environment.cpp: Added.
239         (bmalloc::Environment::Environment):
240         (bmalloc::Environment::computeIsBmallocEnabled):
241         * bmalloc/Environment.h: Added.
242         (bmalloc::Environment::isBmallocEnabled): This is the class that will
243         encapsulate looking for environment variables that turn on heap
244         analysis tools.
245
246         * bmalloc/Heap.h:
247         (bmalloc::Heap::environment):
248
249         * bmalloc/Mutex.h:
250         (bmalloc::Mutex::Mutex):
251         * bmalloc/StaticMutex.h: A little refactoring to clarify these comments,
252         since I got super confused about them while writing this patch.
253
254         * bmalloc/VMHeap.cpp: Fixed an #include.
255
256 2014-12-09  David Kilzer  <ddkilzer@apple.com>
257
258         Switch from using PLATFORM_NAME to SDK selectors in ANGLE, bmalloc, gtest, JavaScriptCore, WTF
259         <http://webkit.org/b/139212>
260
261         Reviewed by Joseph Pecoraro.
262
263         * Configurations/Base.xcconfig:
264         - Only set GCC_ENABLE_OBJC_GC, GCC_MODEL_TUNING and TOOLCHAINS
265           on OS X.
266         * Configurations/DebugRelease.xcconfig:
267         - Only set MACOSX_DEPLOYMENT_TARGET and SDKROOT on OS X.
268
269 2014-11-07  Geoffrey Garen  <ggaren@apple.com>
270
271         bmalloc uses 8X more virtual memory than necessary
272         https://bugs.webkit.org/show_bug.cgi?id=138495
273
274         Reviewed by Mark Lam.
275
276         iOS has a per-process virtual memory cap around 1GB, so there's some
277         value to not going totally ham with virtual memory.
278
279         We currently use about 8X the necessary amount:
280             - 2X to align our VM allocation
281             - 4X to reserve small / medium / (2) large chunk VM ranges per superchunk
282
283         We can cut that down:
284             - Return the unaligned portion of our VM allocation (-2X)
285             - Use all the chunks in a superchunk, instead of allocating one
286               chunk per superchunk (-4X)
287
288         * bmalloc/Algorithm.h:
289         (bmalloc::roundUpToMultipleOf): Added a non-constant version of this
290         function so we can call it with getpagesize() at runtime.
291
292         * bmalloc/Chunk.h:
293         * bmalloc/LargeChunk.h:
294         (bmalloc::LargeChunk::create): Deleted. Instead of each chunk allocating
295         its own VM, VMHeap allocates the superchunk and all the chunks in it at a time.
296
297         * bmalloc/VMAllocate.h:
298         (bmalloc::vmValidate):
299         (bmalloc::vmAllocate): ASSERT that mmap succeeds to make crashes clearer
300         if it does not succeed. Allocate precisely, and give back the extra.
301
302         * bmalloc/VMHeap.cpp:
303         (bmalloc::VMHeap::allocateSuperChunk):
304         (bmalloc::VMHeap::allocateSmallChunk): Deleted.
305         (bmalloc::VMHeap::allocateMediumChunk): Deleted.
306         (bmalloc::VMHeap::allocateLargeChunk): Deleted. Use all the chunks
307         in a superchunk, instead of just one.
308
309         * bmalloc/VMHeap.h:
310         (bmalloc::VMHeap::allocateSmallPage):
311         (bmalloc::VMHeap::allocateMediumPage):
312         (bmalloc::VMHeap::allocateLargeRange):
313         * bmalloc/XLargeChunk.h:
314         (bmalloc::XLargeChunk::create): Updated to match changes above.
315
316 2014-11-01  David Kilzer  <ddkilzer@apple.com>
317
318         JavaScriptCore is missing debug info for bmalloc because libbmalloc.a is stripped
319         <https://webkit.org/b/138286>
320         <rdar://problem/18847087>
321
322         Reviewed by Dan Bernstein.
323
324         * Configurations/bmalloc.xcconfig: Set STRIP_INSTALLED_PRODUCT
325         to NO for the target that produces libbmalloc.a so that the
326         debug symbols will be linked into JavaScriptCore and end up in
327         its dSYM file.
328
329 2014-10-30  Dana Burkart  <dburkart@apple.com>
330
331         <rdar://problem/18821260> Prepare for the mysterious future
332
333         Reviewed by Lucas Forschler.
334
335         * Configurations/Base.xcconfig:
336         * Configurations/DebugRelease.xcconfig:
337
338 2014-09-24  Geoffrey Garen  <ggaren@apple.com>
339
340         bmalloc: cleaned up fast path vs slow path
341         https://bugs.webkit.org/show_bug.cgi?id=137081
342
343         Reviewed by Sam Weinig.
344
345         Might be a 1% speedup on MallocBench. Also cleans up the code a bit.
346
347         * bmalloc/Allocator.cpp:
348         (bmalloc::Allocator::Allocator): Merged the small and medium range
349         caches, just like the small and medium allocators. Ranges are abstract
350         objects that don't really care whether they hold small or medium objects,
351         so they don't need to be segregated.
352
353         (bmalloc::Allocator::scavenge): Ditto.
354
355         (bmalloc::Allocator::allocateBumpRangeSlowCase):
356         (bmalloc::Allocator::allocateBumpRange): Same thing here, except that
357         we do care a tiny bit, because we need to specify small vs medium when
358         allocating new ranges from the heap, to ensure that the heap allocates
359         from the right segment of VM.
360
361         (bmalloc::Allocator::allocateLarge):
362         (bmalloc::Allocator::allocateXLarge): NO_INLINE because this was clouding
363         up the fast path. Large allocation performance is dominated by allocation
364         logic and initialization, so inlining it doesn't help.
365
366         (bmalloc::Allocator::allocateSlowCase): Slow path got a bit cleaner since
367         it doesn't need to distinguish small vs medium objects.
368
369         (bmalloc::Allocator::allocateSmallBumpRange): Deleted.
370         (bmalloc::Allocator::allocateMediumBumpRange): Deleted.
371
372         * bmalloc/Allocator.h:
373         * bmalloc/BumpRange.h:
374
375         * bmalloc/Cache.cpp:
376         (bmalloc::Cache::allocateSlowCase): Deleted.
377         (bmalloc::Cache::deallocateSlowCase): Deleted.
378         * bmalloc/Cache.h:
379         (bmalloc::Cache::allocate):
380         (bmalloc::Cache::deallocate):
381         (bmalloc::Cache::allocateFastCase): Deleted.
382         (bmalloc::Cache::deallocateFastCase): Deleted. Removed the Cache slow
383         paths. The downside to this change is that the fast path branches to two
384         distinct failure cases instead of one. The upside is that the slow path
385         doesn't need to re-read the segment register, which is not as cheap as a
386         normal register, and it doesn't need to do an extra level of function 
387         call. Seems to be worth it.
388
389         * bmalloc/Deallocator.h:
390         * bmalloc/Heap.cpp:
391         (bmalloc::Heap::refillSmallBumpRangeCache):
392         (bmalloc::Heap::refillMediumBumpRangeCache):
393         * bmalloc/Heap.h: Updated for interface changes.
394
395         * bmalloc/Sizes.h: The most ranges a cache will hold is the number of
396         small lines in a page / 2, since any other free lines will coalesce
397         with their neighbors.
398
399 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
400
401         Rolled out r173346.
402
403             bmalloc should honor the FastMalloc statistics API
404             https://bugs.webkit.org/show_bug.cgi?id=136592
405
406         This didn't really work. Because we allow ranges with and without
407         physical pages to merge, and we allow double-committing and
408         double-decommitting, we can't rely on commit actions to track memory
409         footprint.
410
411         * bmalloc/Heap.cpp:
412         (bmalloc::Heap::size): Deleted.
413         (bmalloc::Heap::capacity): Deleted.
414         * bmalloc/Heap.h:
415         * bmalloc/VMHeap.cpp:
416         (bmalloc::VMHeap::VMHeap):
417         (bmalloc::VMHeap::allocateSmallChunk):
418         (bmalloc::VMHeap::allocateMediumChunk):
419         (bmalloc::VMHeap::allocateLargeChunk):
420         * bmalloc/VMHeap.h:
421         (bmalloc::VMHeap::allocateSmallPage):
422         (bmalloc::VMHeap::allocateMediumPage):
423         (bmalloc::VMHeap::allocateLargeRange):
424         (bmalloc::VMHeap::deallocateSmallPage):
425         (bmalloc::VMHeap::deallocateMediumPage):
426         (bmalloc::VMHeap::deallocateLargeRange):
427         (bmalloc::VMHeap::size): Deleted.
428         (bmalloc::VMHeap::capacity): Deleted.
429         * bmalloc/bmalloc.h:
430         (bmalloc::api::heapSize): Deleted.
431         (bmalloc::api::heapCapacity): Deleted.
432
433 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
434
435         bmalloc: Allocation should be more precise
436         https://bugs.webkit.org/show_bug.cgi?id=136993
437
438         Reviewed by Gavin Barraclough.
439
440         13% reduction in heap size on the MallocBench *_memory_warning benchmarks.
441
442         This patch teaches the allocator to merge adjacent free lines into a
443         single allocatable range. This allows us to shrink the size of an
444         individual line without increasing fragmentation or the rate of allocator
445         slow paths.
446
447         We'll only take more slow paths when available memory is sparse, which
448         is exactly when it's worth it. When available memory is dense, we'll
449         take fewer slow paths.
450
451         * bmalloc.xcodeproj/project.pbxproj:
452         * bmalloc/Algorithm.h:
453         (bmalloc::divideRoundingUp):
454
455         * bmalloc/Allocator.cpp:
456         (bmalloc::Allocator::Allocator): Updated for interface changes.
457
458         (bmalloc::Allocator::scavenge): Scavenge by object instead of by line.
459         Now that we merge lines, it's not convenient to scavenge by line.
460
461         (bmalloc::Allocator::allocateSmallBumpRange):
462         (bmalloc::Allocator::allocateMediumBumpRange): Allocate whole ranges
463         instead of individual lines.
464
465         (bmalloc::Allocator::allocateSlowCase):
466         (bmalloc::Allocator::allocateSmallLine): Deleted.
467         (bmalloc::Allocator::allocateMediumLine): Deleted.
468         (bmalloc::Allocator::allocateMedium): Deleted.
469         * bmalloc/Allocator.h:
470         (bmalloc::Allocator::allocateFastCase): Folded medium allocations
471         into the standard fast path with small allocations. Since a BumpAllocator
472         just allocates out of an arbitrary range, it doesn't need to distinguish
473         between small and medium lines.
474
475         * bmalloc/BumpAllocator.h:
476         (bmalloc::BumpAllocator::size):
477         (bmalloc::BumpAllocator::BumpAllocator):
478         (bmalloc::BumpAllocator::init):
479         (bmalloc::BumpAllocator::refill):
480         (bmalloc::BumpAllocator::line): Deleted. No need to track line information
481         anymore: the heap just gives us a pointer and a pre-computed number of
482         objects, and we allocate them.
483
484         * bmalloc/Deallocator.cpp:
485         (bmalloc::Deallocator::processObjectLog): Updated for interface changes.
486
487         * bmalloc/Heap.cpp:
488         (bmalloc::Heap::Heap):
489         (bmalloc::Heap::initializeLineMetadata): Pre-compute precise metadata
490         detailing where all objects will lie in memory. After we merge two lines,
491         we might allocate an object that spans from one line to the next. This
492         metadata details which bits of memory overlap in that way, and how they
493         overlap.
494
495         (bmalloc::Heap::refillSmallBumpRangeCache):
496         (bmalloc::Heap::refillMediumBumpRangeCache): Scan a whole page at a time,
497         and merge adjacent free lines into BumpRanges.
498
499         (bmalloc::Heap::allocateSmallPage):
500         (bmalloc::Heap::allocateMediumPage):
501         (bmalloc::Heap::deallocateSmallLine):
502         (bmalloc::Heap::deallocateMediumLine): Track pages rather than lines,
503         since we scan for free memory a page at a time.
504
505         (bmalloc::Heap::allocateSmallLineSlowCase): Deleted.
506         (bmalloc::Heap::allocateMediumLineSlowCase): Deleted. Folded into the
507         fast path.
508
509         * bmalloc/Heap.h:
510         (bmalloc::Heap::derefSmallLine):
511         (bmalloc::Heap::derefMediumLine):
512         (bmalloc::Heap::deallocateSmallLine): Deleted.
513         (bmalloc::Heap::allocateSmallLine): Deleted.
514         (bmalloc::Heap::deallocateMediumLine): Deleted.
515         (bmalloc::Heap::allocateMediumLine): Deleted. Updated for interface changes.
516
517         * bmalloc/Line.h:
518         (bmalloc::Line<Traits>::ref):
519         (bmalloc::Line<Traits>::deref):
520         (bmalloc::Line<Traits>::concurrentRef): Deleted. We don't pass a derefCount
521         anymore, since we only ever deref by 1 now.
522
523         * bmalloc/MediumAllocator.h:
524         (bmalloc::MediumAllocator::isNull): Deleted.
525         (bmalloc::MediumAllocator::MediumAllocator): Deleted.
526         (bmalloc::MediumAllocator::line): Deleted.
527         (bmalloc::MediumAllocator::allocate): Deleted.
528         (bmalloc::MediumAllocator::derefCount): Deleted.
529         (bmalloc::MediumAllocator::refill): Deleted.
530         (bmalloc::MediumAllocator::clear): Deleted. Deleted some code that's
531         been dead for a while, since it doesn't build anymore with this patch.
532
533         * bmalloc/Page.h:
534         (bmalloc::Page::sizeClass):
535         (bmalloc::Page::setSizeClass):
536         (bmalloc::Page::smallSizeClass): Deleted.
537         (bmalloc::Page::setSmallSizeClass): Deleted. Renamed setSmallSizeClass
538         to sizeClass, since we use it for medium sizes too.
539
540         * bmalloc/Sizes.h:
541         (bmalloc::Sizes::sizeClass):
542         (bmalloc::Sizes::objectSize): Shrank line sizes to save memory.
543
544         (bmalloc::Sizes::smallSizeClassFor): Deleted.
545         (bmalloc::Sizes::mediumSizeClassFor): Deleted.
546
547         * bmalloc/bmalloc.h:
548         (bmalloc::api::realloc): Now that we have precise objects sizes, realloc
549         can be a bit more precise. It also has to be, since we can't guarantee
550         that an object ends at the end of a line anymore.
551
552 2014-09-19  Daniel Bates  <dabates@apple.com>
553
554         Always assume internal SDK when building configuration Production
555         https://bugs.webkit.org/show_bug.cgi?id=136925
556         <rdar://problem/18362399>
557
558         Reviewed by Dan Bernstein.
559
560         * Configurations/Base.xcconfig:
561
562 2014-09-16  Geoffrey Garen  <ggaren@apple.com>
563
564         bmalloc: moved line caches from the deallocator to the allocator
565         https://bugs.webkit.org/show_bug.cgi?id=136868
566
567         Reviewed by Gavin Barraclough.
568
569         I did this mostly as a simplification, to make it easier to change the
570         allocation strategy.
571
572         No throughput change on MallocBench. Saves about 50kB.
573
574         Since the deallocator needs to lock the heap when freeing lines anyway,
575         there isn't much benefit to giving the deallocator a local cache of
576         deallocated lines.
577
578         We still give the allocator a local cache of lines because that does
579         reduce the frequency at which it needs to lock the heap in order to
580         acquire more lines.
581
582         * bmalloc/Allocator.cpp:
583         (bmalloc::Allocator::scavenge):
584         (bmalloc::Allocator::allocateSmallLine):
585         (bmalloc::Allocator::allocateMediumLine):
586         (bmalloc::Allocator::allocateMedium):
587         (bmalloc::Allocator::allocateSlowCase):
588         * bmalloc/Allocator.h:
589         * bmalloc/Deallocator.cpp:
590         (bmalloc::Deallocator::Deallocator):
591         (bmalloc::Deallocator::scavenge):
592         (bmalloc::Deallocator::processObjectLog):
593         (bmalloc::Deallocator::deallocateSmallLine): Deleted.
594         (bmalloc::Deallocator::allocateSmallLine): Deleted.
595         (bmalloc::Deallocator::deallocateMediumLine): Deleted.
596         (bmalloc::Deallocator::allocateMediumLine): Deleted.
597         * bmalloc/Deallocator.h:
598
599         * bmalloc/Sizes.h:
600         * bmalloc/VMAllocate.h: Took the opportunity to make the line cache size
601         exactly one page in size. That's about what we were shooting for anyway,
602         and it may make it easier to switch to per-page allocation in future.
603
604 2014-09-15  Geoffrey Garen  <ggaren@apple.com>
605
606         bmalloc: allocate small and medium objects using the same bump pointer class
607         https://bugs.webkit.org/show_bug.cgi?id=136843
608
609         Reviewed by Gavin Barraclough.
610
611         4% speedup on MallocBench.
612
613         Now that medium-sized objects have dedicated per-size allocators, they
614         don't need to use an arbitrary bump pointer allocator. This means that
615         every allocator knows how many objects it will allocate from the start,
616         and we don't need a post-processing step to adjust refcounts based on
617         real allocation count.
618
619         * bmalloc.xcodeproj/project.pbxproj: Renamed SmallAllocator to BumpAllocator
620         since it's used for small and medium objects now.
621
622         * bmalloc/Allocator.cpp:
623         (bmalloc::Allocator::Allocator): Updated to use new interface.
624         (bmalloc::Allocator::scavenge): To "retire" an allocator, we just need
625         to make sure that we finish allocating all the objects in it.
626
627         (bmalloc::Allocator::allocateMedium):
628         (bmalloc::Allocator::allocateSlowCase):
629         (bmalloc::Allocator::retire): Deleted.
630         (bmalloc::Allocator::processSmallAllocatorLog): Deleted.
631         (bmalloc::Allocator::processMediumAllocatorLog): Deleted.
632         * bmalloc/Allocator.h:
633         (bmalloc::Allocator::allocateFastCase): Removed abstractions and data
634         used to post-process an allocator based on how many objects it allocated.
635
636         * bmalloc/BumpAllocator.h: Copied from Source/bmalloc/bmalloc/SmallAllocator.h.
637         (bmalloc::BumpAllocator::BumpAllocator):
638         (bmalloc::BumpAllocator::init):
639         (bmalloc::BumpAllocator::line):
640         (bmalloc::BumpAllocator::validate):
641         (bmalloc::BumpAllocator::allocate):
642         (bmalloc::BumpAllocator::refill):
643         (bmalloc::BumpAllocator::clear): Updated these functions to be agnostic
644         about the kinds of lines they allocate into. In some cases, the line
645         type must be provided as a template parameter by the caller.
646
647         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
648         (bmalloc::SmallAllocator::line): Deleted.
649         (bmalloc::SmallAllocator::allocate): Deleted.
650         (bmalloc::SmallAllocator::objectCount): Deleted.
651         (bmalloc::SmallAllocator::derefCount): Deleted.
652         (bmalloc::SmallAllocator::refill): Deleted.
653         (bmalloc::SmallAllocator::clear): Deleted.
654
655         * bmalloc/ObjectType.h:
656         (bmalloc::isMedium):
657
658         * bmalloc/SmallAllocator.h:
659         (bmalloc::SmallAllocator::isNull): Deleted.
660         (bmalloc::SmallAllocator::canAllocate): Deleted.
661         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
662         (bmalloc::SmallAllocator::line): Deleted.
663         (bmalloc::SmallAllocator::allocate): Deleted.
664         (bmalloc::SmallAllocator::objectCount): Deleted.
665         (bmalloc::SmallAllocator::derefCount): Deleted.
666         (bmalloc::SmallAllocator::refill): Deleted.
667         (bmalloc::SmallAllocator::clear): Deleted.
668
669 2014-09-12  Geoffrey Garen  <ggaren@apple.com>
670
671         Fixed a goof in bmalloc Vector sizing
672         https://bugs.webkit.org/show_bug.cgi?id=136795
673
674         Reviewed by Gavin Barraclough and Sam Weinig.
675
676         We want our minimum vector to be page-sized since the OS will give us
677         a page no matter what -- but we want that many bytes, and not enough
678         bytes to store that many elements.
679
680         * bmalloc/Vector.h: Math is hard.
681
682 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
683
684         bmalloc should segregate medium-sized objects by line like it does for small-sized objects
685         https://bugs.webkit.org/show_bug.cgi?id=136693
686
687         Reviewed by Gavin Barraclough.
688
689         4% reduction in heap size on the MallocBench *_memory_warning benchmarks.
690
691         No throughput change.
692
693         We keep an array of medium allocators, just like our array of small
694         allocators.
695
696         In future, we can simplify the allocation fast path by merging the small
697         and medium allocator arrays. For now, this is the simplest change that
698         gets the win.
699
700         * bmalloc/Allocator.cpp:
701         (bmalloc::Allocator::Allocator):
702         (bmalloc::Allocator::scavenge):
703         (bmalloc::Allocator::allocateMedium):
704         * bmalloc/Allocator.h:
705         * bmalloc/Sizes.h:
706         (bmalloc::Sizes::mediumSizeClassFor):
707
708 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
709
710         Reviewed by Sam Weinig.
711
712         Renamed log => retire for clarity.
713
714         * bmalloc/Allocator.cpp:
715         (bmalloc::Allocator::scavenge):
716         (bmalloc::Allocator::retire):
717         (bmalloc::Allocator::allocateMedium):
718         (bmalloc::Allocator::allocateSlowCase):
719         (bmalloc::Allocator::log): Deleted.
720         * bmalloc/Allocator.h:
721
722 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
723
724         bmalloc: eager scavenge leaves behind a bogus allocator
725         https://bugs.webkit.org/show_bug.cgi?id=136743
726
727         Reviewed by Sam Weinig.
728
729         Be sure to clear the allocator after logging it in the eager scavenge
730         case, so that we don't later try to allocate out of the lines that we
731         have thrown away.
732
733         We didn't need to do this previously because scavenge would only happen
734         at thread exit time, after which no further allocation from the per-thread
735         cache would take place.
736
737         * bmalloc/Allocator.cpp:
738         (bmalloc::Allocator::scavenge):
739         * bmalloc/MediumAllocator.h:
740         (bmalloc::MediumAllocator::clear):
741         * bmalloc/SmallAllocator.h:
742         (bmalloc::SmallAllocator::clear):
743
744 2014-09-05  Geoffrey Garen  <ggaren@apple.com>
745
746         bmalloc should honor the FastMalloc statistics API
747         https://bugs.webkit.org/show_bug.cgi?id=136592
748
749         Reviewed by Gavin Barraclough.
750
751         We do this by tracking "size" and "capacity" in the VM heap.
752
753         The VM heap's "capacity" is all the VM we ever allocated.
754
755         The VM heap's "size" the subset of VM currently held onto by the
756         VM heap (and therefore not in use by the regular heap).
757
758         Somewhat ironically, reducing the process's memory footprint, increases
759         the size of the VM heap, since the VM heap holds the pages that are
760         purely virtual and not physical.
761
762         * bmalloc/Heap.cpp:
763         (bmalloc::Heap::size):
764         (bmalloc::Heap::capacity):
765         * bmalloc/Heap.h:
766         * bmalloc/VMHeap.cpp:
767         (bmalloc::VMHeap::VMHeap):
768         (bmalloc::VMHeap::allocateSmallChunk):
769         (bmalloc::VMHeap::allocateMediumChunk):
770         (bmalloc::VMHeap::allocateLargeChunk):
771         * bmalloc/VMHeap.h:
772         (bmalloc::VMHeap::size):
773         (bmalloc::VMHeap::capacity):
774         (bmalloc::VMHeap::allocateSmallPage):
775         (bmalloc::VMHeap::allocateMediumPage):
776         (bmalloc::VMHeap::allocateLargeRange):
777         (bmalloc::VMHeap::deallocateSmallPage):
778         (bmalloc::VMHeap::deallocateMediumPage):
779         (bmalloc::VMHeap::deallocateLargeRange):
780         * bmalloc/bmalloc.h:
781         (bmalloc::api::heapSize):
782         (bmalloc::api::heapCapacity):
783
784 2014-09-02  Geoffrey Garen  <ggaren@apple.com>
785
786         bmalloc crashes on the EWS bots (due to bad large object allocation)
787         https://bugs.webkit.org/show_bug.cgi?id=136469
788
789         Reviewed by Andreas Kling.
790
791         It's possible to convince bmalloc to perform a bad large object allocation,
792         through these steps:
793
794         (1) Insert object A into freelist F0.
795
796         (2) Split, merge and split again A's neighbors such that object B is
797         inserted into freelist F0, with boundary tag and size equal to object A,
798         but pointer not completely equal to object A. Put object B at the head of F0.
799
800         (3) Allocate some other object from F0, swapping its position in the
801         freelist with object B, such that object A is now ahead of object B.
802
803         --> Now, the next allocation for size A/B will allocate object A, which
804         has a slightly wrong idea about where the object actually begins.
805         Immediately, you'll corrupt a little memory, and over time, you'll also
806         corrupt boundary tag metadata.
807
808         The solution is to store the begin pointer in the boundary tag. Luckily,
809         this doesn't make the tag any bigger, and it's not a noticeable slowdown
810         on MallocBench.
811
812         * bmalloc/Algorithm.h:
813         (bmalloc::rightShift):
814         * bmalloc/BeginTag.h:
815         (bmalloc::BeginTag::isInFreeList): This is the bug fix. Make sure to
816         validate the start pointer when popping off the free list. Through a
817         very uncommon set of steps, it is possible to have an item in the free
818         list that is valid by all accounts except for its start pointer.
819
820         * bmalloc/BoundaryTag.h:
821         (bmalloc::BoundaryTag::compactBegin):
822         (bmalloc::BoundaryTag::setRange):
823         (bmalloc::BoundaryTag::setSize): Deleted. Record a compact version of the
824         start pointer. We don't need the whole pointer -- just the offset, in
825         largeAlignment increments, into the relevant boundary tag bucket.
826
827         * bmalloc/BoundaryTagInlines.h:
828         (bmalloc::validateNext):
829         (bmalloc::BoundaryTag::init):
830         (bmalloc::BoundaryTag::mergeLarge):
831         (bmalloc::BoundaryTag::splitLarge):
832         * bmalloc/SegregatedFreeList.cpp:
833         (bmalloc::SegregatedFreeList::insert):
834         (bmalloc::SegregatedFreeList::takeGreedy):
835         (bmalloc::SegregatedFreeList::take): Provide the whole range instead of
836         the size when establishing a boundary tag, as required by the new
837         interface.
838
839         * bmalloc/Sizes.h:
840
841 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
842
843         Fixed a bmalloc crash seen on the EWS bot
844         https://bugs.webkit.org/show_bug.cgi?id=135955
845
846         Reviewed by Andreas Kling.
847
848         * bmalloc/Syscall.h: Some CG APIs vm_copy their input buffers. If the
849         input buffer is a malloc region, that region will get marked Copy-On-Write
850         by the kernel. Calls to madvise() for COW regions fail and return EINVAL
851         on older OS X's. In 10.10, they still fail, but they do not return
852         EINVAL.
853
854         So, we can only ASSERT that our syscalls succeed starting with 10.10.
855
856 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
857
858         Fixed the bmalloc build
859         https://bugs.webkit.org/show_bug.cgi?id=135953
860
861         Reviewed by Andreas Kling.
862
863         * bmalloc.xcodeproj/project.pbxproj: Marked a few headers as private.
864         These headers are used, so they must be available outside the project.
865
866 2014-08-13  Daniel Bates  <dabates@apple.com>
867
868         Attempt to fix the build following <http://trac.webkit.org/changeset/172576>
869         (https://bugs.webkit.org/show_bug.cgi?id=135895)
870
871         Substitute PerThreadStorage<T>::initSharedKeyIfNeeded() for initSharedKeyIfNeeded() in
872         implementation of PerThread<T>::getFastCase().
873
874         * bmalloc/PerThread.h:
875         (bmalloc::PerThread<T>::getFastCase):
876
877 2014-08-13  Daniel Bates  <dabates@apple.com>
878
879         Make bmalloc::PerThread work without C++ thread local storage
880         https://bugs.webkit.org/show_bug.cgi?id=135895
881
882         Reviewed by Geoffrey Garen.
883
884         Implement support for building bmalloc without C++ thread local storage.
885
886         * bmalloc/BPlatform.h: Remove macro define BPLATFORM_IOS_SIMULATOR. Added macro function
887         BCOMPILER_SUPPORTS() and macro define BCOMPILER_SUPPORTS_CXX_THREAD_LOCAL that can be used
888         to determine whether the compiler supports C++ thread local storage.
889         * bmalloc/PerThread.h:
890         (bmalloc::PerThreadStorage::get): Modified to call pthread_getspecific() when building
891         without C++ thread local storage.
892         (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Added.
893         (bmalloc::PerThreadStorage::init): Moved logic to initialize shared Pthread key from here to
894         PerThreadStorage::initSharedKeyIfNeeded().
895         (bmalloc::PerThread<T>::getFastCase): Modified to call PerThreadStorage::initSharedKeyIfNeeded()
896         before querying PerThreadStorage::get() when building without C++ thread local storage so as to
897         ensure that the shared key has been initialized.
898         (_pthread_setspecific_direct): Deleted.
899         (_pthread_getspecific_direct): Deleted.
900
901 2014-08-13  Daniel Bates  <dabates@apple.com>
902
903         [iOS] Make JavaScriptCore and bmalloc build with the public SDK
904         https://bugs.webkit.org/show_bug.cgi?id=135848
905
906         Reviewed by Geoffrey Garen.
907
908         * bmalloc/BPlatform.h: Added macro BPLATFORM_IOS_SIMULATOR, which evaluates to true
909         when building for the iOS Simulator.
910         * bmalloc/PerThread.h: Use pthread_machdep.h code path when building for iOS Simulator
911         using the public SDK.
912         (_pthread_setspecific_direct): Added; only defined when building for the iOS Simulator
913         using the public SDK.
914         (_pthread_getspecific_direct): Added; only defined when building for the iOS Simulator
915         using the public SDK.
916
917 2014-08-12  Daniel Bates  <dabates@apple.com>
918
919         BPLATFORM(IOS) always evaluates to false
920         https://bugs.webkit.org/show_bug.cgi?id=135843
921
922         Reviewed by Geoffrey Garen.
923
924         Fix typo in definition of BPLATFORM() and include system header TargetConditionals.h
925         (when building on an Apple platform) so that BPLATFORM(X) evaluates to true when
926         building for platform X. In particular, so that BPLATFORM(IOS) evaluates to true when
927         building for iOS.
928
929         As a side effect of this change, the change made in <http://trac.webkit.org/changeset/167289>
930         will be honored and iOS will assume a VM page size of 16kB (again) instead of 4kB.
931
932         * bmalloc/BPlatform.h:
933
934 2014-08-11  Andy Estes  <aestes@apple.com>
935
936         [iOS] Get rid of iOS.xcconfig
937         https://bugs.webkit.org/show_bug.cgi?id=135809
938
939         Reviewed by Joseph Pecoraro.
940
941         All iOS.xcconfig did was include AspenFamily.xcconfig, so there's no need for the indirection.
942
943         * Configurations/Base.xcconfig:
944         * Configurations/iOS.xcconfig: Removed.
945         * bmalloc.xcodeproj/project.pbxproj:
946
947 2014-05-01  Dan Bernstein  <mitz@apple.com>
948
949         Fixed production builds for the iOS Simulator.
950         <rdar://problem/16792221>
951
952         * Configurations/bmalloc.xcconfig: Include INSTALL_PATH_PREFIX in
953         PRIVATE_HEADERS_FOLDER_PATH when installing.
954
955 2014-04-20  Geoffrey Garen  <ggaren@apple.com>
956
957         bmalloc: Segregate pages by objects size
958         https://bugs.webkit.org/show_bug.cgi?id=131909
959
960         Reviewed by Andreas Kling.
961
962         2% reduction in memory-at-end on the Membuster memory_warning benchmarks.
963
964         * bmalloc/Allocator.cpp:
965         (bmalloc::Allocator::allocateSlowCase):
966         * bmalloc/Allocator.h:
967         (bmalloc::Allocator::allocateFastCase):
968         (bmalloc::Allocator::smallAllocatorFor): Use the new shared helper
969         function for size class calculation.
970
971         * bmalloc/Deallocator.cpp:
972         (bmalloc::Deallocator::Deallocator):
973         (bmalloc::Deallocator::scavenge):
974         (bmalloc::Deallocator::deallocateSmallLine):
975         (bmalloc::Deallocator::allocateSmallLine):
976         * bmalloc/Deallocator.h: Keep a cache for every size class, since the
977         cache can't be shared anymore.
978
979         * bmalloc/Heap.cpp:
980         (bmalloc::Heap::allocateSmallLineSlowCase):
981         * bmalloc/Heap.h:
982         (bmalloc::Heap::deallocateSmallLine): Ditto.
983
984         (bmalloc::Heap::allocateSmallLine): Check size class in addition to
985         page refcount when allocating a line because we might have deallocated
986         the page and the recycled it for another size class.
987
988         (bmalloc::Heap::deallocateMediumLine):
989         (bmalloc::Heap::allocateMediumLine):
990         * bmalloc/Line.h:
991         (bmalloc::Line::refCount):
992         * bmalloc/Page.h:
993         (bmalloc::Page::refCount):
994         (bmalloc::Page::smallSizeClass):
995         (bmalloc::Page::setSmallSizeClass):
996         (bmalloc::Page<Traits>::refCount): Deleted.
997         * bmalloc/Sizes.h:
998         (bmalloc::Sizes::smallSizeClassFor): New shared API for computing
999         an index into an array from a size.
1000
1001 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1002
1003         bmalloc: Improved alignment in LargeChunk
1004         https://bugs.webkit.org/show_bug.cgi?id=131895
1005
1006         Reviewed by Andreas Kling.
1007
1008         * bmalloc/Chunk.h:
1009         * bmalloc/LargeChunk.h: Align to vmPageSize just like Chunk does.
1010         Technically, the previous alignment was harmless, but I would prefer,
1011         dear reader, not to have to explain the interlocking set of
1012         circumstances that made it so.
1013
1014 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1015
1016         Rolled out r167502 because it caused a crash on the facebook benchmark.
1017
1018         Unreviewed.
1019
1020             bmalloc: Added an XSmall line size
1021             https://bugs.webkit.org/show_bug.cgi?id=131851
1022
1023             Reviewed by Sam Weinig.
1024
1025 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1026
1027         bmalloc: Mutex should be harder to use wrong
1028         https://bugs.webkit.org/show_bug.cgi?id=131879
1029
1030         Reviewed by Andreas Kling.
1031
1032         Mutex now has a proper constructor, so you can't deadlock by forgetting
1033         to initialize it.
1034
1035         * bmalloc.xcodeproj/project.pbxproj:
1036         * bmalloc/Allocator.cpp:
1037         (bmalloc::Allocator::processXSmallAllocatorLog):
1038         (bmalloc::Allocator::processSmallAllocatorLog):
1039         (bmalloc::Allocator::processMediumAllocatorLog):
1040         (bmalloc::Allocator::allocateLarge):
1041         (bmalloc::Allocator::allocateXLarge): Global replace Mutex => StaticMutex,
1042         since the Heap mutex is a static.
1043
1044         * bmalloc/AsyncTask.h:
1045         (bmalloc::Function>::AsyncTask): Use Mutex, since we're not static. No
1046         need for explicit initialization anymore.
1047
1048         * bmalloc/Deallocator.cpp:
1049         (bmalloc::Deallocator::scavenge):
1050         (bmalloc::Deallocator::deallocateLarge):
1051         (bmalloc::Deallocator::deallocateXLarge):
1052         (bmalloc::Deallocator::processObjectLog):
1053         (bmalloc::Deallocator::deallocateSmallLine):
1054         (bmalloc::Deallocator::deallocateXSmallLine):
1055         (bmalloc::Deallocator::allocateSmallLine):
1056         (bmalloc::Deallocator::allocateXSmallLine):
1057         (bmalloc::Deallocator::deallocateMediumLine):
1058         (bmalloc::Deallocator::allocateMediumLine):
1059         * bmalloc/Deallocator.h:
1060         * bmalloc/Heap.cpp:
1061         (bmalloc::sleep):
1062         (bmalloc::Heap::Heap):
1063         (bmalloc::Heap::concurrentScavenge):
1064         (bmalloc::Heap::scavenge):
1065         (bmalloc::Heap::scavengeSmallPages):
1066         (bmalloc::Heap::scavengeXSmallPages):
1067         (bmalloc::Heap::scavengeMediumPages):
1068         (bmalloc::Heap::scavengeLargeRanges):
1069         (bmalloc::Heap::allocateXSmallLineSlowCase):
1070         (bmalloc::Heap::allocateSmallLineSlowCase):
1071         (bmalloc::Heap::allocateMediumLineSlowCase):
1072         (bmalloc::Heap::allocateXLarge):
1073         (bmalloc::Heap::deallocateXLarge):
1074         (bmalloc::Heap::allocateLarge):
1075         (bmalloc::Heap::deallocateLarge):
1076         * bmalloc/Heap.h:
1077         (bmalloc::Heap::deallocateXSmallLine):
1078         (bmalloc::Heap::allocateXSmallLine):
1079         (bmalloc::Heap::deallocateSmallLine):
1080         (bmalloc::Heap::allocateSmallLine):
1081         (bmalloc::Heap::deallocateMediumLine):
1082         (bmalloc::Heap::allocateMediumLine):
1083         * bmalloc/Line.h:
1084         (bmalloc::Line<Traits>::deref):
1085         * bmalloc/Mutex.cpp: Removed.
1086         * bmalloc/Mutex.h:
1087         (bmalloc::Mutex::Mutex):
1088         (bmalloc::Mutex::init): Deleted.
1089         (bmalloc::Mutex::try_lock): Deleted.
1090         (bmalloc::Mutex::lock): Deleted.
1091         (bmalloc::Mutex::unlock): Deleted.
1092         * bmalloc/Page.h:
1093         (bmalloc::Page<Traits>::ref):
1094         (bmalloc::Page<Traits>::deref):
1095         (bmalloc::Page<Traits>::refCount):
1096         * bmalloc/PerProcess.h:
1097         (bmalloc::PerProcess::mutex):
1098         (bmalloc::PerProcess<T>::getSlowCase):
1099         * bmalloc/StaticMutex.cpp: Added.
1100         (bmalloc::StaticMutex::lockSlowCase):
1101         * bmalloc/StaticMutex.h: Added.
1102         (bmalloc::StaticMutex::init):
1103         (bmalloc::StaticMutex::try_lock):
1104         (bmalloc::StaticMutex::lock):
1105         (bmalloc::StaticMutex::unlock):
1106         * bmalloc/VMHeap.h:
1107         (bmalloc::VMHeap::deallocateXSmallPage):
1108         (bmalloc::VMHeap::deallocateSmallPage):
1109         (bmalloc::VMHeap::deallocateMediumPage):
1110         (bmalloc::VMHeap::deallocateLargeRange):
1111         * bmalloc/bmalloc.h:
1112         (bmalloc::api::scavenge): Global replace Mutex => StaticMutex,
1113         since the Heap mutex is a static.
1114
1115 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
1116
1117         bmalloc: AsyncTask should use Mutex instead of std::mutex
1118         https://bugs.webkit.org/show_bug.cgi?id=131865
1119
1120         Reviewed by Gavin Barraclough.
1121
1122         std::mutex is so slow that it makes parallelizing simple tasks through
1123         AsyncTask a net regression. Mutex fixes this.
1124
1125         * bmalloc/AsyncTask.h:
1126         (bmalloc::Function>::AsyncTask):
1127         (bmalloc::Function>::join):
1128         (bmalloc::Function>::runSlowCase):
1129         (bmalloc::Function>::entryPoint):
1130         * bmalloc/Mutex.h:
1131         (bmalloc::Mutex::init):
1132
1133 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
1134
1135         bmalloc: Added an XSmall line size
1136         https://bugs.webkit.org/show_bug.cgi?id=131851
1137
1138         Reviewed by Sam Weinig.
1139
1140         Reduces malloc footprint on Membuster recordings by 10%.
1141
1142         This is a throughput regression, but we're still way ahead of TCMalloc.
1143         I have some ideas for how to recover the regression -- but I wanted to
1144         get this win in first.
1145
1146         Full set of benchmark results:
1147
1148                 bmalloc> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks --measure-heap nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1149                                                                                                 
1150                                                                nopatch                      patch                                Δ
1151                 Peak Memory:
1152                     reddit_memory_warning                      7,896kB                    7,532kB                  ^ 1.05x smaller
1153                     flickr_memory_warning                     12,968kB                   12,324kB                  ^ 1.05x smaller
1154                     theverge_memory_warning                   16,672kB                   15,200kB                   ^ 1.1x smaller
1155
1156                     <geometric mean>                          11,952kB                   11,216kB                  ^ 1.07x smaller
1157                     <arithmetic mean>                         12,512kB                   11,685kB                  ^ 1.07x smaller
1158                     <harmonic mean>                           11,375kB                   10,726kB                  ^ 1.06x smaller
1159
1160                 Memory at End:
1161                     reddit_memory_warning                      7,320kB                    6,856kB                  ^ 1.07x smaller
1162                     flickr_memory_warning                     10,848kB                    9,692kB                  ^ 1.12x smaller
1163                     theverge_memory_warning                   16,380kB                   14,872kB                   ^ 1.1x smaller
1164
1165                     <geometric mean>                          10,916kB                    9,961kB                   ^ 1.1x smaller
1166                     <arithmetic mean>                         11,516kB                   10,473kB                   ^ 1.1x smaller
1167                     <harmonic mean>                           10,350kB                    9,485kB                  ^ 1.09x smaller
1168
1169                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1170                                                                                                 
1171                                                    nopatch                patch                         Δ
1172                 Execution Time:
1173                     churn                            127ms                151ms            ! 1.19x slower
1174                     list_allocate                    130ms                164ms            ! 1.26x slower
1175                     tree_allocate                    109ms                127ms            ! 1.17x slower
1176                     tree_churn                       115ms                120ms            ! 1.04x slower
1177                     facebook                         240ms                259ms            ! 1.08x slower
1178                     fragment                          91ms                131ms            ! 1.44x slower
1179                     fragment_iterate                 105ms                106ms            ! 1.01x slower
1180                     message_one                      260ms                259ms             ^ 1.0x faster
1181                     message_many                     149ms                154ms            ! 1.03x slower
1182                     medium                           194ms                248ms            ! 1.28x slower
1183                     big                              157ms                160ms            ! 1.02x slower
1184
1185                     <geometric mean>                 144ms                163ms            ! 1.13x slower
1186                     <arithmetic mean>                152ms                171ms            ! 1.12x slower
1187                     <harmonic mean>                  137ms                156ms            ! 1.14x slower
1188
1189                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1190                                                                                                 
1191                                                                        nopatch                          patch                                     Δ
1192                 Execution Time:
1193                     churn                                                126ms                          148ms                        ! 1.17x slower
1194                     churn --parallel                                      62ms                           76ms                        ! 1.23x slower
1195                     list_allocate                                        130ms                          164ms                        ! 1.26x slower
1196                     list_allocate --parallel                             120ms                          175ms                        ! 1.46x slower
1197                     tree_allocate                                        111ms                          127ms                        ! 1.14x slower
1198                     tree_allocate --parallel                              95ms                          135ms                        ! 1.42x slower
1199                     tree_churn                                           115ms                          124ms                        ! 1.08x slower
1200                     tree_churn --parallel                                107ms                          126ms                        ! 1.18x slower
1201                     facebook                                             240ms                          276ms                        ! 1.15x slower
1202                     facebook --parallel                                  802ms                        1,088ms                        ! 1.36x slower
1203                     fragment                                              92ms                          130ms                        ! 1.41x slower
1204                     fragment --parallel                                   66ms                          124ms                        ! 1.88x slower
1205                     fragment_iterate                                     109ms                          127ms                        ! 1.17x slower
1206                     fragment_iterate --parallel                           55ms                           64ms                        ! 1.16x slower
1207                     message_one                                          260ms                          260ms                                      
1208                     message_many                                         170ms                          238ms                         ! 1.4x slower
1209                     medium                                               185ms                          250ms                        ! 1.35x slower
1210                     medium --parallel                                    210ms                          334ms                        ! 1.59x slower
1211                     big                                                  150ms                          169ms                        ! 1.13x slower
1212                     big --parallel                                       138ms                          144ms                        ! 1.04x slower
1213
1214                     <geometric mean>                                     135ms                          170ms                        ! 1.26x slower
1215                     <arithmetic mean>                                    167ms                          214ms                        ! 1.28x slower
1216                     <harmonic mean>                                      117ms                          148ms                        ! 1.26x slower
1217
1218                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
1219
1220                                                                     TC                      patch                                Δ
1221                 Peak Memory:
1222                     reddit_memory_warning                     13,836kB                   13,436kB                  ^ 1.03x smaller
1223                     flickr_memory_warning                     24,868kB                   25,188kB                   ! 1.01x bigger
1224                     theverge_memory_warning                   24,504kB                   26,636kB                   ! 1.09x bigger
1225
1226                     <geometric mean>                          20,353kB                   20,812kB                   ! 1.02x bigger
1227                     <arithmetic mean>                         21,069kB                   21,753kB                   ! 1.03x bigger
1228                     <harmonic mean>                           19,570kB                   19,780kB                   ! 1.01x bigger
1229
1230                 Memory at End:
1231                     reddit_memory_warning                      8,656kB                   10,016kB                   ! 1.16x bigger
1232                     flickr_memory_warning                     11,844kB                   13,784kB                   ! 1.16x bigger
1233                     theverge_memory_warning                   18,516kB                   22,748kB                   ! 1.23x bigger
1234
1235                     <geometric mean>                          12,382kB                   14,644kB                   ! 1.18x bigger
1236                     <arithmetic mean>                         13,005kB                   15,516kB                   ! 1.19x bigger
1237                     <harmonic mean>                           11,813kB                   13,867kB                   ! 1.17x bigger
1238
1239                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
1240                                                                                                 
1241                                                         TC                patch                         Δ
1242                 Execution Time:
1243                     churn                            416ms                148ms            ^ 2.81x faster
1244                     list_allocate                    463ms                164ms            ^ 2.82x faster
1245                     tree_allocate                    292ms                127ms             ^ 2.3x faster
1246                     tree_churn                       157ms                120ms            ^ 1.31x faster
1247                     facebook                         327ms                276ms            ^ 1.18x faster
1248                     fragment                         335ms                129ms             ^ 2.6x faster
1249                     fragment_iterate                 344ms                108ms            ^ 3.19x faster
1250                     message_one                      386ms                258ms             ^ 1.5x faster
1251                     message_many                     410ms                154ms            ^ 2.66x faster
1252                     medium                           391ms                245ms             ^ 1.6x faster
1253                     big                              261ms                167ms            ^ 1.56x faster
1254
1255                     <geometric mean>                 332ms                164ms            ^ 2.02x faster
1256                     <arithmetic mean>                344ms                172ms            ^ 1.99x faster
1257                     <harmonic mean>                  317ms                157ms            ^ 2.02x faster
1258
1259         * bmalloc.xcodeproj/project.pbxproj:
1260         * bmalloc/Allocator.cpp:
1261         (bmalloc::Allocator::Allocator): Don't assume that each allocator's
1262         index corresponds with its size. Instead, use the size selection function
1263         explicitly. Now that we have XSmall, some small allocator entries are
1264         unused.
1265
1266         (bmalloc::Allocator::scavenge):
1267         (bmalloc::Allocator::log):
1268         (bmalloc::Allocator::processXSmallAllocatorLog):
1269         (bmalloc::Allocator::allocateSlowCase):
1270         * bmalloc/Allocator.h:
1271         (bmalloc::Allocator::xSmallAllocatorFor):
1272         (bmalloc::Allocator::allocateFastCase):
1273         * bmalloc/Chunk.h:
1274         * bmalloc/Deallocator.cpp:
1275         (bmalloc::Deallocator::scavenge):
1276         (bmalloc::Deallocator::processObjectLog):
1277         (bmalloc::Deallocator::deallocateSlowCase):
1278         (bmalloc::Deallocator::deallocateXSmallLine):
1279         (bmalloc::Deallocator::allocateXSmallLine):
1280         * bmalloc/Deallocator.h:
1281         (bmalloc::Deallocator::deallocateFastCase):
1282         * bmalloc/Heap.cpp:
1283         (bmalloc::Heap::scavenge):
1284         (bmalloc::Heap::scavengeXSmallPages):
1285         (bmalloc::Heap::allocateXSmallLineSlowCase):
1286         * bmalloc/Heap.h:
1287         (bmalloc::Heap::deallocateXSmallLine):
1288         (bmalloc::Heap::allocateXSmallLine):
1289         * bmalloc/LargeChunk.h:
1290         (bmalloc::LargeChunk::get):
1291         (bmalloc::LargeChunk::endTag):
1292         * bmalloc/Line.h:
1293         * bmalloc/MediumAllocator.h:
1294         (bmalloc::MediumAllocator::allocate):
1295         (bmalloc::MediumAllocator::refill):
1296         * bmalloc/ObjectType.cpp:
1297         (bmalloc::objectType):
1298         * bmalloc/ObjectType.h:
1299         (bmalloc::isXSmall):
1300         (bmalloc::isSmall):
1301         (bmalloc::isMedium):
1302         (bmalloc::isLarge):
1303         (bmalloc::isSmallOrMedium): Deleted.
1304         * bmalloc/SegregatedFreeList.h: I boiler-plate copied existing code for
1305         handling small objects. There's probably a reasonable way to share this
1306         code in the future -- I'll look into that once it's stopped changing.
1307
1308         * bmalloc/Sizes.h: Tweaked size classes to make Membuster happy. This
1309         is the main reason things got slower.
1310
1311         * bmalloc/SmallAllocator.h:
1312         (bmalloc::SmallAllocator::allocate):
1313         * bmalloc/SmallTraits.h:
1314         * bmalloc/VMHeap.cpp:
1315         (bmalloc::VMHeap::allocateXSmallChunk):
1316         * bmalloc/VMHeap.h:
1317         (bmalloc::VMHeap::allocateXSmallPage):
1318         (bmalloc::VMHeap::deallocateXSmallPage):
1319         * bmalloc/XSmallAllocator.h: Added.
1320         (bmalloc::XSmallAllocator::isNull):
1321         (bmalloc::XSmallAllocator::canAllocate):
1322         (bmalloc::XSmallAllocator::XSmallAllocator):
1323         (bmalloc::XSmallAllocator::line):
1324         (bmalloc::XSmallAllocator::allocate):
1325         (bmalloc::XSmallAllocator::objectCount):
1326         (bmalloc::XSmallAllocator::derefCount):
1327         (bmalloc::XSmallAllocator::refill):
1328         * bmalloc/XSmallChunk.h: Added.
1329         * bmalloc/XSmallLine.h: Added.
1330         * bmalloc/XSmallPage.h: Added.
1331         * bmalloc/XSmallTraits.h: Added.
1332         * bmalloc/bmalloc.h:
1333         (bmalloc::api::realloc): Boiler-plate copy, as above.
1334
1335 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
1336
1337         MallocBench should scavenge explicitly instead of waiting
1338         https://bugs.webkit.org/show_bug.cgi?id=131661
1339
1340         Reviewed by Andreas Kling.
1341
1342         Added explicit scavenge support to bmalloc. This isn't a memory win,
1343         since bmalloc's per-thread cache is so small. But it makes testing
1344         simpler.
1345
1346         * bmalloc/Allocator.cpp:
1347         (bmalloc::Allocator::~Allocator):
1348         (bmalloc::Allocator::scavenge):
1349         * bmalloc/Allocator.h:
1350         * bmalloc/Cache.cpp:
1351         (bmalloc::Cache::operator new):
1352         (bmalloc::Cache::operator delete):
1353         (bmalloc::Cache::Cache):
1354         (bmalloc::Cache::scavenge):
1355         * bmalloc/Cache.h:
1356         * bmalloc/Deallocator.cpp:
1357         (bmalloc::Deallocator::~Deallocator):
1358         (bmalloc::Deallocator::scavenge):
1359         * bmalloc/Deallocator.h: Factored existing scavenging code into helper
1360         functions, for reuse.
1361
1362         * bmalloc/Heap.cpp:
1363         (bmalloc::sleep):
1364         (bmalloc::Heap::concurrentScavenge):
1365         (bmalloc::Heap::scavenge):
1366         (bmalloc::Heap::scavengeSmallPages):
1367         (bmalloc::Heap::scavengeMediumPages):
1368         (bmalloc::Heap::scavengeLargeRanges):
1369         * bmalloc/Heap.h: Made scavenge sleep duration a parameter. Forced
1370         scavenging -- in response to a benchmark or a low memory warning --
1371         wants to complete as soon as possible, so its sleep duration is 0.
1372
1373         * bmalloc/bmalloc.h:
1374         (bmalloc::api::scavenge):
1375         * bmalloc/mbmalloc.cpp: Exported the scavenge API for MallocBench's use.
1376
1377 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
1378
1379         Use 4kB pages on Mac
1380         https://bugs.webkit.org/show_bug.cgi?id=131658
1381
1382         Reviewed by Sam Weinig.
1383
1384         This reduces memory use a lot on Membuster:
1385
1386                                                                   base                      patch                                Δ
1387                 Execution Time:
1388                     reddit_memory_warning                         18ms                       17ms                   ^ 1.06x faster
1389                     flickr_memory_warning                         34ms                       36ms                   ! 1.06x slower
1390                     theverge_memory_warning                       39ms                       41ms                   ! 1.05x slower
1391
1392                     <geometric mean>                              29ms                       29ms                   ! 1.02x slower
1393                     <arithmetic mean>                             30ms                       31ms                   ! 1.03x slower
1394                     <harmonic mean>                               27ms                       27ms                    ^ 1.0x faster
1395
1396                 Peak Memory:
1397                     reddit_memory_warning                     16,412kB                   16,436kB                    ! 1.0x bigger
1398                     flickr_memory_warning                     30,120kB                   30,184kB                    ! 1.0x bigger
1399                     theverge_memory_warning                   33,408kB                   33,420kB                    ! 1.0x bigger
1400
1401                     <geometric mean>                          25,466kB                   25,499kB                    ! 1.0x bigger
1402                     <arithmetic mean>                         26,647kB                   26,680kB                    ! 1.0x bigger
1403                     <harmonic mean>                           24,181kB                   24,214kB                    ! 1.0x bigger
1404
1405                 Memory at End:
1406                     reddit_memory_warning                      2,404kB                    1,920kB                  ^ 1.25x smaller
1407                     flickr_memory_warning                      3,764kB                    3,072kB                  ^ 1.23x smaller
1408                     theverge_memory_warning                    3,648kB                    3,132kB                  ^ 1.16x smaller
1409
1410                     <geometric mean>                           3,208kB                    2,644kB                  ^ 1.21x smaller
1411                     <arithmetic mean>                          3,272kB                    2,708kB                  ^ 1.21x smaller
1412                     <harmonic mean>                            3,139kB                    2,574kB                  ^ 1.22x smaller
1413
1414
1415         * bmalloc.xcodeproj/project.pbxproj:
1416         * bmalloc/BPlatform.h: Added.
1417         * bmalloc/VMAllocate.h: Only use 16kB pages on iOS because the page size
1418         is 4kB on Mac.
1419
1420 2014-04-14  Alexey Proskuryakov  <ap@apple.com>
1421
1422         Fixed svn:ignore on bmalloc.xcodeproj, it had erroneous leading spaces.
1423
1424         * bmalloc.xcodeproj: Modified property svn:ignore.
1425
1426 2014-04-13  Geoffrey Garen  <ggaren@apple.com>
1427
1428         Fixed some mbmalloc exports
1429         https://bugs.webkit.org/show_bug.cgi?id=131599
1430
1431         Reviewed by Ryosuke Niwa.
1432
1433         * bmalloc.xcodeproj/project.pbxproj: Made some headers a private part
1434         of the project, so we can call them from API.
1435
1436         * bmalloc/mbmalloc.cpp: Marked the mbmalloc functions with default
1437         visibility, so they show up as exported in the .dylib.
1438
1439 2014-04-09  Geoffrey Garen  <ggaren@apple.com>
1440
1441         Put bmalloc headers in the right place
1442         https://bugs.webkit.org/show_bug.cgi?id=131464
1443
1444         Reviewed by Mark Rowe.
1445
1446         * Configurations/bmalloc.xcconfig: Set PRIVATE_HEADERS_FOLDER_PATH to
1447         specify that we don't just want to dump all of our generically-named
1448         headers into /usr/local/include.
1449
1450 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
1451
1452         Made bmalloc more #include friendly
1453         https://bugs.webkit.org/show_bug.cgi?id=131386
1454
1455         Reviewed by Andreas Kling.
1456
1457         Marked a bunch of headers private so they can be used from client code
1458         that #includes bmalloc.h.
1459
1460         Renamed ASSERT macros to BASSERT. This matches their header, which already
1461         had to be renamed, and fixes conflicts with WTF's ASSERT macros.
1462
1463         * bmalloc.xcodeproj/project.pbxproj:
1464         * bmalloc/Allocator.cpp:
1465         (bmalloc::Allocator::allocateSlowCase):
1466         * bmalloc/AsyncTask.h:
1467         (bmalloc::Function>::runSlowCase):
1468         * bmalloc/BAssert.h:
1469         * bmalloc/BoundaryTag.h:
1470         (bmalloc::BoundaryTag::setSize):
1471         * bmalloc/BoundaryTagInlines.h:
1472         (bmalloc::validate):
1473         (bmalloc::BoundaryTag::init):
1474         (bmalloc::BoundaryTag::deallocate):
1475         (bmalloc::BoundaryTag::splitLarge):
1476         (bmalloc::BoundaryTag::allocate):
1477         * bmalloc/Chunk.h:
1478         * bmalloc/Deallocator.cpp:
1479         (bmalloc::Deallocator::processObjectLog):
1480         (bmalloc::Deallocator::deallocateSlowCase):
1481         * bmalloc/Deallocator.h:
1482         (bmalloc::Deallocator::deallocateFastCase):
1483         * bmalloc/FixedVector.h:
1484         (bmalloc::Capacity>::operator):
1485         (bmalloc::Capacity>::push):
1486         (bmalloc::Capacity>::pop):
1487         (bmalloc::Capacity>::shrink):
1488         * bmalloc/Heap.cpp:
1489         (bmalloc::Heap::allocateLarge):
1490         * bmalloc/LargeChunk.h:
1491         (bmalloc::LargeChunk::get):
1492         (bmalloc::LargeChunk::endTag):
1493         * bmalloc/Line.h:
1494         (bmalloc::Line<Traits>::concurrentRef):
1495         (bmalloc::Line<Traits>::deref):
1496         * bmalloc/MediumAllocator.h:
1497         (bmalloc::MediumAllocator::allocate):
1498         * bmalloc/ObjectType.h:
1499         (bmalloc::isSmall):
1500         * bmalloc/Page.h:
1501         (bmalloc::Page<Traits>::ref):
1502         (bmalloc::Page<Traits>::deref):
1503         * bmalloc/PerThread.h:
1504         (bmalloc::PerThread<T>::getSlowCase):
1505         * bmalloc/SegregatedFreeList.cpp:
1506         (bmalloc::SegregatedFreeList::SegregatedFreeList):
1507         (bmalloc::SegregatedFreeList::insert):
1508         * bmalloc/SmallAllocator.h:
1509         (bmalloc::SmallAllocator::allocate):
1510         (bmalloc::SmallAllocator::refill):
1511         * bmalloc/Syscall.h:
1512         * bmalloc/VMAllocate.h:
1513         (bmalloc::vmValidate):
1514         (bmalloc::vmAllocate):
1515         (bmalloc::vmDeallocatePhysicalPagesSloppy):
1516         * bmalloc/Vector.h:
1517         (bmalloc::Vector<T>::operator):
1518         (bmalloc::Vector<T>::pop):
1519         (bmalloc::Vector<T>::shrink):
1520         * bmalloc/XLargeChunk.h:
1521         (bmalloc::XLargeChunk::range):
1522         (bmalloc::XLargeChunk::size):
1523
1524 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
1525
1526         Removed an unused file.
1527
1528         Unreviewed.
1529
1530         * bmalloc/AsyncTask.cpp: Removed.
1531
1532 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1533
1534         Build bmalloc on Mac
1535         https://bugs.webkit.org/show_bug.cgi?id=131333
1536
1537         Reviewed by Mark Rowe.
1538
1539         * Makefile: Added. For make clients.
1540
1541         These files are required for building any project in WebKit. I copied
1542         them from WTF:
1543         * Configurations: Added.
1544         * Configurations/Base.xcconfig: Added.
1545         * Configurations/DebugRelease.xcconfig: Added.
1546         * Configurations/bmalloc.xcconfig: Added.
1547         * Configurations/iOS.xcconfig: Added.
1548         * Configurations/mbmalloc.xcconfig: Added.
1549
1550         * bmalloc.xcodeproj/project.pbxproj: I removed per-project-file stuff
1551         from here because everything is in .xcconfig files now.
1552
1553         I had to fix a bunch of minor warnings, since they're enabled in our
1554         .xcconfig files:
1555
1556         * bmalloc/AsyncTask.h:
1557         (bmalloc::Function>::AsyncTask):
1558         * bmalloc/BAssert.h:
1559         * bmalloc/BoundaryTagInlines.h:
1560         (bmalloc::validate):
1561         * bmalloc/Heap.cpp:
1562         (bmalloc::Heap::Heap):
1563         (bmalloc::Heap::allocateLarge):
1564         (bmalloc::Heap::deallocateLarge):
1565         * bmalloc/Mutex.h:
1566         (bmalloc::Mutex::Mutex): Deleted.
1567         * bmalloc/VMAllocate.h:
1568         (bmalloc::vmValidate):
1569         * bmalloc/mbmalloc.cpp:
1570
1571 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1572
1573         bmalloc: Fixed a leak in the per-thread cache
1574         https://bugs.webkit.org/show_bug.cgi?id=131330
1575
1576         Reviewed by Andreas Kling.
1577
1578         Remember to deallocate our line caches upon thread exit.
1579
1580         * bmalloc/Deallocator.cpp:
1581         (bmalloc::Deallocator::~Deallocator):
1582
1583 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1584
1585         bmalloc: rolled out the tryLock experiment
1586         https://bugs.webkit.org/show_bug.cgi?id=131328
1587
1588         Reviewed by Andreas Kling.
1589
1590         It wasn't a speedup.
1591
1592         * bmalloc.xcodeproj/project.pbxproj:
1593         * bmalloc/Allocator.cpp:
1594         (bmalloc::Allocator::processSmallAllocatorLog):
1595         (bmalloc::Allocator::processMediumAllocatorLog):
1596         * bmalloc/Deallocator.cpp:
1597         (bmalloc::Deallocator::processObjectLog):
1598         (bmalloc::Deallocator::deallocateSlowCase):
1599         (bmalloc::Deallocator::deallocateSmallLine):
1600         (bmalloc::Deallocator::deallocateMediumLine):
1601         * bmalloc/Deallocator.h:
1602         (bmalloc::Deallocator::deallocateFastCase):
1603         * bmalloc/Heap.h:
1604         (bmalloc::Heap::deallocateSmallLine):
1605         (bmalloc::Heap::deallocateMediumLine):
1606         * bmalloc/Line.h:
1607         (bmalloc::Line<Traits>::deref):
1608         * bmalloc/Page.h:
1609         (bmalloc::Page<Traits>::deref):
1610
1611 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1612
1613         bmalloc
1614         https://bugs.webkit.org/show_bug.cgi?id=131170
1615
1616         Reviewed by Andreas Kling.
1617
1618         Initial commit.
1619
1620         * bmalloc: Added.
1621         * bmalloc.xcodeproj: Added.
1622         * bmalloc.xcodeproj/project.pbxproj: Added.
1623         * bmalloc/Algorithm.h: Added.
1624         (bmalloc::max):
1625         (bmalloc::min):
1626         (bmalloc::mask):
1627         (bmalloc::test):
1628         (bmalloc::roundUpToMultipleOf):
1629         (bmalloc::roundDownToMultipleOf):
1630         (bmalloc::sizeOf):
1631         (bmalloc::bitCount):
1632         (bmalloc::isPowerOfTwo):
1633         * bmalloc/Allocator.cpp: Added.
1634         (bmalloc::Allocator::Allocator):
1635         (bmalloc::Allocator::~Allocator):
1636         (bmalloc::Allocator::log):
1637         (bmalloc::Allocator::processSmallAllocatorLog):
1638         (bmalloc::Allocator::processMediumAllocatorLog):
1639         (bmalloc::Allocator::allocateLarge):
1640         (bmalloc::Allocator::allocateXLarge):
1641         (bmalloc::Allocator::allocateMedium):
1642         (bmalloc::Allocator::allocateSlowCase):
1643         * bmalloc/Allocator.h: Added.
1644         (bmalloc::Allocator::smallAllocatorFor):
1645         (bmalloc::Allocator::allocateFastCase):
1646         (bmalloc::Allocator::allocate):
1647         * bmalloc/AsyncTask.cpp: Added.
1648         (bmalloc::AsyncTask<Function>::runSlowCase):
1649         (bmalloc::AsyncTask<Function>::pthreadEntryPoint):
1650         (bmalloc::AsyncTask<Function>::entryPoint):
1651         * bmalloc/AsyncTask.h: Added.
1652         (bmalloc::Function>::AsyncTask):
1653         (bmalloc::Function>::join):
1654         (bmalloc::Function>::run):
1655         (bmalloc::Function>::runSlowCase):
1656         (bmalloc::Function>::pthreadEntryPoint):
1657         (bmalloc::Function>::entryPoint):
1658         * bmalloc/BAssert.h: Added.
1659         * bmalloc/BeginTag.h: Added.
1660         (bmalloc::BeginTag::isInFreeList):
1661         * bmalloc/BoundaryTag.h: Added.
1662         (bmalloc::BoundaryTag::isXLarge):
1663         (bmalloc::BoundaryTag::setXLarge):
1664         (bmalloc::BoundaryTag::isFree):
1665         (bmalloc::BoundaryTag::setFree):
1666         (bmalloc::BoundaryTag::isEnd):
1667         (bmalloc::BoundaryTag::setEnd):
1668         (bmalloc::BoundaryTag::hasPhysicalPages):
1669         (bmalloc::BoundaryTag::setHasPhysicalPages):
1670         (bmalloc::BoundaryTag::isNull):
1671         (bmalloc::BoundaryTag::clear):
1672         (bmalloc::BoundaryTag::size):
1673         (bmalloc::BoundaryTag::setSize):
1674         (bmalloc::BoundaryTag::prev):
1675         (bmalloc::BoundaryTag::next):
1676         * bmalloc/BoundaryTagInlines.h: Added.
1677         (bmalloc::validate):
1678         (bmalloc::validatePrev):
1679         (bmalloc::validateNext):
1680         (bmalloc::BoundaryTag::init):
1681         (bmalloc::BoundaryTag::mergeLargeLeft):
1682         (bmalloc::BoundaryTag::mergeLargeRight):
1683         (bmalloc::BoundaryTag::mergeLarge):
1684         (bmalloc::BoundaryTag::deallocate):
1685         (bmalloc::BoundaryTag::splitLarge):
1686         (bmalloc::BoundaryTag::allocate):
1687         * bmalloc/Cache.cpp: Added.
1688         (bmalloc::Cache::operator new):
1689         (bmalloc::Cache::operator delete):
1690         (bmalloc::Cache::Cache):
1691         (bmalloc::Cache::allocateSlowCase):
1692         (bmalloc::Cache::allocateSlowCaseNullCache):
1693         (bmalloc::Cache::deallocateSlowCase):
1694         (bmalloc::Cache::deallocateSlowCaseNullCache):
1695         * bmalloc/Cache.h: Added.
1696         (bmalloc::Cache::allocator):
1697         (bmalloc::Cache::deallocator):
1698         (bmalloc::Cache::allocateFastCase):
1699         (bmalloc::Cache::deallocateFastCase):
1700         (bmalloc::Cache::allocate):
1701         (bmalloc::Cache::deallocate):
1702         * bmalloc/Chunk.h: Added.
1703         (bmalloc::Chunk::begin):
1704         (bmalloc::Chunk::end):
1705         (bmalloc::Chunk::lines):
1706         (bmalloc::Chunk::pages):
1707         * bmalloc/Deallocator.cpp: Added.
1708         (bmalloc::Deallocator::Deallocator):
1709         (bmalloc::Deallocator::~Deallocator):
1710         (bmalloc::Deallocator::deallocateLarge):
1711         (bmalloc::Deallocator::deallocateXLarge):
1712         (bmalloc::Deallocator::processObjectLog):
1713         (bmalloc::Deallocator::deallocateSlowCase):
1714         (bmalloc::Deallocator::deallocateSmallLine):
1715         (bmalloc::Deallocator::allocateSmallLine):
1716         (bmalloc::Deallocator::deallocateMediumLine):
1717         (bmalloc::Deallocator::allocateMediumLine):
1718         * bmalloc/Deallocator.h: Added.
1719         (bmalloc::Deallocator::deallocateFastCase):
1720         (bmalloc::Deallocator::deallocate):
1721         * bmalloc/EndTag.h: Added.
1722         (bmalloc::EndTag::operator=):
1723         * bmalloc/FixedVector.h: Added.
1724         (bmalloc::FixedVector::begin):
1725         (bmalloc::FixedVector::end):
1726         (bmalloc::FixedVector::size):
1727         (bmalloc::FixedVector::capacity):
1728         (bmalloc::FixedVector::clear):
1729         (bmalloc::FixedVector::isEmpty):
1730         (bmalloc::Capacity>::FixedVector):
1731         (bmalloc::Capacity>::operator):
1732         (bmalloc::Capacity>::push):
1733         (bmalloc::Capacity>::pop):
1734         (bmalloc::Capacity>::shrink):
1735         * bmalloc/Heap.cpp: Added.
1736         (bmalloc::sleep):
1737         (bmalloc::Heap::Heap):
1738         (bmalloc::Heap::concurrentScavenge):
1739         (bmalloc::Heap::scavengeSmallPages):
1740         (bmalloc::Heap::scavengeMediumPages):
1741         (bmalloc::Heap::scavengeLargeRanges):
1742         (bmalloc::Heap::allocateSmallLineSlowCase):
1743         (bmalloc::Heap::allocateMediumLineSlowCase):
1744         (bmalloc::Heap::allocateXLarge):
1745         (bmalloc::Heap::deallocateXLarge):
1746         (bmalloc::Heap::allocateLarge):
1747         (bmalloc::Heap::deallocateLarge):
1748         * bmalloc/Heap.h: Added.
1749         (bmalloc::Heap::deallocateSmallLine):
1750         (bmalloc::Heap::allocateSmallLine):
1751         (bmalloc::Heap::deallocateMediumLine):
1752         (bmalloc::Heap::allocateMediumLine):
1753         * bmalloc/Inline.h: Added.
1754         * bmalloc/LargeChunk.h: Added.
1755         (bmalloc::LargeChunk::begin):
1756         (bmalloc::LargeChunk::end):
1757         (bmalloc::LargeChunk::create):
1758         (bmalloc::LargeChunk::get):
1759         (bmalloc::LargeChunk::beginTag):
1760         (bmalloc::LargeChunk::endTag):
1761         * bmalloc/Line.h: Added.
1762         (bmalloc::Line<Traits>::begin):
1763         (bmalloc::Line<Traits>::end):
1764         (bmalloc::Line<Traits>::concurrentRef):
1765         (bmalloc::Line<Traits>::deref):
1766         * bmalloc/MediumAllocator.h: Added.
1767         (bmalloc::MediumAllocator::isNull):
1768         (bmalloc::MediumAllocator::MediumAllocator):
1769         (bmalloc::MediumAllocator::line):
1770         (bmalloc::MediumAllocator::allocate):
1771         (bmalloc::MediumAllocator::derefCount):
1772         (bmalloc::MediumAllocator::refill):
1773         * bmalloc/MediumChunk.h: Added.
1774         * bmalloc/MediumLine.h: Added.
1775         * bmalloc/MediumPage.h: Added.
1776         * bmalloc/MediumTraits.h: Added.
1777         * bmalloc/Mutex.cpp: Added.
1778         (bmalloc::Mutex::lockSlowCase):
1779         * bmalloc/Mutex.h: Added.
1780         (bmalloc::Mutex::Mutex):
1781         (bmalloc::Mutex::try_lock):
1782         (bmalloc::Mutex::lock):
1783         (bmalloc::Mutex::unlock):
1784         * bmalloc/ObjectType.cpp: Added.
1785         (bmalloc::objectType):
1786         * bmalloc/ObjectType.h: Added.
1787         (bmalloc::isSmallOrMedium):
1788         (bmalloc::isSmall):
1789         * bmalloc/Page.h: Added.
1790         (bmalloc::Page<Traits>::ref):
1791         (bmalloc::Page<Traits>::deref):
1792         (bmalloc::Page<Traits>::refCount):
1793         * bmalloc/PerProcess.h: Added.
1794         (bmalloc::PerProcess::mutex):
1795         (bmalloc::PerProcess<T>::getFastCase):
1796         (bmalloc::PerProcess<T>::get):
1797         (bmalloc::PerProcess<T>::getSlowCase):
1798         * bmalloc/PerThread.h: Added.
1799         (bmalloc::PerThreadStorage<Cache>::get):
1800         (bmalloc::PerThreadStorage<Cache>::init):
1801         (bmalloc::PerThreadStorage::get):
1802         (bmalloc::PerThreadStorage::init):
1803         (bmalloc::PerThread<T>::getFastCase):
1804         (bmalloc::PerThread<T>::get):
1805         (bmalloc::PerThread<T>::destructor):
1806         (bmalloc::PerThread<T>::getSlowCase):
1807         * bmalloc/Range.h: Added.
1808         (bmalloc::Range::Range):
1809         (bmalloc::Range::begin):
1810         (bmalloc::Range::end):
1811         (bmalloc::Range::size):
1812         (bmalloc::Range::operator!):
1813         (bmalloc::Range::operator<):
1814         * bmalloc/SegregatedFreeList.cpp: Added.
1815         (bmalloc::SegregatedFreeList::SegregatedFreeList):
1816         (bmalloc::SegregatedFreeList::insert):
1817         (bmalloc::SegregatedFreeList::takeGreedy):
1818         (bmalloc::SegregatedFreeList::take):
1819         * bmalloc/SegregatedFreeList.h: Added.
1820         * bmalloc/Sizes.h: Added.
1821         * bmalloc/SmallAllocator.h: Added.
1822         (bmalloc::SmallAllocator::isNull):
1823         (bmalloc::SmallAllocator::canAllocate):
1824         (bmalloc::SmallAllocator::SmallAllocator):
1825         (bmalloc::SmallAllocator::line):
1826         (bmalloc::SmallAllocator::allocate):
1827         (bmalloc::SmallAllocator::objectCount):
1828         (bmalloc::SmallAllocator::derefCount):
1829         (bmalloc::SmallAllocator::refill):
1830         * bmalloc/SmallChunk.h: Added.
1831         * bmalloc/SmallLine.h: Added.
1832         * bmalloc/SmallPage.h: Added.
1833         * bmalloc/SmallTraits.h: Added.
1834         * bmalloc/Syscall.h: Added.
1835         * bmalloc/VMAllocate.h: Added.
1836         (bmalloc::vmSize):
1837         (bmalloc::vmValidate):
1838         (bmalloc::vmAllocate):
1839         (bmalloc::vmDeallocate):
1840         (bmalloc::vmDeallocatePhysicalPages):
1841         (bmalloc::vmAllocatePhysicalPages):
1842         (bmalloc::vmDeallocatePhysicalPagesSloppy):
1843         (bmalloc::vmAllocatePhysicalPagesSloppy):
1844         * bmalloc/VMHeap.cpp: Added.
1845         (bmalloc::VMHeap::VMHeap):
1846         (bmalloc::VMHeap::allocateSmallChunk):
1847         (bmalloc::VMHeap::allocateMediumChunk):
1848         (bmalloc::VMHeap::allocateLargeChunk):
1849         * bmalloc/VMHeap.h: Added.
1850         (bmalloc::VMHeap::allocateSmallPage):
1851         (bmalloc::VMHeap::allocateMediumPage):
1852         (bmalloc::VMHeap::allocateLargeRange):
1853         (bmalloc::VMHeap::deallocateSmallPage):
1854         (bmalloc::VMHeap::deallocateMediumPage):
1855         (bmalloc::VMHeap::deallocateLargeRange):
1856         * bmalloc/Vector.h: Added.
1857         (bmalloc::Vector::begin):
1858         (bmalloc::Vector::end):
1859         (bmalloc::Vector::size):
1860         (bmalloc::Vector::capacity):
1861         (bmalloc::Vector::last):
1862         (bmalloc::Vector::pop):
1863         (bmalloc::Vector<T>::Vector):
1864         (bmalloc::Vector<T>::~Vector):
1865         (bmalloc::Vector<T>::operator):
1866         (bmalloc::Vector<T>::push):
1867         (bmalloc::Vector<T>::pop):
1868         (bmalloc::Vector<T>::shrink):
1869         (bmalloc::Vector<T>::reallocateBuffer):
1870         (bmalloc::Vector<T>::shrinkCapacity):
1871         (bmalloc::Vector<T>::growCapacity):
1872         * bmalloc/XLargeChunk.h: Added.
1873         (bmalloc::XLargeChunk::get):
1874         (bmalloc::XLargeChunk::begin):
1875         (bmalloc::XLargeChunk::XLargeChunk):
1876         (bmalloc::XLargeChunk::create):
1877         (bmalloc::XLargeChunk::destroy):
1878         (bmalloc::XLargeChunk::range):
1879         (bmalloc::XLargeChunk::size):
1880         * bmalloc/bmalloc.h: Added.
1881         (bmalloc::api::malloc):
1882         (bmalloc::api::free):
1883         (bmalloc::api::realloc):
1884         * bmalloc/mbmalloc.cpp: Added.
1885