bmalloc: a little bit of cleanup
[WebKit-https.git] / Source / bmalloc / ChangeLog
1 2015-01-20  Geoffrey Garen  <ggaren@apple.com>
2
3         bmalloc: a little bit of cleanup
4         https://bugs.webkit.org/show_bug.cgi?id=140687
5
6         Reviewed by Anders Carlsson.
7
8         * bmalloc/Algorithm.h:
9         (bmalloc::isPowerOfTwo): Added a check for 0, since 0 would break a lot
10         of code.
11
12         * bmalloc/BoundaryTag.h:
13         * bmalloc/BoundaryTagInlines.h:
14         (bmalloc::BoundaryTag::mergeLeft):
15         (bmalloc::BoundaryTag::mergeRight):
16         (bmalloc::BoundaryTag::merge):
17         (bmalloc::BoundaryTag::deallocate):
18         (bmalloc::BoundaryTag::split):
19         (bmalloc::BoundaryTag::allocate):
20         (bmalloc::BoundaryTag::mergeLargeLeft): Deleted.
21         (bmalloc::BoundaryTag::mergeLargeRight): Deleted.
22         (bmalloc::BoundaryTag::mergeLarge): Deleted.
23         (bmalloc::BoundaryTag::splitLarge): Deleted. Removed the word "Large"
24         from all these functions, since boundary tags always pertain to large
25         objects, and putting the word "Large" everywhere wasn't helping to
26         explain that.
27
28         * bmalloc/Heap.cpp:
29         (bmalloc::Heap::allocateXLarge):
30         (bmalloc::Heap::findXLarge):
31         (bmalloc::Heap::deallocateXLarge):
32         * bmalloc/Heap.h:
33         * bmalloc/VMHeap.h:
34         (bmalloc::VMHeap::allocateXLarge): Deleted.
35         (bmalloc::VMHeap::findXLarge): Deleted.
36         (bmalloc::VMHeap::deallocateXLarge): Deleted. Moved XLarge allocation
37         from VMHeap to Heap. Since the purpose of the VMHeap is to cache VM
38         ranges, and the VMHeap never caches any XLarge ranges, it doesn't
39         really make sense for the VMHeap to be involved.
40
41 2015-01-16  Geoffrey Garen  <ggaren@apple.com>
42
43         bmalloc: refactored XLarge allocation for better alignment
44         https://bugs.webkit.org/show_bug.cgi?id=140582
45
46         Reviewed by Andreas Kling.
47
48         XLarge objects used to be Large objects with an extra bit of metadata
49         that said "actually, I'm not large -- I'm extra large".
50
51         The metadata header in an XLarge allocation made it impossible for the
52         XLarge object to honor a very large alignment request.
53
54         The solution is to stop using a metadata header for XLarge objects, and
55         instead to store explicit metadata on the side.
56
57         This is a bit less astonishing, which is also nice.
58
59         Finding XLarge metadata is now a linear search. That's probably OK, since
60         it was always so in TCMalloc, and the usual number of XLarge allocations
61         in a process is 0.
62
63         This design makes it possible for the heap to cache XLarge allocations
64         with and/or without physical pages. I haven't actually done that yet
65         because the tradeoffs are subtle, so I don't want to do anything without
66         a motivating test case.
67
68         * bmalloc.xcodeproj/project.pbxproj:
69         * bmalloc/Allocator.cpp:
70         (bmalloc::Allocator::reallocate): Removed the concept of an XLargeChunk,
71         since an XLarge allocation is now just a naked buffer without a header.
72
73         (bmalloc::Allocator::allocateXLarge): Added an explicit qualifier for
74         XLarge alignment, since XLargeChunk won't give this to us implicitly
75         anymore.
76
77         * bmalloc/BoundaryTag.h:
78         (bmalloc::BoundaryTag::setRange):
79         (bmalloc::BoundaryTag::isXLarge): Deleted.
80         (bmalloc::BoundaryTag::setXLarge): Deleted.
81         * bmalloc/BoundaryTagInlines.h:
82         (bmalloc::validate):
83         (bmalloc::BoundaryTag::deallocate): Removed the XLarge hacks from Large allocations.
84
85         * bmalloc/Deallocator.cpp:
86         (bmalloc::Deallocator::deallocateXLarge):
87         (bmalloc::Deallocator::deallocateSlowCase):
88         * bmalloc/Heap.cpp:
89         (bmalloc::Heap::findXLarge):
90         (bmalloc::Heap::allocateXLarge):
91         (bmalloc::Heap::deallocateXLarge):
92         * bmalloc/Heap.h: Updated for interface changes.
93
94         * bmalloc/ObjectType.cpp:
95         (bmalloc::objectType):
96         * bmalloc/ObjectType.h:
97         (bmalloc::isXLarge): We can now tell if a pointer is XLarge just by
98         examining its bit pattern -- just like we do for other kinds of
99         allocations -- which is nice.
100
101         * bmalloc/Sizes.h:
102         * bmalloc/VMHeap.h:
103         (bmalloc::VMHeap::allocateXLarge):
104         (bmalloc::VMHeap::findXLarge):
105         (bmalloc::VMHeap::deallocateXLarge): Keep an explicit vector of metadata
106         for XLarge allocations.
107
108         * bmalloc/XLargeChunk.h: Removed.
109
110 2015-01-16  Geoffrey Garen  <ggaren@apple.com>
111
112         bmalloc: added some infrastructure for aligned allocation
113         https://bugs.webkit.org/show_bug.cgi?id=140572
114
115         Reviewed by Andreas Kling.
116
117         * bmalloc/Algorithm.h:
118         (bmalloc::isPowerOfTwo):
119         (bmalloc::roundUpToMultipleOf):
120         (bmalloc::roundDownToMultipleOf): Refactored some duplicate code to use our
121         isPowerOfTwo helper function.
122
123         * bmalloc/Allocator.cpp:
124         (bmalloc::Allocator::allocate):
125         * bmalloc/Allocator.h: Stubbed out an implementation of aligned allocation.
126         Doesn't do anything yet, but does correctly forward to system malloc
127         when bmalloc is disabled.
128
129         * bmalloc/Cache.cpp:
130         (bmalloc::Cache::allocateSlowCaseNullCache):
131         * bmalloc/Cache.h:
132         (bmalloc::Cache::allocate):
133         * bmalloc/bmalloc.h:
134         (bmalloc::api::memalign):
135         * bmalloc/mbmalloc.cpp: Stubbed out an API for aligned allocation.
136
137 2015-01-13  Geoffrey Garen  <ggaren@apple.com>
138
139         Consider alignment when allocating from a SegregatedFreeList
140         https://bugs.webkit.org/show_bug.cgi?id=140408
141
142         Reviewed by Sam Weinig.
143
144         In preparation for supporting aligned allocation.
145
146         No performance change.
147
148         Since this is just one extra branch in an already expensive function,
149         I decided not to duplicate the function just to avoid the branch in
150         the un-aligned case.
151
152         * bmalloc/SegregatedFreeList.cpp:
153         (bmalloc::SegregatedFreeList::take):
154         * bmalloc/SegregatedFreeList.h:
155
156 2015-01-13  Geoffrey Garen  <ggaren@apple.com>
157
158         Renamed minimum to size in SegregatedFreeList
159         https://bugs.webkit.org/show_bug.cgi?id=140406
160
161         Reviewed by Sam Weinig.
162
163         In preparation for supporting aligned allocation.
164
165         * bmalloc/SegregatedFreeList.cpp:
166         (bmalloc::SegregatedFreeList::takeGreedy):
167         (bmalloc::SegregatedFreeList::take): Every size passed to malloc is
168         really just a minimum. Let's not imply that this value is special.
169
170 2015-01-11  Dan Bernstein  <mitz@apple.com>
171
172         Geoff is organized, but he is not an organization.
173
174         Rubber-stamped by Anders Carlsson.
175
176         * bmalloc.xcodeproj/project.pbxproj: Removed the ORGANIZATIONNAME project attribute.
177
178 2015-01-07  Geoffrey Garen  <ggaren@apple.com>
179
180         Make bmalloc work with ASan
181         https://bugs.webkit.org/show_bug.cgi?id=140194
182
183         Reviewed by Mark Lam.
184
185         * bmalloc/BPlatform.h: Added a way to detect Darwin OSes, since we need
186         an OS-specific API to test for loaded runtime libraries.
187
188         * bmalloc/Environment.cpp:
189         (bmalloc::isASanEnabled):
190         (bmalloc::Environment::computeIsBmallocEnabled): Disabled bmalloc if
191         ASan is enabled, since system malloc has the Asan hooks we need.
192
193         You could check for the ASan compile-time flag instead, but doing this
194         check at runtime prepares bmalloc for a world where it is a dynamic
195         library that might be loaded into projects it did not compile with.
196
197 2015-01-05  Geoffrey Garen  <ggaren@apple.com>
198
199         Fix up bmalloc's PerThread for use on Linux
200         https://bugs.webkit.org/show_bug.cgi?id=139804
201
202         Reviewed by Anders Carlsson.
203
204         The previous implementation was a bit slow.
205
206         * bmalloc/PerThread.h:
207         (bmalloc::PerThreadStorage<Cache>::get):
208         (bmalloc::PerThreadStorage::get):
209         (bmalloc::PerThreadStorage::init): Added a catch-all cross-platform Unix
210         way to do fast per-thread access without taking a lock every time. This
211         probably works on all the platforms we care about, and it matches other
212         techniques we use elsewhere in WebKit.
213
214         (bmalloc::PerThread<T>::getFastCase): Removed the conditional from
215         this class because PerThreadStorage now encapsulates everything that
216         needs to be conditional.
217
218         (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Deleted.
219
220 2014-12-26  Dan Bernstein  <mitz@apple.com>
221
222         <rdar://problem/19348208> REGRESSION (r177027): iOS builds use the wrong toolchain
223         https://bugs.webkit.org/show_bug.cgi?id=139950
224
225         Reviewed by David Kilzer.
226
227         * Configurations/Base.xcconfig: Only define TOOLCHAINS when building for OS X, doing so
228         in a manner that works with Xcode 5.1.1.
229
230 2014-12-15  Geoffrey Garen  <ggaren@apple.com>
231
232         Safari crashes when you set Malloc environment variables
233         https://bugs.webkit.org/show_bug.cgi?id=139656
234
235         Reviewed by Michael Saboff.
236
237         I forgot to cover the realloc() case. Whoops. (OoPS?)
238
239         This time around, I ran the full MallocBench test suite in Malloc=1
240         mode, and it passed.
241
242         * bmalloc/Allocator.cpp:
243         (bmalloc::Allocator::reallocate):
244         * bmalloc/Allocator.h: Pushed realloc() logic down into the allocator.
245         It needs to be down there so that we can do the short-circuiting check
246         for whether bmalloc is enabled first.
247
248         Also added the check.
249
250         * bmalloc/Cache.cpp:
251         (bmalloc::Cache::scavenge):
252         (bmalloc::Cache::Cache):
253         (bmalloc::Cache::reallocateSlowCaseNullCache):
254         * bmalloc/Cache.h:
255         (bmalloc::Cache::deallocator):
256         (bmalloc::Cache::reallocate): Ditto.
257
258         * bmalloc/bmalloc.h:
259         (bmalloc::api::free):
260         (bmalloc::api::realloc): Ditto.
261
262         (bmalloc::api::scavenge): Pushed this down into Cache to match the
263         surrounding functions.
264
265 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
266
267         bmalloc should support system memory analysis tools (part 2)
268         https://bugs.webkit.org/show_bug.cgi?id=139565
269
270         Reviewed by Mark Lam.
271
272         This patch actually queries the environment to see if memory analysis
273         tools have been enabled.
274
275         * bmalloc/Deallocator.cpp:
276         (bmalloc::Deallocator::scavenge): Don't process the object log if
277         we've disabled bmalloc because it will be full of invalid nullptrs.
278
279         * bmalloc/Environment.cpp:
280         (bmalloc::isMallocEnvironmentVariableSet): Test for the list of known
281         Malloc debugging flags. I also added a plain "Malloc" catch-all for
282         when you want to disable bmalloc without enabling any kind of funny
283         business.
284
285         It would be slightly nicer just to iterate the list of environment
286         variables and strstr them, but getenv is the more portable option,
287         and performance here doesn't really matter.
288
289         (bmalloc::isLibgmallocEnabled): Test for the libgmalloc insertion
290         environment variable.
291
292         (bmalloc::Environment::computeIsBmallocEnabled):
293
294 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
295
296         Try to fix the iOS simulator build.
297
298         #include the declaration of malloc / free.
299
300         * bmalloc/Allocator.cpp:
301         * bmalloc/Deallocator.cpp:
302
303 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
304
305         Try to fix the build.
306
307         * bmalloc.xcodeproj/project.pbxproj: Marked a header exported.
308
309 2014-12-11  Geoffrey Garen  <ggaren@apple.com>
310
311         bmalloc should support system memory analysis tools (part 1)
312         https://bugs.webkit.org/show_bug.cgi?id=139559
313
314         Reviewed by Mark Lam.
315
316         This patch adds the hooks to disable bmalloc at runtime if certain
317         environment variables are set, but doesn't actually read from the
318         environment yet.
319
320         No performance change.
321
322         * bmalloc.xcodeproj/project.pbxproj: Added the Environment class, which
323         we'll use to read environment variables and see if memory analysis tools
324         have been enabled.
325
326         * bmalloc/Allocator.cpp:
327         (bmalloc::Allocator::Allocator):
328         (bmalloc::Allocator::allocateSlowCase): Added a hook to disable bmalloc
329         on the allocation path. We cache the setting to make the check fast.
330
331         * bmalloc/Allocator.h: Interface changes.
332
333         * bmalloc/Cache.cpp:
334         (bmalloc::Cache::Cache): Pass a heap pointer through to our allocator
335         and deallocator. This main purpose is to enable them to query the
336         environment for whether bmalloc is enabled; but this is also a slightly
337         cleaner way to guarantee to them that the Heap has been pre-initialized.
338
339         * bmalloc/Deallocator.cpp:
340         (bmalloc::Deallocator::Deallocator): If bmalloc is disable, artificially
341         fill the object log to force us to take the slow path on all deallocations.
342
343         (bmalloc::Deallocator::deallocateSlowCase): Do the disabled check.
344
345         * bmalloc/Deallocator.h: Interface changes.
346
347         * bmalloc/Environment.cpp: Added.
348         (bmalloc::Environment::Environment):
349         (bmalloc::Environment::computeIsBmallocEnabled):
350         * bmalloc/Environment.h: Added.
351         (bmalloc::Environment::isBmallocEnabled): This is the class that will
352         encapsulate looking for environment variables that turn on heap
353         analysis tools.
354
355         * bmalloc/Heap.h:
356         (bmalloc::Heap::environment):
357
358         * bmalloc/Mutex.h:
359         (bmalloc::Mutex::Mutex):
360         * bmalloc/StaticMutex.h: A little refactoring to clarify these comments,
361         since I got super confused about them while writing this patch.
362
363         * bmalloc/VMHeap.cpp: Fixed an #include.
364
365 2014-12-09  David Kilzer  <ddkilzer@apple.com>
366
367         Switch from using PLATFORM_NAME to SDK selectors in ANGLE, bmalloc, gtest, JavaScriptCore, WTF
368         <http://webkit.org/b/139212>
369
370         Reviewed by Joseph Pecoraro.
371
372         * Configurations/Base.xcconfig:
373         - Only set GCC_ENABLE_OBJC_GC, GCC_MODEL_TUNING and TOOLCHAINS
374           on OS X.
375         * Configurations/DebugRelease.xcconfig:
376         - Only set MACOSX_DEPLOYMENT_TARGET and SDKROOT on OS X.
377
378 2014-11-07  Geoffrey Garen  <ggaren@apple.com>
379
380         bmalloc uses 8X more virtual memory than necessary
381         https://bugs.webkit.org/show_bug.cgi?id=138495
382
383         Reviewed by Mark Lam.
384
385         iOS has a per-process virtual memory cap around 1GB, so there's some
386         value to not going totally ham with virtual memory.
387
388         We currently use about 8X the necessary amount:
389             - 2X to align our VM allocation
390             - 4X to reserve small / medium / (2) large chunk VM ranges per superchunk
391
392         We can cut that down:
393             - Return the unaligned portion of our VM allocation (-2X)
394             - Use all the chunks in a superchunk, instead of allocating one
395               chunk per superchunk (-4X)
396
397         * bmalloc/Algorithm.h:
398         (bmalloc::roundUpToMultipleOf): Added a non-constant version of this
399         function so we can call it with getpagesize() at runtime.
400
401         * bmalloc/Chunk.h:
402         * bmalloc/LargeChunk.h:
403         (bmalloc::LargeChunk::create): Deleted. Instead of each chunk allocating
404         its own VM, VMHeap allocates the superchunk and all the chunks in it at a time.
405
406         * bmalloc/VMAllocate.h:
407         (bmalloc::vmValidate):
408         (bmalloc::vmAllocate): ASSERT that mmap succeeds to make crashes clearer
409         if it does not succeed. Allocate precisely, and give back the extra.
410
411         * bmalloc/VMHeap.cpp:
412         (bmalloc::VMHeap::allocateSuperChunk):
413         (bmalloc::VMHeap::allocateSmallChunk): Deleted.
414         (bmalloc::VMHeap::allocateMediumChunk): Deleted.
415         (bmalloc::VMHeap::allocateLargeChunk): Deleted. Use all the chunks
416         in a superchunk, instead of just one.
417
418         * bmalloc/VMHeap.h:
419         (bmalloc::VMHeap::allocateSmallPage):
420         (bmalloc::VMHeap::allocateMediumPage):
421         (bmalloc::VMHeap::allocateLargeRange):
422         * bmalloc/XLargeChunk.h:
423         (bmalloc::XLargeChunk::create): Updated to match changes above.
424
425 2014-11-01  David Kilzer  <ddkilzer@apple.com>
426
427         JavaScriptCore is missing debug info for bmalloc because libbmalloc.a is stripped
428         <https://webkit.org/b/138286>
429         <rdar://problem/18847087>
430
431         Reviewed by Dan Bernstein.
432
433         * Configurations/bmalloc.xcconfig: Set STRIP_INSTALLED_PRODUCT
434         to NO for the target that produces libbmalloc.a so that the
435         debug symbols will be linked into JavaScriptCore and end up in
436         its dSYM file.
437
438 2014-10-30  Dana Burkart  <dburkart@apple.com>
439
440         <rdar://problem/18821260> Prepare for the mysterious future
441
442         Reviewed by Lucas Forschler.
443
444         * Configurations/Base.xcconfig:
445         * Configurations/DebugRelease.xcconfig:
446
447 2014-09-24  Geoffrey Garen  <ggaren@apple.com>
448
449         bmalloc: cleaned up fast path vs slow path
450         https://bugs.webkit.org/show_bug.cgi?id=137081
451
452         Reviewed by Sam Weinig.
453
454         Might be a 1% speedup on MallocBench. Also cleans up the code a bit.
455
456         * bmalloc/Allocator.cpp:
457         (bmalloc::Allocator::Allocator): Merged the small and medium range
458         caches, just like the small and medium allocators. Ranges are abstract
459         objects that don't really care whether they hold small or medium objects,
460         so they don't need to be segregated.
461
462         (bmalloc::Allocator::scavenge): Ditto.
463
464         (bmalloc::Allocator::allocateBumpRangeSlowCase):
465         (bmalloc::Allocator::allocateBumpRange): Same thing here, except that
466         we do care a tiny bit, because we need to specify small vs medium when
467         allocating new ranges from the heap, to ensure that the heap allocates
468         from the right segment of VM.
469
470         (bmalloc::Allocator::allocateLarge):
471         (bmalloc::Allocator::allocateXLarge): NO_INLINE because this was clouding
472         up the fast path. Large allocation performance is dominated by allocation
473         logic and initialization, so inlining it doesn't help.
474
475         (bmalloc::Allocator::allocateSlowCase): Slow path got a bit cleaner since
476         it doesn't need to distinguish small vs medium objects.
477
478         (bmalloc::Allocator::allocateSmallBumpRange): Deleted.
479         (bmalloc::Allocator::allocateMediumBumpRange): Deleted.
480
481         * bmalloc/Allocator.h:
482         * bmalloc/BumpRange.h:
483
484         * bmalloc/Cache.cpp:
485         (bmalloc::Cache::allocateSlowCase): Deleted.
486         (bmalloc::Cache::deallocateSlowCase): Deleted.
487         * bmalloc/Cache.h:
488         (bmalloc::Cache::allocate):
489         (bmalloc::Cache::deallocate):
490         (bmalloc::Cache::allocateFastCase): Deleted.
491         (bmalloc::Cache::deallocateFastCase): Deleted. Removed the Cache slow
492         paths. The downside to this change is that the fast path branches to two
493         distinct failure cases instead of one. The upside is that the slow path
494         doesn't need to re-read the segment register, which is not as cheap as a
495         normal register, and it doesn't need to do an extra level of function 
496         call. Seems to be worth it.
497
498         * bmalloc/Deallocator.h:
499         * bmalloc/Heap.cpp:
500         (bmalloc::Heap::refillSmallBumpRangeCache):
501         (bmalloc::Heap::refillMediumBumpRangeCache):
502         * bmalloc/Heap.h: Updated for interface changes.
503
504         * bmalloc/Sizes.h: The most ranges a cache will hold is the number of
505         small lines in a page / 2, since any other free lines will coalesce
506         with their neighbors.
507
508 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
509
510         Rolled out r173346.
511
512             bmalloc should honor the FastMalloc statistics API
513             https://bugs.webkit.org/show_bug.cgi?id=136592
514
515         This didn't really work. Because we allow ranges with and without
516         physical pages to merge, and we allow double-committing and
517         double-decommitting, we can't rely on commit actions to track memory
518         footprint.
519
520         * bmalloc/Heap.cpp:
521         (bmalloc::Heap::size): Deleted.
522         (bmalloc::Heap::capacity): Deleted.
523         * bmalloc/Heap.h:
524         * bmalloc/VMHeap.cpp:
525         (bmalloc::VMHeap::VMHeap):
526         (bmalloc::VMHeap::allocateSmallChunk):
527         (bmalloc::VMHeap::allocateMediumChunk):
528         (bmalloc::VMHeap::allocateLargeChunk):
529         * bmalloc/VMHeap.h:
530         (bmalloc::VMHeap::allocateSmallPage):
531         (bmalloc::VMHeap::allocateMediumPage):
532         (bmalloc::VMHeap::allocateLargeRange):
533         (bmalloc::VMHeap::deallocateSmallPage):
534         (bmalloc::VMHeap::deallocateMediumPage):
535         (bmalloc::VMHeap::deallocateLargeRange):
536         (bmalloc::VMHeap::size): Deleted.
537         (bmalloc::VMHeap::capacity): Deleted.
538         * bmalloc/bmalloc.h:
539         (bmalloc::api::heapSize): Deleted.
540         (bmalloc::api::heapCapacity): Deleted.
541
542 2014-09-23  Geoffrey Garen  <ggaren@apple.com>
543
544         bmalloc: Allocation should be more precise
545         https://bugs.webkit.org/show_bug.cgi?id=136993
546
547         Reviewed by Gavin Barraclough.
548
549         13% reduction in heap size on the MallocBench *_memory_warning benchmarks.
550
551         This patch teaches the allocator to merge adjacent free lines into a
552         single allocatable range. This allows us to shrink the size of an
553         individual line without increasing fragmentation or the rate of allocator
554         slow paths.
555
556         We'll only take more slow paths when available memory is sparse, which
557         is exactly when it's worth it. When available memory is dense, we'll
558         take fewer slow paths.
559
560         * bmalloc.xcodeproj/project.pbxproj:
561         * bmalloc/Algorithm.h:
562         (bmalloc::divideRoundingUp):
563
564         * bmalloc/Allocator.cpp:
565         (bmalloc::Allocator::Allocator): Updated for interface changes.
566
567         (bmalloc::Allocator::scavenge): Scavenge by object instead of by line.
568         Now that we merge lines, it's not convenient to scavenge by line.
569
570         (bmalloc::Allocator::allocateSmallBumpRange):
571         (bmalloc::Allocator::allocateMediumBumpRange): Allocate whole ranges
572         instead of individual lines.
573
574         (bmalloc::Allocator::allocateSlowCase):
575         (bmalloc::Allocator::allocateSmallLine): Deleted.
576         (bmalloc::Allocator::allocateMediumLine): Deleted.
577         (bmalloc::Allocator::allocateMedium): Deleted.
578         * bmalloc/Allocator.h:
579         (bmalloc::Allocator::allocateFastCase): Folded medium allocations
580         into the standard fast path with small allocations. Since a BumpAllocator
581         just allocates out of an arbitrary range, it doesn't need to distinguish
582         between small and medium lines.
583
584         * bmalloc/BumpAllocator.h:
585         (bmalloc::BumpAllocator::size):
586         (bmalloc::BumpAllocator::BumpAllocator):
587         (bmalloc::BumpAllocator::init):
588         (bmalloc::BumpAllocator::refill):
589         (bmalloc::BumpAllocator::line): Deleted. No need to track line information
590         anymore: the heap just gives us a pointer and a pre-computed number of
591         objects, and we allocate them.
592
593         * bmalloc/Deallocator.cpp:
594         (bmalloc::Deallocator::processObjectLog): Updated for interface changes.
595
596         * bmalloc/Heap.cpp:
597         (bmalloc::Heap::Heap):
598         (bmalloc::Heap::initializeLineMetadata): Pre-compute precise metadata
599         detailing where all objects will lie in memory. After we merge two lines,
600         we might allocate an object that spans from one line to the next. This
601         metadata details which bits of memory overlap in that way, and how they
602         overlap.
603
604         (bmalloc::Heap::refillSmallBumpRangeCache):
605         (bmalloc::Heap::refillMediumBumpRangeCache): Scan a whole page at a time,
606         and merge adjacent free lines into BumpRanges.
607
608         (bmalloc::Heap::allocateSmallPage):
609         (bmalloc::Heap::allocateMediumPage):
610         (bmalloc::Heap::deallocateSmallLine):
611         (bmalloc::Heap::deallocateMediumLine): Track pages rather than lines,
612         since we scan for free memory a page at a time.
613
614         (bmalloc::Heap::allocateSmallLineSlowCase): Deleted.
615         (bmalloc::Heap::allocateMediumLineSlowCase): Deleted. Folded into the
616         fast path.
617
618         * bmalloc/Heap.h:
619         (bmalloc::Heap::derefSmallLine):
620         (bmalloc::Heap::derefMediumLine):
621         (bmalloc::Heap::deallocateSmallLine): Deleted.
622         (bmalloc::Heap::allocateSmallLine): Deleted.
623         (bmalloc::Heap::deallocateMediumLine): Deleted.
624         (bmalloc::Heap::allocateMediumLine): Deleted. Updated for interface changes.
625
626         * bmalloc/Line.h:
627         (bmalloc::Line<Traits>::ref):
628         (bmalloc::Line<Traits>::deref):
629         (bmalloc::Line<Traits>::concurrentRef): Deleted. We don't pass a derefCount
630         anymore, since we only ever deref by 1 now.
631
632         * bmalloc/MediumAllocator.h:
633         (bmalloc::MediumAllocator::isNull): Deleted.
634         (bmalloc::MediumAllocator::MediumAllocator): Deleted.
635         (bmalloc::MediumAllocator::line): Deleted.
636         (bmalloc::MediumAllocator::allocate): Deleted.
637         (bmalloc::MediumAllocator::derefCount): Deleted.
638         (bmalloc::MediumAllocator::refill): Deleted.
639         (bmalloc::MediumAllocator::clear): Deleted. Deleted some code that's
640         been dead for a while, since it doesn't build anymore with this patch.
641
642         * bmalloc/Page.h:
643         (bmalloc::Page::sizeClass):
644         (bmalloc::Page::setSizeClass):
645         (bmalloc::Page::smallSizeClass): Deleted.
646         (bmalloc::Page::setSmallSizeClass): Deleted. Renamed setSmallSizeClass
647         to sizeClass, since we use it for medium sizes too.
648
649         * bmalloc/Sizes.h:
650         (bmalloc::Sizes::sizeClass):
651         (bmalloc::Sizes::objectSize): Shrank line sizes to save memory.
652
653         (bmalloc::Sizes::smallSizeClassFor): Deleted.
654         (bmalloc::Sizes::mediumSizeClassFor): Deleted.
655
656         * bmalloc/bmalloc.h:
657         (bmalloc::api::realloc): Now that we have precise objects sizes, realloc
658         can be a bit more precise. It also has to be, since we can't guarantee
659         that an object ends at the end of a line anymore.
660
661 2014-09-19  Daniel Bates  <dabates@apple.com>
662
663         Always assume internal SDK when building configuration Production
664         https://bugs.webkit.org/show_bug.cgi?id=136925
665         <rdar://problem/18362399>
666
667         Reviewed by Dan Bernstein.
668
669         * Configurations/Base.xcconfig:
670
671 2014-09-16  Geoffrey Garen  <ggaren@apple.com>
672
673         bmalloc: moved line caches from the deallocator to the allocator
674         https://bugs.webkit.org/show_bug.cgi?id=136868
675
676         Reviewed by Gavin Barraclough.
677
678         I did this mostly as a simplification, to make it easier to change the
679         allocation strategy.
680
681         No throughput change on MallocBench. Saves about 50kB.
682
683         Since the deallocator needs to lock the heap when freeing lines anyway,
684         there isn't much benefit to giving the deallocator a local cache of
685         deallocated lines.
686
687         We still give the allocator a local cache of lines because that does
688         reduce the frequency at which it needs to lock the heap in order to
689         acquire more lines.
690
691         * bmalloc/Allocator.cpp:
692         (bmalloc::Allocator::scavenge):
693         (bmalloc::Allocator::allocateSmallLine):
694         (bmalloc::Allocator::allocateMediumLine):
695         (bmalloc::Allocator::allocateMedium):
696         (bmalloc::Allocator::allocateSlowCase):
697         * bmalloc/Allocator.h:
698         * bmalloc/Deallocator.cpp:
699         (bmalloc::Deallocator::Deallocator):
700         (bmalloc::Deallocator::scavenge):
701         (bmalloc::Deallocator::processObjectLog):
702         (bmalloc::Deallocator::deallocateSmallLine): Deleted.
703         (bmalloc::Deallocator::allocateSmallLine): Deleted.
704         (bmalloc::Deallocator::deallocateMediumLine): Deleted.
705         (bmalloc::Deallocator::allocateMediumLine): Deleted.
706         * bmalloc/Deallocator.h:
707
708         * bmalloc/Sizes.h:
709         * bmalloc/VMAllocate.h: Took the opportunity to make the line cache size
710         exactly one page in size. That's about what we were shooting for anyway,
711         and it may make it easier to switch to per-page allocation in future.
712
713 2014-09-15  Geoffrey Garen  <ggaren@apple.com>
714
715         bmalloc: allocate small and medium objects using the same bump pointer class
716         https://bugs.webkit.org/show_bug.cgi?id=136843
717
718         Reviewed by Gavin Barraclough.
719
720         4% speedup on MallocBench.
721
722         Now that medium-sized objects have dedicated per-size allocators, they
723         don't need to use an arbitrary bump pointer allocator. This means that
724         every allocator knows how many objects it will allocate from the start,
725         and we don't need a post-processing step to adjust refcounts based on
726         real allocation count.
727
728         * bmalloc.xcodeproj/project.pbxproj: Renamed SmallAllocator to BumpAllocator
729         since it's used for small and medium objects now.
730
731         * bmalloc/Allocator.cpp:
732         (bmalloc::Allocator::Allocator): Updated to use new interface.
733         (bmalloc::Allocator::scavenge): To "retire" an allocator, we just need
734         to make sure that we finish allocating all the objects in it.
735
736         (bmalloc::Allocator::allocateMedium):
737         (bmalloc::Allocator::allocateSlowCase):
738         (bmalloc::Allocator::retire): Deleted.
739         (bmalloc::Allocator::processSmallAllocatorLog): Deleted.
740         (bmalloc::Allocator::processMediumAllocatorLog): Deleted.
741         * bmalloc/Allocator.h:
742         (bmalloc::Allocator::allocateFastCase): Removed abstractions and data
743         used to post-process an allocator based on how many objects it allocated.
744
745         * bmalloc/BumpAllocator.h: Copied from Source/bmalloc/bmalloc/SmallAllocator.h.
746         (bmalloc::BumpAllocator::BumpAllocator):
747         (bmalloc::BumpAllocator::init):
748         (bmalloc::BumpAllocator::line):
749         (bmalloc::BumpAllocator::validate):
750         (bmalloc::BumpAllocator::allocate):
751         (bmalloc::BumpAllocator::refill):
752         (bmalloc::BumpAllocator::clear): Updated these functions to be agnostic
753         about the kinds of lines they allocate into. In some cases, the line
754         type must be provided as a template parameter by the caller.
755
756         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
757         (bmalloc::SmallAllocator::line): Deleted.
758         (bmalloc::SmallAllocator::allocate): Deleted.
759         (bmalloc::SmallAllocator::objectCount): Deleted.
760         (bmalloc::SmallAllocator::derefCount): Deleted.
761         (bmalloc::SmallAllocator::refill): Deleted.
762         (bmalloc::SmallAllocator::clear): Deleted.
763
764         * bmalloc/ObjectType.h:
765         (bmalloc::isMedium):
766
767         * bmalloc/SmallAllocator.h:
768         (bmalloc::SmallAllocator::isNull): Deleted.
769         (bmalloc::SmallAllocator::canAllocate): Deleted.
770         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
771         (bmalloc::SmallAllocator::line): Deleted.
772         (bmalloc::SmallAllocator::allocate): Deleted.
773         (bmalloc::SmallAllocator::objectCount): Deleted.
774         (bmalloc::SmallAllocator::derefCount): Deleted.
775         (bmalloc::SmallAllocator::refill): Deleted.
776         (bmalloc::SmallAllocator::clear): Deleted.
777
778 2014-09-12  Geoffrey Garen  <ggaren@apple.com>
779
780         Fixed a goof in bmalloc Vector sizing
781         https://bugs.webkit.org/show_bug.cgi?id=136795
782
783         Reviewed by Gavin Barraclough and Sam Weinig.
784
785         We want our minimum vector to be page-sized since the OS will give us
786         a page no matter what -- but we want that many bytes, and not enough
787         bytes to store that many elements.
788
789         * bmalloc/Vector.h: Math is hard.
790
791 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
792
793         bmalloc should segregate medium-sized objects by line like it does for small-sized objects
794         https://bugs.webkit.org/show_bug.cgi?id=136693
795
796         Reviewed by Gavin Barraclough.
797
798         4% reduction in heap size on the MallocBench *_memory_warning benchmarks.
799
800         No throughput change.
801
802         We keep an array of medium allocators, just like our array of small
803         allocators.
804
805         In future, we can simplify the allocation fast path by merging the small
806         and medium allocator arrays. For now, this is the simplest change that
807         gets the win.
808
809         * bmalloc/Allocator.cpp:
810         (bmalloc::Allocator::Allocator):
811         (bmalloc::Allocator::scavenge):
812         (bmalloc::Allocator::allocateMedium):
813         * bmalloc/Allocator.h:
814         * bmalloc/Sizes.h:
815         (bmalloc::Sizes::mediumSizeClassFor):
816
817 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
818
819         Reviewed by Sam Weinig.
820
821         Renamed log => retire for clarity.
822
823         * bmalloc/Allocator.cpp:
824         (bmalloc::Allocator::scavenge):
825         (bmalloc::Allocator::retire):
826         (bmalloc::Allocator::allocateMedium):
827         (bmalloc::Allocator::allocateSlowCase):
828         (bmalloc::Allocator::log): Deleted.
829         * bmalloc/Allocator.h:
830
831 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
832
833         bmalloc: eager scavenge leaves behind a bogus allocator
834         https://bugs.webkit.org/show_bug.cgi?id=136743
835
836         Reviewed by Sam Weinig.
837
838         Be sure to clear the allocator after logging it in the eager scavenge
839         case, so that we don't later try to allocate out of the lines that we
840         have thrown away.
841
842         We didn't need to do this previously because scavenge would only happen
843         at thread exit time, after which no further allocation from the per-thread
844         cache would take place.
845
846         * bmalloc/Allocator.cpp:
847         (bmalloc::Allocator::scavenge):
848         * bmalloc/MediumAllocator.h:
849         (bmalloc::MediumAllocator::clear):
850         * bmalloc/SmallAllocator.h:
851         (bmalloc::SmallAllocator::clear):
852
853 2014-09-05  Geoffrey Garen  <ggaren@apple.com>
854
855         bmalloc should honor the FastMalloc statistics API
856         https://bugs.webkit.org/show_bug.cgi?id=136592
857
858         Reviewed by Gavin Barraclough.
859
860         We do this by tracking "size" and "capacity" in the VM heap.
861
862         The VM heap's "capacity" is all the VM we ever allocated.
863
864         The VM heap's "size" the subset of VM currently held onto by the
865         VM heap (and therefore not in use by the regular heap).
866
867         Somewhat ironically, reducing the process's memory footprint, increases
868         the size of the VM heap, since the VM heap holds the pages that are
869         purely virtual and not physical.
870
871         * bmalloc/Heap.cpp:
872         (bmalloc::Heap::size):
873         (bmalloc::Heap::capacity):
874         * bmalloc/Heap.h:
875         * bmalloc/VMHeap.cpp:
876         (bmalloc::VMHeap::VMHeap):
877         (bmalloc::VMHeap::allocateSmallChunk):
878         (bmalloc::VMHeap::allocateMediumChunk):
879         (bmalloc::VMHeap::allocateLargeChunk):
880         * bmalloc/VMHeap.h:
881         (bmalloc::VMHeap::size):
882         (bmalloc::VMHeap::capacity):
883         (bmalloc::VMHeap::allocateSmallPage):
884         (bmalloc::VMHeap::allocateMediumPage):
885         (bmalloc::VMHeap::allocateLargeRange):
886         (bmalloc::VMHeap::deallocateSmallPage):
887         (bmalloc::VMHeap::deallocateMediumPage):
888         (bmalloc::VMHeap::deallocateLargeRange):
889         * bmalloc/bmalloc.h:
890         (bmalloc::api::heapSize):
891         (bmalloc::api::heapCapacity):
892
893 2014-09-02  Geoffrey Garen  <ggaren@apple.com>
894
895         bmalloc crashes on the EWS bots (due to bad large object allocation)
896         https://bugs.webkit.org/show_bug.cgi?id=136469
897
898         Reviewed by Andreas Kling.
899
900         It's possible to convince bmalloc to perform a bad large object allocation,
901         through these steps:
902
903         (1) Insert object A into freelist F0.
904
905         (2) Split, merge and split again A's neighbors such that object B is
906         inserted into freelist F0, with boundary tag and size equal to object A,
907         but pointer not completely equal to object A. Put object B at the head of F0.
908
909         (3) Allocate some other object from F0, swapping its position in the
910         freelist with object B, such that object A is now ahead of object B.
911
912         --> Now, the next allocation for size A/B will allocate object A, which
913         has a slightly wrong idea about where the object actually begins.
914         Immediately, you'll corrupt a little memory, and over time, you'll also
915         corrupt boundary tag metadata.
916
917         The solution is to store the begin pointer in the boundary tag. Luckily,
918         this doesn't make the tag any bigger, and it's not a noticeable slowdown
919         on MallocBench.
920
921         * bmalloc/Algorithm.h:
922         (bmalloc::rightShift):
923         * bmalloc/BeginTag.h:
924         (bmalloc::BeginTag::isInFreeList): This is the bug fix. Make sure to
925         validate the start pointer when popping off the free list. Through a
926         very uncommon set of steps, it is possible to have an item in the free
927         list that is valid by all accounts except for its start pointer.
928
929         * bmalloc/BoundaryTag.h:
930         (bmalloc::BoundaryTag::compactBegin):
931         (bmalloc::BoundaryTag::setRange):
932         (bmalloc::BoundaryTag::setSize): Deleted. Record a compact version of the
933         start pointer. We don't need the whole pointer -- just the offset, in
934         largeAlignment increments, into the relevant boundary tag bucket.
935
936         * bmalloc/BoundaryTagInlines.h:
937         (bmalloc::validateNext):
938         (bmalloc::BoundaryTag::init):
939         (bmalloc::BoundaryTag::mergeLarge):
940         (bmalloc::BoundaryTag::splitLarge):
941         * bmalloc/SegregatedFreeList.cpp:
942         (bmalloc::SegregatedFreeList::insert):
943         (bmalloc::SegregatedFreeList::takeGreedy):
944         (bmalloc::SegregatedFreeList::take): Provide the whole range instead of
945         the size when establishing a boundary tag, as required by the new
946         interface.
947
948         * bmalloc/Sizes.h:
949
950 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
951
952         Fixed a bmalloc crash seen on the EWS bot
953         https://bugs.webkit.org/show_bug.cgi?id=135955
954
955         Reviewed by Andreas Kling.
956
957         * bmalloc/Syscall.h: Some CG APIs vm_copy their input buffers. If the
958         input buffer is a malloc region, that region will get marked Copy-On-Write
959         by the kernel. Calls to madvise() for COW regions fail and return EINVAL
960         on older OS X's. In 10.10, they still fail, but they do not return
961         EINVAL.
962
963         So, we can only ASSERT that our syscalls succeed starting with 10.10.
964
965 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
966
967         Fixed the bmalloc build
968         https://bugs.webkit.org/show_bug.cgi?id=135953
969
970         Reviewed by Andreas Kling.
971
972         * bmalloc.xcodeproj/project.pbxproj: Marked a few headers as private.
973         These headers are used, so they must be available outside the project.
974
975 2014-08-13  Daniel Bates  <dabates@apple.com>
976
977         Attempt to fix the build following <http://trac.webkit.org/changeset/172576>
978         (https://bugs.webkit.org/show_bug.cgi?id=135895)
979
980         Substitute PerThreadStorage<T>::initSharedKeyIfNeeded() for initSharedKeyIfNeeded() in
981         implementation of PerThread<T>::getFastCase().
982
983         * bmalloc/PerThread.h:
984         (bmalloc::PerThread<T>::getFastCase):
985
986 2014-08-13  Daniel Bates  <dabates@apple.com>
987
988         Make bmalloc::PerThread work without C++ thread local storage
989         https://bugs.webkit.org/show_bug.cgi?id=135895
990
991         Reviewed by Geoffrey Garen.
992
993         Implement support for building bmalloc without C++ thread local storage.
994
995         * bmalloc/BPlatform.h: Remove macro define BPLATFORM_IOS_SIMULATOR. Added macro function
996         BCOMPILER_SUPPORTS() and macro define BCOMPILER_SUPPORTS_CXX_THREAD_LOCAL that can be used
997         to determine whether the compiler supports C++ thread local storage.
998         * bmalloc/PerThread.h:
999         (bmalloc::PerThreadStorage::get): Modified to call pthread_getspecific() when building
1000         without C++ thread local storage.
1001         (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Added.
1002         (bmalloc::PerThreadStorage::init): Moved logic to initialize shared Pthread key from here to
1003         PerThreadStorage::initSharedKeyIfNeeded().
1004         (bmalloc::PerThread<T>::getFastCase): Modified to call PerThreadStorage::initSharedKeyIfNeeded()
1005         before querying PerThreadStorage::get() when building without C++ thread local storage so as to
1006         ensure that the shared key has been initialized.
1007         (_pthread_setspecific_direct): Deleted.
1008         (_pthread_getspecific_direct): Deleted.
1009
1010 2014-08-13  Daniel Bates  <dabates@apple.com>
1011
1012         [iOS] Make JavaScriptCore and bmalloc build with the public SDK
1013         https://bugs.webkit.org/show_bug.cgi?id=135848
1014
1015         Reviewed by Geoffrey Garen.
1016
1017         * bmalloc/BPlatform.h: Added macro BPLATFORM_IOS_SIMULATOR, which evaluates to true
1018         when building for the iOS Simulator.
1019         * bmalloc/PerThread.h: Use pthread_machdep.h code path when building for iOS Simulator
1020         using the public SDK.
1021         (_pthread_setspecific_direct): Added; only defined when building for the iOS Simulator
1022         using the public SDK.
1023         (_pthread_getspecific_direct): Added; only defined when building for the iOS Simulator
1024         using the public SDK.
1025
1026 2014-08-12  Daniel Bates  <dabates@apple.com>
1027
1028         BPLATFORM(IOS) always evaluates to false
1029         https://bugs.webkit.org/show_bug.cgi?id=135843
1030
1031         Reviewed by Geoffrey Garen.
1032
1033         Fix typo in definition of BPLATFORM() and include system header TargetConditionals.h
1034         (when building on an Apple platform) so that BPLATFORM(X) evaluates to true when
1035         building for platform X. In particular, so that BPLATFORM(IOS) evaluates to true when
1036         building for iOS.
1037
1038         As a side effect of this change, the change made in <http://trac.webkit.org/changeset/167289>
1039         will be honored and iOS will assume a VM page size of 16kB (again) instead of 4kB.
1040
1041         * bmalloc/BPlatform.h:
1042
1043 2014-08-11  Andy Estes  <aestes@apple.com>
1044
1045         [iOS] Get rid of iOS.xcconfig
1046         https://bugs.webkit.org/show_bug.cgi?id=135809
1047
1048         Reviewed by Joseph Pecoraro.
1049
1050         All iOS.xcconfig did was include AspenFamily.xcconfig, so there's no need for the indirection.
1051
1052         * Configurations/Base.xcconfig:
1053         * Configurations/iOS.xcconfig: Removed.
1054         * bmalloc.xcodeproj/project.pbxproj:
1055
1056 2014-05-01  Dan Bernstein  <mitz@apple.com>
1057
1058         Fixed production builds for the iOS Simulator.
1059         <rdar://problem/16792221>
1060
1061         * Configurations/bmalloc.xcconfig: Include INSTALL_PATH_PREFIX in
1062         PRIVATE_HEADERS_FOLDER_PATH when installing.
1063
1064 2014-04-20  Geoffrey Garen  <ggaren@apple.com>
1065
1066         bmalloc: Segregate pages by objects size
1067         https://bugs.webkit.org/show_bug.cgi?id=131909
1068
1069         Reviewed by Andreas Kling.
1070
1071         2% reduction in memory-at-end on the Membuster memory_warning benchmarks.
1072
1073         * bmalloc/Allocator.cpp:
1074         (bmalloc::Allocator::allocateSlowCase):
1075         * bmalloc/Allocator.h:
1076         (bmalloc::Allocator::allocateFastCase):
1077         (bmalloc::Allocator::smallAllocatorFor): Use the new shared helper
1078         function for size class calculation.
1079
1080         * bmalloc/Deallocator.cpp:
1081         (bmalloc::Deallocator::Deallocator):
1082         (bmalloc::Deallocator::scavenge):
1083         (bmalloc::Deallocator::deallocateSmallLine):
1084         (bmalloc::Deallocator::allocateSmallLine):
1085         * bmalloc/Deallocator.h: Keep a cache for every size class, since the
1086         cache can't be shared anymore.
1087
1088         * bmalloc/Heap.cpp:
1089         (bmalloc::Heap::allocateSmallLineSlowCase):
1090         * bmalloc/Heap.h:
1091         (bmalloc::Heap::deallocateSmallLine): Ditto.
1092
1093         (bmalloc::Heap::allocateSmallLine): Check size class in addition to
1094         page refcount when allocating a line because we might have deallocated
1095         the page and the recycled it for another size class.
1096
1097         (bmalloc::Heap::deallocateMediumLine):
1098         (bmalloc::Heap::allocateMediumLine):
1099         * bmalloc/Line.h:
1100         (bmalloc::Line::refCount):
1101         * bmalloc/Page.h:
1102         (bmalloc::Page::refCount):
1103         (bmalloc::Page::smallSizeClass):
1104         (bmalloc::Page::setSmallSizeClass):
1105         (bmalloc::Page<Traits>::refCount): Deleted.
1106         * bmalloc/Sizes.h:
1107         (bmalloc::Sizes::smallSizeClassFor): New shared API for computing
1108         an index into an array from a size.
1109
1110 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1111
1112         bmalloc: Improved alignment in LargeChunk
1113         https://bugs.webkit.org/show_bug.cgi?id=131895
1114
1115         Reviewed by Andreas Kling.
1116
1117         * bmalloc/Chunk.h:
1118         * bmalloc/LargeChunk.h: Align to vmPageSize just like Chunk does.
1119         Technically, the previous alignment was harmless, but I would prefer,
1120         dear reader, not to have to explain the interlocking set of
1121         circumstances that made it so.
1122
1123 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1124
1125         Rolled out r167502 because it caused a crash on the facebook benchmark.
1126
1127         Unreviewed.
1128
1129             bmalloc: Added an XSmall line size
1130             https://bugs.webkit.org/show_bug.cgi?id=131851
1131
1132             Reviewed by Sam Weinig.
1133
1134 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
1135
1136         bmalloc: Mutex should be harder to use wrong
1137         https://bugs.webkit.org/show_bug.cgi?id=131879
1138
1139         Reviewed by Andreas Kling.
1140
1141         Mutex now has a proper constructor, so you can't deadlock by forgetting
1142         to initialize it.
1143
1144         * bmalloc.xcodeproj/project.pbxproj:
1145         * bmalloc/Allocator.cpp:
1146         (bmalloc::Allocator::processXSmallAllocatorLog):
1147         (bmalloc::Allocator::processSmallAllocatorLog):
1148         (bmalloc::Allocator::processMediumAllocatorLog):
1149         (bmalloc::Allocator::allocateLarge):
1150         (bmalloc::Allocator::allocateXLarge): Global replace Mutex => StaticMutex,
1151         since the Heap mutex is a static.
1152
1153         * bmalloc/AsyncTask.h:
1154         (bmalloc::Function>::AsyncTask): Use Mutex, since we're not static. No
1155         need for explicit initialization anymore.
1156
1157         * bmalloc/Deallocator.cpp:
1158         (bmalloc::Deallocator::scavenge):
1159         (bmalloc::Deallocator::deallocateLarge):
1160         (bmalloc::Deallocator::deallocateXLarge):
1161         (bmalloc::Deallocator::processObjectLog):
1162         (bmalloc::Deallocator::deallocateSmallLine):
1163         (bmalloc::Deallocator::deallocateXSmallLine):
1164         (bmalloc::Deallocator::allocateSmallLine):
1165         (bmalloc::Deallocator::allocateXSmallLine):
1166         (bmalloc::Deallocator::deallocateMediumLine):
1167         (bmalloc::Deallocator::allocateMediumLine):
1168         * bmalloc/Deallocator.h:
1169         * bmalloc/Heap.cpp:
1170         (bmalloc::sleep):
1171         (bmalloc::Heap::Heap):
1172         (bmalloc::Heap::concurrentScavenge):
1173         (bmalloc::Heap::scavenge):
1174         (bmalloc::Heap::scavengeSmallPages):
1175         (bmalloc::Heap::scavengeXSmallPages):
1176         (bmalloc::Heap::scavengeMediumPages):
1177         (bmalloc::Heap::scavengeLargeRanges):
1178         (bmalloc::Heap::allocateXSmallLineSlowCase):
1179         (bmalloc::Heap::allocateSmallLineSlowCase):
1180         (bmalloc::Heap::allocateMediumLineSlowCase):
1181         (bmalloc::Heap::allocateXLarge):
1182         (bmalloc::Heap::deallocateXLarge):
1183         (bmalloc::Heap::allocateLarge):
1184         (bmalloc::Heap::deallocateLarge):
1185         * bmalloc/Heap.h:
1186         (bmalloc::Heap::deallocateXSmallLine):
1187         (bmalloc::Heap::allocateXSmallLine):
1188         (bmalloc::Heap::deallocateSmallLine):
1189         (bmalloc::Heap::allocateSmallLine):
1190         (bmalloc::Heap::deallocateMediumLine):
1191         (bmalloc::Heap::allocateMediumLine):
1192         * bmalloc/Line.h:
1193         (bmalloc::Line<Traits>::deref):
1194         * bmalloc/Mutex.cpp: Removed.
1195         * bmalloc/Mutex.h:
1196         (bmalloc::Mutex::Mutex):
1197         (bmalloc::Mutex::init): Deleted.
1198         (bmalloc::Mutex::try_lock): Deleted.
1199         (bmalloc::Mutex::lock): Deleted.
1200         (bmalloc::Mutex::unlock): Deleted.
1201         * bmalloc/Page.h:
1202         (bmalloc::Page<Traits>::ref):
1203         (bmalloc::Page<Traits>::deref):
1204         (bmalloc::Page<Traits>::refCount):
1205         * bmalloc/PerProcess.h:
1206         (bmalloc::PerProcess::mutex):
1207         (bmalloc::PerProcess<T>::getSlowCase):
1208         * bmalloc/StaticMutex.cpp: Added.
1209         (bmalloc::StaticMutex::lockSlowCase):
1210         * bmalloc/StaticMutex.h: Added.
1211         (bmalloc::StaticMutex::init):
1212         (bmalloc::StaticMutex::try_lock):
1213         (bmalloc::StaticMutex::lock):
1214         (bmalloc::StaticMutex::unlock):
1215         * bmalloc/VMHeap.h:
1216         (bmalloc::VMHeap::deallocateXSmallPage):
1217         (bmalloc::VMHeap::deallocateSmallPage):
1218         (bmalloc::VMHeap::deallocateMediumPage):
1219         (bmalloc::VMHeap::deallocateLargeRange):
1220         * bmalloc/bmalloc.h:
1221         (bmalloc::api::scavenge): Global replace Mutex => StaticMutex,
1222         since the Heap mutex is a static.
1223
1224 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
1225
1226         bmalloc: AsyncTask should use Mutex instead of std::mutex
1227         https://bugs.webkit.org/show_bug.cgi?id=131865
1228
1229         Reviewed by Gavin Barraclough.
1230
1231         std::mutex is so slow that it makes parallelizing simple tasks through
1232         AsyncTask a net regression. Mutex fixes this.
1233
1234         * bmalloc/AsyncTask.h:
1235         (bmalloc::Function>::AsyncTask):
1236         (bmalloc::Function>::join):
1237         (bmalloc::Function>::runSlowCase):
1238         (bmalloc::Function>::entryPoint):
1239         * bmalloc/Mutex.h:
1240         (bmalloc::Mutex::init):
1241
1242 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
1243
1244         bmalloc: Added an XSmall line size
1245         https://bugs.webkit.org/show_bug.cgi?id=131851
1246
1247         Reviewed by Sam Weinig.
1248
1249         Reduces malloc footprint on Membuster recordings by 10%.
1250
1251         This is a throughput regression, but we're still way ahead of TCMalloc.
1252         I have some ideas for how to recover the regression -- but I wanted to
1253         get this win in first.
1254
1255         Full set of benchmark results:
1256
1257                 bmalloc> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks --measure-heap nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1258                                                                                                 
1259                                                                nopatch                      patch                                Δ
1260                 Peak Memory:
1261                     reddit_memory_warning                      7,896kB                    7,532kB                  ^ 1.05x smaller
1262                     flickr_memory_warning                     12,968kB                   12,324kB                  ^ 1.05x smaller
1263                     theverge_memory_warning                   16,672kB                   15,200kB                   ^ 1.1x smaller
1264
1265                     <geometric mean>                          11,952kB                   11,216kB                  ^ 1.07x smaller
1266                     <arithmetic mean>                         12,512kB                   11,685kB                  ^ 1.07x smaller
1267                     <harmonic mean>                           11,375kB                   10,726kB                  ^ 1.06x smaller
1268
1269                 Memory at End:
1270                     reddit_memory_warning                      7,320kB                    6,856kB                  ^ 1.07x smaller
1271                     flickr_memory_warning                     10,848kB                    9,692kB                  ^ 1.12x smaller
1272                     theverge_memory_warning                   16,380kB                   14,872kB                   ^ 1.1x smaller
1273
1274                     <geometric mean>                          10,916kB                    9,961kB                   ^ 1.1x smaller
1275                     <arithmetic mean>                         11,516kB                   10,473kB                   ^ 1.1x smaller
1276                     <harmonic mean>                           10,350kB                    9,485kB                  ^ 1.09x smaller
1277
1278                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1279                                                                                                 
1280                                                    nopatch                patch                         Δ
1281                 Execution Time:
1282                     churn                            127ms                151ms            ! 1.19x slower
1283                     list_allocate                    130ms                164ms            ! 1.26x slower
1284                     tree_allocate                    109ms                127ms            ! 1.17x slower
1285                     tree_churn                       115ms                120ms            ! 1.04x slower
1286                     facebook                         240ms                259ms            ! 1.08x slower
1287                     fragment                          91ms                131ms            ! 1.44x slower
1288                     fragment_iterate                 105ms                106ms            ! 1.01x slower
1289                     message_one                      260ms                259ms             ^ 1.0x faster
1290                     message_many                     149ms                154ms            ! 1.03x slower
1291                     medium                           194ms                248ms            ! 1.28x slower
1292                     big                              157ms                160ms            ! 1.02x slower
1293
1294                     <geometric mean>                 144ms                163ms            ! 1.13x slower
1295                     <arithmetic mean>                152ms                171ms            ! 1.12x slower
1296                     <harmonic mean>                  137ms                156ms            ! 1.14x slower
1297
1298                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
1299                                                                                                 
1300                                                                        nopatch                          patch                                     Δ
1301                 Execution Time:
1302                     churn                                                126ms                          148ms                        ! 1.17x slower
1303                     churn --parallel                                      62ms                           76ms                        ! 1.23x slower
1304                     list_allocate                                        130ms                          164ms                        ! 1.26x slower
1305                     list_allocate --parallel                             120ms                          175ms                        ! 1.46x slower
1306                     tree_allocate                                        111ms                          127ms                        ! 1.14x slower
1307                     tree_allocate --parallel                              95ms                          135ms                        ! 1.42x slower
1308                     tree_churn                                           115ms                          124ms                        ! 1.08x slower
1309                     tree_churn --parallel                                107ms                          126ms                        ! 1.18x slower
1310                     facebook                                             240ms                          276ms                        ! 1.15x slower
1311                     facebook --parallel                                  802ms                        1,088ms                        ! 1.36x slower
1312                     fragment                                              92ms                          130ms                        ! 1.41x slower
1313                     fragment --parallel                                   66ms                          124ms                        ! 1.88x slower
1314                     fragment_iterate                                     109ms                          127ms                        ! 1.17x slower
1315                     fragment_iterate --parallel                           55ms                           64ms                        ! 1.16x slower
1316                     message_one                                          260ms                          260ms                                      
1317                     message_many                                         170ms                          238ms                         ! 1.4x slower
1318                     medium                                               185ms                          250ms                        ! 1.35x slower
1319                     medium --parallel                                    210ms                          334ms                        ! 1.59x slower
1320                     big                                                  150ms                          169ms                        ! 1.13x slower
1321                     big --parallel                                       138ms                          144ms                        ! 1.04x slower
1322
1323                     <geometric mean>                                     135ms                          170ms                        ! 1.26x slower
1324                     <arithmetic mean>                                    167ms                          214ms                        ! 1.28x slower
1325                     <harmonic mean>                                      117ms                          148ms                        ! 1.26x slower
1326
1327                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
1328
1329                                                                     TC                      patch                                Δ
1330                 Peak Memory:
1331                     reddit_memory_warning                     13,836kB                   13,436kB                  ^ 1.03x smaller
1332                     flickr_memory_warning                     24,868kB                   25,188kB                   ! 1.01x bigger
1333                     theverge_memory_warning                   24,504kB                   26,636kB                   ! 1.09x bigger
1334
1335                     <geometric mean>                          20,353kB                   20,812kB                   ! 1.02x bigger
1336                     <arithmetic mean>                         21,069kB                   21,753kB                   ! 1.03x bigger
1337                     <harmonic mean>                           19,570kB                   19,780kB                   ! 1.01x bigger
1338
1339                 Memory at End:
1340                     reddit_memory_warning                      8,656kB                   10,016kB                   ! 1.16x bigger
1341                     flickr_memory_warning                     11,844kB                   13,784kB                   ! 1.16x bigger
1342                     theverge_memory_warning                   18,516kB                   22,748kB                   ! 1.23x bigger
1343
1344                     <geometric mean>                          12,382kB                   14,644kB                   ! 1.18x bigger
1345                     <arithmetic mean>                         13,005kB                   15,516kB                   ! 1.19x bigger
1346                     <harmonic mean>                           11,813kB                   13,867kB                   ! 1.17x bigger
1347
1348                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
1349                                                                                                 
1350                                                         TC                patch                         Δ
1351                 Execution Time:
1352                     churn                            416ms                148ms            ^ 2.81x faster
1353                     list_allocate                    463ms                164ms            ^ 2.82x faster
1354                     tree_allocate                    292ms                127ms             ^ 2.3x faster
1355                     tree_churn                       157ms                120ms            ^ 1.31x faster
1356                     facebook                         327ms                276ms            ^ 1.18x faster
1357                     fragment                         335ms                129ms             ^ 2.6x faster
1358                     fragment_iterate                 344ms                108ms            ^ 3.19x faster
1359                     message_one                      386ms                258ms             ^ 1.5x faster
1360                     message_many                     410ms                154ms            ^ 2.66x faster
1361                     medium                           391ms                245ms             ^ 1.6x faster
1362                     big                              261ms                167ms            ^ 1.56x faster
1363
1364                     <geometric mean>                 332ms                164ms            ^ 2.02x faster
1365                     <arithmetic mean>                344ms                172ms            ^ 1.99x faster
1366                     <harmonic mean>                  317ms                157ms            ^ 2.02x faster
1367
1368         * bmalloc.xcodeproj/project.pbxproj:
1369         * bmalloc/Allocator.cpp:
1370         (bmalloc::Allocator::Allocator): Don't assume that each allocator's
1371         index corresponds with its size. Instead, use the size selection function
1372         explicitly. Now that we have XSmall, some small allocator entries are
1373         unused.
1374
1375         (bmalloc::Allocator::scavenge):
1376         (bmalloc::Allocator::log):
1377         (bmalloc::Allocator::processXSmallAllocatorLog):
1378         (bmalloc::Allocator::allocateSlowCase):
1379         * bmalloc/Allocator.h:
1380         (bmalloc::Allocator::xSmallAllocatorFor):
1381         (bmalloc::Allocator::allocateFastCase):
1382         * bmalloc/Chunk.h:
1383         * bmalloc/Deallocator.cpp:
1384         (bmalloc::Deallocator::scavenge):
1385         (bmalloc::Deallocator::processObjectLog):
1386         (bmalloc::Deallocator::deallocateSlowCase):
1387         (bmalloc::Deallocator::deallocateXSmallLine):
1388         (bmalloc::Deallocator::allocateXSmallLine):
1389         * bmalloc/Deallocator.h:
1390         (bmalloc::Deallocator::deallocateFastCase):
1391         * bmalloc/Heap.cpp:
1392         (bmalloc::Heap::scavenge):
1393         (bmalloc::Heap::scavengeXSmallPages):
1394         (bmalloc::Heap::allocateXSmallLineSlowCase):
1395         * bmalloc/Heap.h:
1396         (bmalloc::Heap::deallocateXSmallLine):
1397         (bmalloc::Heap::allocateXSmallLine):
1398         * bmalloc/LargeChunk.h:
1399         (bmalloc::LargeChunk::get):
1400         (bmalloc::LargeChunk::endTag):
1401         * bmalloc/Line.h:
1402         * bmalloc/MediumAllocator.h:
1403         (bmalloc::MediumAllocator::allocate):
1404         (bmalloc::MediumAllocator::refill):
1405         * bmalloc/ObjectType.cpp:
1406         (bmalloc::objectType):
1407         * bmalloc/ObjectType.h:
1408         (bmalloc::isXSmall):
1409         (bmalloc::isSmall):
1410         (bmalloc::isMedium):
1411         (bmalloc::isLarge):
1412         (bmalloc::isSmallOrMedium): Deleted.
1413         * bmalloc/SegregatedFreeList.h: I boiler-plate copied existing code for
1414         handling small objects. There's probably a reasonable way to share this
1415         code in the future -- I'll look into that once it's stopped changing.
1416
1417         * bmalloc/Sizes.h: Tweaked size classes to make Membuster happy. This
1418         is the main reason things got slower.
1419
1420         * bmalloc/SmallAllocator.h:
1421         (bmalloc::SmallAllocator::allocate):
1422         * bmalloc/SmallTraits.h:
1423         * bmalloc/VMHeap.cpp:
1424         (bmalloc::VMHeap::allocateXSmallChunk):
1425         * bmalloc/VMHeap.h:
1426         (bmalloc::VMHeap::allocateXSmallPage):
1427         (bmalloc::VMHeap::deallocateXSmallPage):
1428         * bmalloc/XSmallAllocator.h: Added.
1429         (bmalloc::XSmallAllocator::isNull):
1430         (bmalloc::XSmallAllocator::canAllocate):
1431         (bmalloc::XSmallAllocator::XSmallAllocator):
1432         (bmalloc::XSmallAllocator::line):
1433         (bmalloc::XSmallAllocator::allocate):
1434         (bmalloc::XSmallAllocator::objectCount):
1435         (bmalloc::XSmallAllocator::derefCount):
1436         (bmalloc::XSmallAllocator::refill):
1437         * bmalloc/XSmallChunk.h: Added.
1438         * bmalloc/XSmallLine.h: Added.
1439         * bmalloc/XSmallPage.h: Added.
1440         * bmalloc/XSmallTraits.h: Added.
1441         * bmalloc/bmalloc.h:
1442         (bmalloc::api::realloc): Boiler-plate copy, as above.
1443
1444 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
1445
1446         MallocBench should scavenge explicitly instead of waiting
1447         https://bugs.webkit.org/show_bug.cgi?id=131661
1448
1449         Reviewed by Andreas Kling.
1450
1451         Added explicit scavenge support to bmalloc. This isn't a memory win,
1452         since bmalloc's per-thread cache is so small. But it makes testing
1453         simpler.
1454
1455         * bmalloc/Allocator.cpp:
1456         (bmalloc::Allocator::~Allocator):
1457         (bmalloc::Allocator::scavenge):
1458         * bmalloc/Allocator.h:
1459         * bmalloc/Cache.cpp:
1460         (bmalloc::Cache::operator new):
1461         (bmalloc::Cache::operator delete):
1462         (bmalloc::Cache::Cache):
1463         (bmalloc::Cache::scavenge):
1464         * bmalloc/Cache.h:
1465         * bmalloc/Deallocator.cpp:
1466         (bmalloc::Deallocator::~Deallocator):
1467         (bmalloc::Deallocator::scavenge):
1468         * bmalloc/Deallocator.h: Factored existing scavenging code into helper
1469         functions, for reuse.
1470
1471         * bmalloc/Heap.cpp:
1472         (bmalloc::sleep):
1473         (bmalloc::Heap::concurrentScavenge):
1474         (bmalloc::Heap::scavenge):
1475         (bmalloc::Heap::scavengeSmallPages):
1476         (bmalloc::Heap::scavengeMediumPages):
1477         (bmalloc::Heap::scavengeLargeRanges):
1478         * bmalloc/Heap.h: Made scavenge sleep duration a parameter. Forced
1479         scavenging -- in response to a benchmark or a low memory warning --
1480         wants to complete as soon as possible, so its sleep duration is 0.
1481
1482         * bmalloc/bmalloc.h:
1483         (bmalloc::api::scavenge):
1484         * bmalloc/mbmalloc.cpp: Exported the scavenge API for MallocBench's use.
1485
1486 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
1487
1488         Use 4kB pages on Mac
1489         https://bugs.webkit.org/show_bug.cgi?id=131658
1490
1491         Reviewed by Sam Weinig.
1492
1493         This reduces memory use a lot on Membuster:
1494
1495                                                                   base                      patch                                Δ
1496                 Execution Time:
1497                     reddit_memory_warning                         18ms                       17ms                   ^ 1.06x faster
1498                     flickr_memory_warning                         34ms                       36ms                   ! 1.06x slower
1499                     theverge_memory_warning                       39ms                       41ms                   ! 1.05x slower
1500
1501                     <geometric mean>                              29ms                       29ms                   ! 1.02x slower
1502                     <arithmetic mean>                             30ms                       31ms                   ! 1.03x slower
1503                     <harmonic mean>                               27ms                       27ms                    ^ 1.0x faster
1504
1505                 Peak Memory:
1506                     reddit_memory_warning                     16,412kB                   16,436kB                    ! 1.0x bigger
1507                     flickr_memory_warning                     30,120kB                   30,184kB                    ! 1.0x bigger
1508                     theverge_memory_warning                   33,408kB                   33,420kB                    ! 1.0x bigger
1509
1510                     <geometric mean>                          25,466kB                   25,499kB                    ! 1.0x bigger
1511                     <arithmetic mean>                         26,647kB                   26,680kB                    ! 1.0x bigger
1512                     <harmonic mean>                           24,181kB                   24,214kB                    ! 1.0x bigger
1513
1514                 Memory at End:
1515                     reddit_memory_warning                      2,404kB                    1,920kB                  ^ 1.25x smaller
1516                     flickr_memory_warning                      3,764kB                    3,072kB                  ^ 1.23x smaller
1517                     theverge_memory_warning                    3,648kB                    3,132kB                  ^ 1.16x smaller
1518
1519                     <geometric mean>                           3,208kB                    2,644kB                  ^ 1.21x smaller
1520                     <arithmetic mean>                          3,272kB                    2,708kB                  ^ 1.21x smaller
1521                     <harmonic mean>                            3,139kB                    2,574kB                  ^ 1.22x smaller
1522
1523
1524         * bmalloc.xcodeproj/project.pbxproj:
1525         * bmalloc/BPlatform.h: Added.
1526         * bmalloc/VMAllocate.h: Only use 16kB pages on iOS because the page size
1527         is 4kB on Mac.
1528
1529 2014-04-14  Alexey Proskuryakov  <ap@apple.com>
1530
1531         Fixed svn:ignore on bmalloc.xcodeproj, it had erroneous leading spaces.
1532
1533         * bmalloc.xcodeproj: Modified property svn:ignore.
1534
1535 2014-04-13  Geoffrey Garen  <ggaren@apple.com>
1536
1537         Fixed some mbmalloc exports
1538         https://bugs.webkit.org/show_bug.cgi?id=131599
1539
1540         Reviewed by Ryosuke Niwa.
1541
1542         * bmalloc.xcodeproj/project.pbxproj: Made some headers a private part
1543         of the project, so we can call them from API.
1544
1545         * bmalloc/mbmalloc.cpp: Marked the mbmalloc functions with default
1546         visibility, so they show up as exported in the .dylib.
1547
1548 2014-04-09  Geoffrey Garen  <ggaren@apple.com>
1549
1550         Put bmalloc headers in the right place
1551         https://bugs.webkit.org/show_bug.cgi?id=131464
1552
1553         Reviewed by Mark Rowe.
1554
1555         * Configurations/bmalloc.xcconfig: Set PRIVATE_HEADERS_FOLDER_PATH to
1556         specify that we don't just want to dump all of our generically-named
1557         headers into /usr/local/include.
1558
1559 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
1560
1561         Made bmalloc more #include friendly
1562         https://bugs.webkit.org/show_bug.cgi?id=131386
1563
1564         Reviewed by Andreas Kling.
1565
1566         Marked a bunch of headers private so they can be used from client code
1567         that #includes bmalloc.h.
1568
1569         Renamed ASSERT macros to BASSERT. This matches their header, which already
1570         had to be renamed, and fixes conflicts with WTF's ASSERT macros.
1571
1572         * bmalloc.xcodeproj/project.pbxproj:
1573         * bmalloc/Allocator.cpp:
1574         (bmalloc::Allocator::allocateSlowCase):
1575         * bmalloc/AsyncTask.h:
1576         (bmalloc::Function>::runSlowCase):
1577         * bmalloc/BAssert.h:
1578         * bmalloc/BoundaryTag.h:
1579         (bmalloc::BoundaryTag::setSize):
1580         * bmalloc/BoundaryTagInlines.h:
1581         (bmalloc::validate):
1582         (bmalloc::BoundaryTag::init):
1583         (bmalloc::BoundaryTag::deallocate):
1584         (bmalloc::BoundaryTag::splitLarge):
1585         (bmalloc::BoundaryTag::allocate):
1586         * bmalloc/Chunk.h:
1587         * bmalloc/Deallocator.cpp:
1588         (bmalloc::Deallocator::processObjectLog):
1589         (bmalloc::Deallocator::deallocateSlowCase):
1590         * bmalloc/Deallocator.h:
1591         (bmalloc::Deallocator::deallocateFastCase):
1592         * bmalloc/FixedVector.h:
1593         (bmalloc::Capacity>::operator):
1594         (bmalloc::Capacity>::push):
1595         (bmalloc::Capacity>::pop):
1596         (bmalloc::Capacity>::shrink):
1597         * bmalloc/Heap.cpp:
1598         (bmalloc::Heap::allocateLarge):
1599         * bmalloc/LargeChunk.h:
1600         (bmalloc::LargeChunk::get):
1601         (bmalloc::LargeChunk::endTag):
1602         * bmalloc/Line.h:
1603         (bmalloc::Line<Traits>::concurrentRef):
1604         (bmalloc::Line<Traits>::deref):
1605         * bmalloc/MediumAllocator.h:
1606         (bmalloc::MediumAllocator::allocate):
1607         * bmalloc/ObjectType.h:
1608         (bmalloc::isSmall):
1609         * bmalloc/Page.h:
1610         (bmalloc::Page<Traits>::ref):
1611         (bmalloc::Page<Traits>::deref):
1612         * bmalloc/PerThread.h:
1613         (bmalloc::PerThread<T>::getSlowCase):
1614         * bmalloc/SegregatedFreeList.cpp:
1615         (bmalloc::SegregatedFreeList::SegregatedFreeList):
1616         (bmalloc::SegregatedFreeList::insert):
1617         * bmalloc/SmallAllocator.h:
1618         (bmalloc::SmallAllocator::allocate):
1619         (bmalloc::SmallAllocator::refill):
1620         * bmalloc/Syscall.h:
1621         * bmalloc/VMAllocate.h:
1622         (bmalloc::vmValidate):
1623         (bmalloc::vmAllocate):
1624         (bmalloc::vmDeallocatePhysicalPagesSloppy):
1625         * bmalloc/Vector.h:
1626         (bmalloc::Vector<T>::operator):
1627         (bmalloc::Vector<T>::pop):
1628         (bmalloc::Vector<T>::shrink):
1629         * bmalloc/XLargeChunk.h:
1630         (bmalloc::XLargeChunk::range):
1631         (bmalloc::XLargeChunk::size):
1632
1633 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
1634
1635         Removed an unused file.
1636
1637         Unreviewed.
1638
1639         * bmalloc/AsyncTask.cpp: Removed.
1640
1641 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1642
1643         Build bmalloc on Mac
1644         https://bugs.webkit.org/show_bug.cgi?id=131333
1645
1646         Reviewed by Mark Rowe.
1647
1648         * Makefile: Added. For make clients.
1649
1650         These files are required for building any project in WebKit. I copied
1651         them from WTF:
1652         * Configurations: Added.
1653         * Configurations/Base.xcconfig: Added.
1654         * Configurations/DebugRelease.xcconfig: Added.
1655         * Configurations/bmalloc.xcconfig: Added.
1656         * Configurations/iOS.xcconfig: Added.
1657         * Configurations/mbmalloc.xcconfig: Added.
1658
1659         * bmalloc.xcodeproj/project.pbxproj: I removed per-project-file stuff
1660         from here because everything is in .xcconfig files now.
1661
1662         I had to fix a bunch of minor warnings, since they're enabled in our
1663         .xcconfig files:
1664
1665         * bmalloc/AsyncTask.h:
1666         (bmalloc::Function>::AsyncTask):
1667         * bmalloc/BAssert.h:
1668         * bmalloc/BoundaryTagInlines.h:
1669         (bmalloc::validate):
1670         * bmalloc/Heap.cpp:
1671         (bmalloc::Heap::Heap):
1672         (bmalloc::Heap::allocateLarge):
1673         (bmalloc::Heap::deallocateLarge):
1674         * bmalloc/Mutex.h:
1675         (bmalloc::Mutex::Mutex): Deleted.
1676         * bmalloc/VMAllocate.h:
1677         (bmalloc::vmValidate):
1678         * bmalloc/mbmalloc.cpp:
1679
1680 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1681
1682         bmalloc: Fixed a leak in the per-thread cache
1683         https://bugs.webkit.org/show_bug.cgi?id=131330
1684
1685         Reviewed by Andreas Kling.
1686
1687         Remember to deallocate our line caches upon thread exit.
1688
1689         * bmalloc/Deallocator.cpp:
1690         (bmalloc::Deallocator::~Deallocator):
1691
1692 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1693
1694         bmalloc: rolled out the tryLock experiment
1695         https://bugs.webkit.org/show_bug.cgi?id=131328
1696
1697         Reviewed by Andreas Kling.
1698
1699         It wasn't a speedup.
1700
1701         * bmalloc.xcodeproj/project.pbxproj:
1702         * bmalloc/Allocator.cpp:
1703         (bmalloc::Allocator::processSmallAllocatorLog):
1704         (bmalloc::Allocator::processMediumAllocatorLog):
1705         * bmalloc/Deallocator.cpp:
1706         (bmalloc::Deallocator::processObjectLog):
1707         (bmalloc::Deallocator::deallocateSlowCase):
1708         (bmalloc::Deallocator::deallocateSmallLine):
1709         (bmalloc::Deallocator::deallocateMediumLine):
1710         * bmalloc/Deallocator.h:
1711         (bmalloc::Deallocator::deallocateFastCase):
1712         * bmalloc/Heap.h:
1713         (bmalloc::Heap::deallocateSmallLine):
1714         (bmalloc::Heap::deallocateMediumLine):
1715         * bmalloc/Line.h:
1716         (bmalloc::Line<Traits>::deref):
1717         * bmalloc/Page.h:
1718         (bmalloc::Page<Traits>::deref):
1719
1720 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1721
1722         bmalloc
1723         https://bugs.webkit.org/show_bug.cgi?id=131170
1724
1725         Reviewed by Andreas Kling.
1726
1727         Initial commit.
1728
1729         * bmalloc: Added.
1730         * bmalloc.xcodeproj: Added.
1731         * bmalloc.xcodeproj/project.pbxproj: Added.
1732         * bmalloc/Algorithm.h: Added.
1733         (bmalloc::max):
1734         (bmalloc::min):
1735         (bmalloc::mask):
1736         (bmalloc::test):
1737         (bmalloc::roundUpToMultipleOf):
1738         (bmalloc::roundDownToMultipleOf):
1739         (bmalloc::sizeOf):
1740         (bmalloc::bitCount):
1741         (bmalloc::isPowerOfTwo):
1742         * bmalloc/Allocator.cpp: Added.
1743         (bmalloc::Allocator::Allocator):
1744         (bmalloc::Allocator::~Allocator):
1745         (bmalloc::Allocator::log):
1746         (bmalloc::Allocator::processSmallAllocatorLog):
1747         (bmalloc::Allocator::processMediumAllocatorLog):
1748         (bmalloc::Allocator::allocateLarge):
1749         (bmalloc::Allocator::allocateXLarge):
1750         (bmalloc::Allocator::allocateMedium):
1751         (bmalloc::Allocator::allocateSlowCase):
1752         * bmalloc/Allocator.h: Added.
1753         (bmalloc::Allocator::smallAllocatorFor):
1754         (bmalloc::Allocator::allocateFastCase):
1755         (bmalloc::Allocator::allocate):
1756         * bmalloc/AsyncTask.cpp: Added.
1757         (bmalloc::AsyncTask<Function>::runSlowCase):
1758         (bmalloc::AsyncTask<Function>::pthreadEntryPoint):
1759         (bmalloc::AsyncTask<Function>::entryPoint):
1760         * bmalloc/AsyncTask.h: Added.
1761         (bmalloc::Function>::AsyncTask):
1762         (bmalloc::Function>::join):
1763         (bmalloc::Function>::run):
1764         (bmalloc::Function>::runSlowCase):
1765         (bmalloc::Function>::pthreadEntryPoint):
1766         (bmalloc::Function>::entryPoint):
1767         * bmalloc/BAssert.h: Added.
1768         * bmalloc/BeginTag.h: Added.
1769         (bmalloc::BeginTag::isInFreeList):
1770         * bmalloc/BoundaryTag.h: Added.
1771         (bmalloc::BoundaryTag::isXLarge):
1772         (bmalloc::BoundaryTag::setXLarge):
1773         (bmalloc::BoundaryTag::isFree):
1774         (bmalloc::BoundaryTag::setFree):
1775         (bmalloc::BoundaryTag::isEnd):
1776         (bmalloc::BoundaryTag::setEnd):
1777         (bmalloc::BoundaryTag::hasPhysicalPages):
1778         (bmalloc::BoundaryTag::setHasPhysicalPages):
1779         (bmalloc::BoundaryTag::isNull):
1780         (bmalloc::BoundaryTag::clear):
1781         (bmalloc::BoundaryTag::size):
1782         (bmalloc::BoundaryTag::setSize):
1783         (bmalloc::BoundaryTag::prev):
1784         (bmalloc::BoundaryTag::next):
1785         * bmalloc/BoundaryTagInlines.h: Added.
1786         (bmalloc::validate):
1787         (bmalloc::validatePrev):
1788         (bmalloc::validateNext):
1789         (bmalloc::BoundaryTag::init):
1790         (bmalloc::BoundaryTag::mergeLargeLeft):
1791         (bmalloc::BoundaryTag::mergeLargeRight):
1792         (bmalloc::BoundaryTag::mergeLarge):
1793         (bmalloc::BoundaryTag::deallocate):
1794         (bmalloc::BoundaryTag::splitLarge):
1795         (bmalloc::BoundaryTag::allocate):
1796         * bmalloc/Cache.cpp: Added.
1797         (bmalloc::Cache::operator new):
1798         (bmalloc::Cache::operator delete):
1799         (bmalloc::Cache::Cache):
1800         (bmalloc::Cache::allocateSlowCase):
1801         (bmalloc::Cache::allocateSlowCaseNullCache):
1802         (bmalloc::Cache::deallocateSlowCase):
1803         (bmalloc::Cache::deallocateSlowCaseNullCache):
1804         * bmalloc/Cache.h: Added.
1805         (bmalloc::Cache::allocator):
1806         (bmalloc::Cache::deallocator):
1807         (bmalloc::Cache::allocateFastCase):
1808         (bmalloc::Cache::deallocateFastCase):
1809         (bmalloc::Cache::allocate):
1810         (bmalloc::Cache::deallocate):
1811         * bmalloc/Chunk.h: Added.
1812         (bmalloc::Chunk::begin):
1813         (bmalloc::Chunk::end):
1814         (bmalloc::Chunk::lines):
1815         (bmalloc::Chunk::pages):
1816         * bmalloc/Deallocator.cpp: Added.
1817         (bmalloc::Deallocator::Deallocator):
1818         (bmalloc::Deallocator::~Deallocator):
1819         (bmalloc::Deallocator::deallocateLarge):
1820         (bmalloc::Deallocator::deallocateXLarge):
1821         (bmalloc::Deallocator::processObjectLog):
1822         (bmalloc::Deallocator::deallocateSlowCase):
1823         (bmalloc::Deallocator::deallocateSmallLine):
1824         (bmalloc::Deallocator::allocateSmallLine):
1825         (bmalloc::Deallocator::deallocateMediumLine):
1826         (bmalloc::Deallocator::allocateMediumLine):
1827         * bmalloc/Deallocator.h: Added.
1828         (bmalloc::Deallocator::deallocateFastCase):
1829         (bmalloc::Deallocator::deallocate):
1830         * bmalloc/EndTag.h: Added.
1831         (bmalloc::EndTag::operator=):
1832         * bmalloc/FixedVector.h: Added.
1833         (bmalloc::FixedVector::begin):
1834         (bmalloc::FixedVector::end):
1835         (bmalloc::FixedVector::size):
1836         (bmalloc::FixedVector::capacity):
1837         (bmalloc::FixedVector::clear):
1838         (bmalloc::FixedVector::isEmpty):
1839         (bmalloc::Capacity>::FixedVector):
1840         (bmalloc::Capacity>::operator):
1841         (bmalloc::Capacity>::push):
1842         (bmalloc::Capacity>::pop):
1843         (bmalloc::Capacity>::shrink):
1844         * bmalloc/Heap.cpp: Added.
1845         (bmalloc::sleep):
1846         (bmalloc::Heap::Heap):
1847         (bmalloc::Heap::concurrentScavenge):
1848         (bmalloc::Heap::scavengeSmallPages):
1849         (bmalloc::Heap::scavengeMediumPages):
1850         (bmalloc::Heap::scavengeLargeRanges):
1851         (bmalloc::Heap::allocateSmallLineSlowCase):
1852         (bmalloc::Heap::allocateMediumLineSlowCase):
1853         (bmalloc::Heap::allocateXLarge):
1854         (bmalloc::Heap::deallocateXLarge):
1855         (bmalloc::Heap::allocateLarge):
1856         (bmalloc::Heap::deallocateLarge):
1857         * bmalloc/Heap.h: Added.
1858         (bmalloc::Heap::deallocateSmallLine):
1859         (bmalloc::Heap::allocateSmallLine):
1860         (bmalloc::Heap::deallocateMediumLine):
1861         (bmalloc::Heap::allocateMediumLine):
1862         * bmalloc/Inline.h: Added.
1863         * bmalloc/LargeChunk.h: Added.
1864         (bmalloc::LargeChunk::begin):
1865         (bmalloc::LargeChunk::end):
1866         (bmalloc::LargeChunk::create):
1867         (bmalloc::LargeChunk::get):
1868         (bmalloc::LargeChunk::beginTag):
1869         (bmalloc::LargeChunk::endTag):
1870         * bmalloc/Line.h: Added.
1871         (bmalloc::Line<Traits>::begin):
1872         (bmalloc::Line<Traits>::end):
1873         (bmalloc::Line<Traits>::concurrentRef):
1874         (bmalloc::Line<Traits>::deref):
1875         * bmalloc/MediumAllocator.h: Added.
1876         (bmalloc::MediumAllocator::isNull):
1877         (bmalloc::MediumAllocator::MediumAllocator):
1878         (bmalloc::MediumAllocator::line):
1879         (bmalloc::MediumAllocator::allocate):
1880         (bmalloc::MediumAllocator::derefCount):
1881         (bmalloc::MediumAllocator::refill):
1882         * bmalloc/MediumChunk.h: Added.
1883         * bmalloc/MediumLine.h: Added.
1884         * bmalloc/MediumPage.h: Added.
1885         * bmalloc/MediumTraits.h: Added.
1886         * bmalloc/Mutex.cpp: Added.
1887         (bmalloc::Mutex::lockSlowCase):
1888         * bmalloc/Mutex.h: Added.
1889         (bmalloc::Mutex::Mutex):
1890         (bmalloc::Mutex::try_lock):
1891         (bmalloc::Mutex::lock):
1892         (bmalloc::Mutex::unlock):
1893         * bmalloc/ObjectType.cpp: Added.
1894         (bmalloc::objectType):
1895         * bmalloc/ObjectType.h: Added.
1896         (bmalloc::isSmallOrMedium):
1897         (bmalloc::isSmall):
1898         * bmalloc/Page.h: Added.
1899         (bmalloc::Page<Traits>::ref):
1900         (bmalloc::Page<Traits>::deref):
1901         (bmalloc::Page<Traits>::refCount):
1902         * bmalloc/PerProcess.h: Added.
1903         (bmalloc::PerProcess::mutex):
1904         (bmalloc::PerProcess<T>::getFastCase):
1905         (bmalloc::PerProcess<T>::get):
1906         (bmalloc::PerProcess<T>::getSlowCase):
1907         * bmalloc/PerThread.h: Added.
1908         (bmalloc::PerThreadStorage<Cache>::get):
1909         (bmalloc::PerThreadStorage<Cache>::init):
1910         (bmalloc::PerThreadStorage::get):
1911         (bmalloc::PerThreadStorage::init):
1912         (bmalloc::PerThread<T>::getFastCase):
1913         (bmalloc::PerThread<T>::get):
1914         (bmalloc::PerThread<T>::destructor):
1915         (bmalloc::PerThread<T>::getSlowCase):
1916         * bmalloc/Range.h: Added.
1917         (bmalloc::Range::Range):
1918         (bmalloc::Range::begin):
1919         (bmalloc::Range::end):
1920         (bmalloc::Range::size):
1921         (bmalloc::Range::operator!):
1922         (bmalloc::Range::operator<):
1923         * bmalloc/SegregatedFreeList.cpp: Added.
1924         (bmalloc::SegregatedFreeList::SegregatedFreeList):
1925         (bmalloc::SegregatedFreeList::insert):
1926         (bmalloc::SegregatedFreeList::takeGreedy):
1927         (bmalloc::SegregatedFreeList::take):
1928         * bmalloc/SegregatedFreeList.h: Added.
1929         * bmalloc/Sizes.h: Added.
1930         * bmalloc/SmallAllocator.h: Added.
1931         (bmalloc::SmallAllocator::isNull):
1932         (bmalloc::SmallAllocator::canAllocate):
1933         (bmalloc::SmallAllocator::SmallAllocator):
1934         (bmalloc::SmallAllocator::line):
1935         (bmalloc::SmallAllocator::allocate):
1936         (bmalloc::SmallAllocator::objectCount):
1937         (bmalloc::SmallAllocator::derefCount):
1938         (bmalloc::SmallAllocator::refill):
1939         * bmalloc/SmallChunk.h: Added.
1940         * bmalloc/SmallLine.h: Added.
1941         * bmalloc/SmallPage.h: Added.
1942         * bmalloc/SmallTraits.h: Added.
1943         * bmalloc/Syscall.h: Added.
1944         * bmalloc/VMAllocate.h: Added.
1945         (bmalloc::vmSize):
1946         (bmalloc::vmValidate):
1947         (bmalloc::vmAllocate):
1948         (bmalloc::vmDeallocate):
1949         (bmalloc::vmDeallocatePhysicalPages):
1950         (bmalloc::vmAllocatePhysicalPages):
1951         (bmalloc::vmDeallocatePhysicalPagesSloppy):
1952         (bmalloc::vmAllocatePhysicalPagesSloppy):
1953         * bmalloc/VMHeap.cpp: Added.
1954         (bmalloc::VMHeap::VMHeap):
1955         (bmalloc::VMHeap::allocateSmallChunk):
1956         (bmalloc::VMHeap::allocateMediumChunk):
1957         (bmalloc::VMHeap::allocateLargeChunk):
1958         * bmalloc/VMHeap.h: Added.
1959         (bmalloc::VMHeap::allocateSmallPage):
1960         (bmalloc::VMHeap::allocateMediumPage):
1961         (bmalloc::VMHeap::allocateLargeRange):
1962         (bmalloc::VMHeap::deallocateSmallPage):
1963         (bmalloc::VMHeap::deallocateMediumPage):
1964         (bmalloc::VMHeap::deallocateLargeRange):
1965         * bmalloc/Vector.h: Added.
1966         (bmalloc::Vector::begin):
1967         (bmalloc::Vector::end):
1968         (bmalloc::Vector::size):
1969         (bmalloc::Vector::capacity):
1970         (bmalloc::Vector::last):
1971         (bmalloc::Vector::pop):
1972         (bmalloc::Vector<T>::Vector):
1973         (bmalloc::Vector<T>::~Vector):
1974         (bmalloc::Vector<T>::operator):
1975         (bmalloc::Vector<T>::push):
1976         (bmalloc::Vector<T>::pop):
1977         (bmalloc::Vector<T>::shrink):
1978         (bmalloc::Vector<T>::reallocateBuffer):
1979         (bmalloc::Vector<T>::shrinkCapacity):
1980         (bmalloc::Vector<T>::growCapacity):
1981         * bmalloc/XLargeChunk.h: Added.
1982         (bmalloc::XLargeChunk::get):
1983         (bmalloc::XLargeChunk::begin):
1984         (bmalloc::XLargeChunk::XLargeChunk):
1985         (bmalloc::XLargeChunk::create):
1986         (bmalloc::XLargeChunk::destroy):
1987         (bmalloc::XLargeChunk::range):
1988         (bmalloc::XLargeChunk::size):
1989         * bmalloc/bmalloc.h: Added.
1990         (bmalloc::api::malloc):
1991         (bmalloc::api::free):
1992         (bmalloc::api::realloc):
1993         * bmalloc/mbmalloc.cpp: Added.
1994