Source/bmalloc/ChangeLog

   1 2014-09-15  Geoffrey Garen  <ggaren@apple.com>
   2
   3         bmalloc: allocate small and medium objects using the same bump pointer class
   4         https://bugs.webkit.org/show_bug.cgi?id=136843
   5
   6         Reviewed by Gavin Barraclough.
   7
   8         4% speedup on MallocBench.
   9
  10         Now that medium-sized objects have dedicated per-size allocators, they
  11         don't need to use an arbitrary bump pointer allocator. This means that
  12         every allocator knows how many objects it will allocate from the start,
  13         and we don't need a post-processing step to adjust refcounts based on
  14         real allocation count.
  15
  16         * bmalloc.xcodeproj/project.pbxproj: Renamed SmallAllocator to BumpAllocator
  17         since it's used for small and medium objects now.
  18
  19         * bmalloc/Allocator.cpp:
  20         (bmalloc::Allocator::Allocator): Updated to use new interface.
  21         (bmalloc::Allocator::scavenge): To "retire" an allocator, we just need
  22         to make sure that we finish allocating all the objects in it.
  23
  24         (bmalloc::Allocator::allocateMedium):
  25         (bmalloc::Allocator::allocateSlowCase):
  26         (bmalloc::Allocator::retire): Deleted.
  27         (bmalloc::Allocator::processSmallAllocatorLog): Deleted.
  28         (bmalloc::Allocator::processMediumAllocatorLog): Deleted.
  29         * bmalloc/Allocator.h:
  30         (bmalloc::Allocator::allocateFastCase): Removed abstractions and data
  31         used to post-process an allocator based on how many objects it allocated.
  32
  33         * bmalloc/BumpAllocator.h: Copied from Source/bmalloc/bmalloc/SmallAllocator.h.
  34         (bmalloc::BumpAllocator::BumpAllocator):
  35         (bmalloc::BumpAllocator::init):
  36         (bmalloc::BumpAllocator::line):
  37         (bmalloc::BumpAllocator::validate):
  38         (bmalloc::BumpAllocator::allocate):
  39         (bmalloc::BumpAllocator::refill):
  40         (bmalloc::BumpAllocator::clear): Updated these functions to be agnostic
  41         about the kinds of lines they allocate into. In some cases, the line
  42         type must be provided as a template parameter by the caller.
  43
  44         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
  45         (bmalloc::SmallAllocator::line): Deleted.
  46         (bmalloc::SmallAllocator::allocate): Deleted.
  47         (bmalloc::SmallAllocator::objectCount): Deleted.
  48         (bmalloc::SmallAllocator::derefCount): Deleted.
  49         (bmalloc::SmallAllocator::refill): Deleted.
  50         (bmalloc::SmallAllocator::clear): Deleted.
  51
  52         * bmalloc/ObjectType.h:
  53         (bmalloc::isMedium):
  54
  55         * bmalloc/SmallAllocator.h:
  56         (bmalloc::SmallAllocator::isNull): Deleted.
  57         (bmalloc::SmallAllocator::canAllocate): Deleted.
  58         (bmalloc::SmallAllocator::SmallAllocator): Deleted.
  59         (bmalloc::SmallAllocator::line): Deleted.
  60         (bmalloc::SmallAllocator::allocate): Deleted.
  61         (bmalloc::SmallAllocator::objectCount): Deleted.
  62         (bmalloc::SmallAllocator::derefCount): Deleted.
  63         (bmalloc::SmallAllocator::refill): Deleted.
  64         (bmalloc::SmallAllocator::clear): Deleted.
  65
  66 2014-09-12  Geoffrey Garen  <ggaren@apple.com>
  67
  68         Fixed a goof in bmalloc Vector sizing
  69         https://bugs.webkit.org/show_bug.cgi?id=136795
  70
  71         Reviewed by Gavin Barraclough and Sam Weinig.
  72
  73         We want our minimum vector to be page-sized since the OS will give us
  74         a page no matter what -- but we want that many bytes, and not enough
  75         bytes to store that many elements.
  76
  77         * bmalloc/Vector.h: Math is hard.
  78
  79 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
  80
  81         bmalloc should segregate medium-sized objects by line like it does for small-sized objects
  82         https://bugs.webkit.org/show_bug.cgi?id=136693
  83
  84         Reviewed by Gavin Barraclough.
  85
  86         4% reduction in heap size on the MallocBench *_memory_warning benchmarks.
  87
  88         No throughput change.
  89
  90         We keep an array of medium allocators, just like our array of small
  91         allocators.
  92
  93         In future, we can simplify the allocation fast path by merging the small
  94         and medium allocator arrays. For now, this is the simplest change that
  95         gets the win.
  96
  97         * bmalloc/Allocator.cpp:
  98         (bmalloc::Allocator::Allocator):
  99         (bmalloc::Allocator::scavenge):
 100         (bmalloc::Allocator::allocateMedium):
 101         * bmalloc/Allocator.h:
 102         * bmalloc/Sizes.h:
 103         (bmalloc::Sizes::mediumSizeClassFor):
 104
 105 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
 106
 107         Reviewed by Sam Weinig.
 108
 109         Renamed log => retire for clarity.
 110
 111         * bmalloc/Allocator.cpp:
 112         (bmalloc::Allocator::scavenge):
 113         (bmalloc::Allocator::retire):
 114         (bmalloc::Allocator::allocateMedium):
 115         (bmalloc::Allocator::allocateSlowCase):
 116         (bmalloc::Allocator::log): Deleted.
 117         * bmalloc/Allocator.h:
 118
 119 2014-09-11  Geoffrey Garen  <ggaren@apple.com>
 120
 121         bmalloc: eager scavenge leaves behind a bogus allocator
 122         https://bugs.webkit.org/show_bug.cgi?id=136743
 123
 124         Reviewed by Sam Weinig.
 125
 126         Be sure to clear the allocator after logging it in the eager scavenge
 127         case, so that we don't later try to allocate out of the lines that we
 128         have thrown away.
 129
 130         We didn't need to do this previously because scavenge would only happen
 131         at thread exit time, after which no further allocation from the per-thread
 132         cache would take place.
 133
 134         * bmalloc/Allocator.cpp:
 135         (bmalloc::Allocator::scavenge):
 136         * bmalloc/MediumAllocator.h:
 137         (bmalloc::MediumAllocator::clear):
 138         * bmalloc/SmallAllocator.h:
 139         (bmalloc::SmallAllocator::clear):
 140
 141 2014-09-05  Geoffrey Garen  <ggaren@apple.com>
 142
 143         bmalloc should honor the FastMalloc statistics API
 144         https://bugs.webkit.org/show_bug.cgi?id=136592
 145
 146         Reviewed by Gavin Barraclough.
 147
 148         We do this by tracking "size" and "capacity" in the VM heap.
 149
 150         The VM heap's "capacity" is all the VM we ever allocated.
 151
 152         The VM heap's "size" the subset of VM currently held onto by the
 153         VM heap (and therefore not in use by the regular heap).
 154
 155         Somewhat ironically, reducing the process's memory footprint, increases
 156         the size of the VM heap, since the VM heap holds the pages that are
 157         purely virtual and not physical.
 158
 159         * bmalloc/Heap.cpp:
 160         (bmalloc::Heap::size):
 161         (bmalloc::Heap::capacity):
 162         * bmalloc/Heap.h:
 163         * bmalloc/VMHeap.cpp:
 164         (bmalloc::VMHeap::VMHeap):
 165         (bmalloc::VMHeap::allocateSmallChunk):
 166         (bmalloc::VMHeap::allocateMediumChunk):
 167         (bmalloc::VMHeap::allocateLargeChunk):
 168         * bmalloc/VMHeap.h:
 169         (bmalloc::VMHeap::size):
 170         (bmalloc::VMHeap::capacity):
 171         (bmalloc::VMHeap::allocateSmallPage):
 172         (bmalloc::VMHeap::allocateMediumPage):
 173         (bmalloc::VMHeap::allocateLargeRange):
 174         (bmalloc::VMHeap::deallocateSmallPage):
 175         (bmalloc::VMHeap::deallocateMediumPage):
 176         (bmalloc::VMHeap::deallocateLargeRange):
 177         * bmalloc/bmalloc.h:
 178         (bmalloc::api::heapSize):
 179         (bmalloc::api::heapCapacity):
 180
 181 2014-09-02  Geoffrey Garen  <ggaren@apple.com>
 182
 183         bmalloc crashes on the EWS bots (due to bad large object allocation)
 184         https://bugs.webkit.org/show_bug.cgi?id=136469
 185
 186         Reviewed by Andreas Kling.
 187
 188         It's possible to convince bmalloc to perform a bad large object allocation,
 189         through these steps:
 190
 191         (1) Insert object A into freelist F0.
 192
 193         (2) Split, merge and split again A's neighbors such that object B is
 194         inserted into freelist F0, with boundary tag and size equal to object A,
 195         but pointer not completely equal to object A. Put object B at the head of F0.
 196
 197         (3) Allocate some other object from F0, swapping its position in the
 198         freelist with object B, such that object A is now ahead of object B.
 199
 200         --> Now, the next allocation for size A/B will allocate object A, which
 201         has a slightly wrong idea about where the object actually begins.
 202         Immediately, you'll corrupt a little memory, and over time, you'll also
 203         corrupt boundary tag metadata.
 204
 205         The solution is to store the begin pointer in the boundary tag. Luckily,
 206         this doesn't make the tag any bigger, and it's not a noticeable slowdown
 207         on MallocBench.
 208
 209         * bmalloc/Algorithm.h:
 210         (bmalloc::rightShift):
 211         * bmalloc/BeginTag.h:
 212         (bmalloc::BeginTag::isInFreeList): This is the bug fix. Make sure to
 213         validate the start pointer when popping off the free list. Through a
 214         very uncommon set of steps, it is possible to have an item in the free
 215         list that is valid by all accounts except for its start pointer.
 216
 217         * bmalloc/BoundaryTag.h:
 218         (bmalloc::BoundaryTag::compactBegin):
 219         (bmalloc::BoundaryTag::setRange):
 220         (bmalloc::BoundaryTag::setSize): Deleted. Record a compact version of the
 221         start pointer. We don't need the whole pointer -- just the offset, in
 222         largeAlignment increments, into the relevant boundary tag bucket.
 223
 224         * bmalloc/BoundaryTagInlines.h:
 225         (bmalloc::validateNext):
 226         (bmalloc::BoundaryTag::init):
 227         (bmalloc::BoundaryTag::mergeLarge):
 228         (bmalloc::BoundaryTag::splitLarge):
 229         * bmalloc/SegregatedFreeList.cpp:
 230         (bmalloc::SegregatedFreeList::insert):
 231         (bmalloc::SegregatedFreeList::takeGreedy):
 232         (bmalloc::SegregatedFreeList::take): Provide the whole range instead of
 233         the size when establishing a boundary tag, as required by the new
 234         interface.
 235
 236         * bmalloc/Sizes.h:
 237
 238 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
 239
 240         Fixed a bmalloc crash seen on the EWS bot
 241         https://bugs.webkit.org/show_bug.cgi?id=135955
 242
 243         Reviewed by Andreas Kling.
 244
 245         * bmalloc/Syscall.h: Some CG APIs vm_copy their input buffers. If the
 246         input buffer is a malloc region, that region will get marked Copy-On-Write
 247         by the kernel. Calls to madvise() for COW regions fail and return EINVAL
 248         on older OS X's. In 10.10, they still fail, but they do not return
 249         EINVAL.
 250
 251         So, we can only ASSERT that our syscalls succeed starting with 10.10.
 252
 253 2014-08-14  Geoffrey Garen  <ggaren@apple.com>
 254
 255         Fixed the bmalloc build
 256         https://bugs.webkit.org/show_bug.cgi?id=135953
 257
 258         Reviewed by Andreas Kling.
 259
 260         * bmalloc.xcodeproj/project.pbxproj: Marked a few headers as private.
 261         These headers are used, so they must be available outside the project.
 262
 263 2014-08-13  Daniel Bates  <dabates@apple.com>
 264
 265         Attempt to fix the build following <http://trac.webkit.org/changeset/172576>
 266         (https://bugs.webkit.org/show_bug.cgi?id=135895)
 267
 268         Substitute PerThreadStorage<T>::initSharedKeyIfNeeded() for initSharedKeyIfNeeded() in
 269         implementation of PerThread<T>::getFastCase().
 270
 271         * bmalloc/PerThread.h:
 272         (bmalloc::PerThread<T>::getFastCase):
 273
 274 2014-08-13  Daniel Bates  <dabates@apple.com>
 275
 276         Make bmalloc::PerThread work without C++ thread local storage
 277         https://bugs.webkit.org/show_bug.cgi?id=135895
 278
 279         Reviewed by Geoffrey Garen.
 280
 281         Implement support for building bmalloc without C++ thread local storage.
 282
 283         * bmalloc/BPlatform.h: Remove macro define BPLATFORM_IOS_SIMULATOR. Added macro function
 284         BCOMPILER_SUPPORTS() and macro define BCOMPILER_SUPPORTS_CXX_THREAD_LOCAL that can be used
 285         to determine whether the compiler supports C++ thread local storage.
 286         * bmalloc/PerThread.h:
 287         (bmalloc::PerThreadStorage::get): Modified to call pthread_getspecific() when building
 288         without C++ thread local storage.
 289         (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Added.
 290         (bmalloc::PerThreadStorage::init): Moved logic to initialize shared Pthread key from here to
 291         PerThreadStorage::initSharedKeyIfNeeded().
 292         (bmalloc::PerThread<T>::getFastCase): Modified to call PerThreadStorage::initSharedKeyIfNeeded()
 293         before querying PerThreadStorage::get() when building without C++ thread local storage so as to
 294         ensure that the shared key has been initialized.
 295         (_pthread_setspecific_direct): Deleted.
 296         (_pthread_getspecific_direct): Deleted.
 297
 298 2014-08-13  Daniel Bates  <dabates@apple.com>
 299
 300         [iOS] Make JavaScriptCore and bmalloc build with the public SDK
 301         https://bugs.webkit.org/show_bug.cgi?id=135848
 302
 303         Reviewed by Geoffrey Garen.
 304
 305         * bmalloc/BPlatform.h: Added macro BPLATFORM_IOS_SIMULATOR, which evaluates to true
 306         when building for the iOS Simulator.
 307         * bmalloc/PerThread.h: Use pthread_machdep.h code path when building for iOS Simulator
 308         using the public SDK.
 309         (_pthread_setspecific_direct): Added; only defined when building for the iOS Simulator
 310         using the public SDK.
 311         (_pthread_getspecific_direct): Added; only defined when building for the iOS Simulator
 312         using the public SDK.
 313
 314 2014-08-12  Daniel Bates  <dabates@apple.com>
 315
 316         BPLATFORM(IOS) always evaluates to false
 317         https://bugs.webkit.org/show_bug.cgi?id=135843
 318
 319         Reviewed by Geoffrey Garen.
 320
 321         Fix typo in definition of BPLATFORM() and include system header TargetConditionals.h
 322         (when building on an Apple platform) so that BPLATFORM(X) evaluates to true when
 323         building for platform X. In particular, so that BPLATFORM(IOS) evaluates to true when
 324         building for iOS.
 325
 326         As a side effect of this change, the change made in <http://trac.webkit.org/changeset/167289>
 327         will be honored and iOS will assume a VM page size of 16kB (again) instead of 4kB.
 328
 329         * bmalloc/BPlatform.h:
 330
 331 2014-08-11  Andy Estes  <aestes@apple.com>
 332
 333         [iOS] Get rid of iOS.xcconfig
 334         https://bugs.webkit.org/show_bug.cgi?id=135809
 335
 336         Reviewed by Joseph Pecoraro.
 337
 338         All iOS.xcconfig did was include AspenFamily.xcconfig, so there's no need for the indirection.
 339
 340         * Configurations/Base.xcconfig:
 341         * Configurations/iOS.xcconfig: Removed.
 342         * bmalloc.xcodeproj/project.pbxproj:
 343
 344 2014-05-01  Dan Bernstein  <mitz@apple.com>
 345
 346         Fixed production builds for the iOS Simulator.
 347         <rdar://problem/16792221>
 348
 349         * Configurations/bmalloc.xcconfig: Include INSTALL_PATH_PREFIX in
 350         PRIVATE_HEADERS_FOLDER_PATH when installing.
 351
 352 2014-04-20  Geoffrey Garen  <ggaren@apple.com>
 353
 354         bmalloc: Segregate pages by objects size
 355         https://bugs.webkit.org/show_bug.cgi?id=131909
 356
 357         Reviewed by Andreas Kling.
 358
 359         2% reduction in memory-at-end on the Membuster memory_warning benchmarks.
 360
 361         * bmalloc/Allocator.cpp:
 362         (bmalloc::Allocator::allocateSlowCase):
 363         * bmalloc/Allocator.h:
 364         (bmalloc::Allocator::allocateFastCase):
 365         (bmalloc::Allocator::smallAllocatorFor): Use the new shared helper
 366         function for size class calculation.
 367
 368         * bmalloc/Deallocator.cpp:
 369         (bmalloc::Deallocator::Deallocator):
 370         (bmalloc::Deallocator::scavenge):
 371         (bmalloc::Deallocator::deallocateSmallLine):
 372         (bmalloc::Deallocator::allocateSmallLine):
 373         * bmalloc/Deallocator.h: Keep a cache for every size class, since the
 374         cache can't be shared anymore.
 375
 376         * bmalloc/Heap.cpp:
 377         (bmalloc::Heap::allocateSmallLineSlowCase):
 378         * bmalloc/Heap.h:
 379         (bmalloc::Heap::deallocateSmallLine): Ditto.
 380
 381         (bmalloc::Heap::allocateSmallLine): Check size class in addition to
 382         page refcount when allocating a line because we might have deallocated
 383         the page and the recycled it for another size class.
 384
 385         (bmalloc::Heap::deallocateMediumLine):
 386         (bmalloc::Heap::allocateMediumLine):
 387         * bmalloc/Line.h:
 388         (bmalloc::Line::refCount):
 389         * bmalloc/Page.h:
 390         (bmalloc::Page::refCount):
 391         (bmalloc::Page::smallSizeClass):
 392         (bmalloc::Page::setSmallSizeClass):
 393         (bmalloc::Page<Traits>::refCount): Deleted.
 394         * bmalloc/Sizes.h:
 395         (bmalloc::Sizes::smallSizeClassFor): New shared API for computing
 396         an index into an array from a size.
 397
 398 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
 399
 400         bmalloc: Improved alignment in LargeChunk
 401         https://bugs.webkit.org/show_bug.cgi?id=131895
 402
 403         Reviewed by Andreas Kling.
 404
 405         * bmalloc/Chunk.h:
 406         * bmalloc/LargeChunk.h: Align to vmPageSize just like Chunk does.
 407         Technically, the previous alignment was harmless, but I would prefer,
 408         dear reader, not to have to explain the interlocking set of
 409         circumstances that made it so.
 410
 411 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
 412
 413         Rolled out r167502 because it caused a crash on the facebook benchmark.
 414
 415         Unreviewed.
 416
 417             bmalloc: Added an XSmall line size
 418             https://bugs.webkit.org/show_bug.cgi?id=131851
 419
 420             Reviewed by Sam Weinig.
 421
 422 2014-04-19  Geoffrey Garen  <ggaren@apple.com>
 423
 424         bmalloc: Mutex should be harder to use wrong
 425         https://bugs.webkit.org/show_bug.cgi?id=131879
 426
 427         Reviewed by Andreas Kling.
 428
 429         Mutex now has a proper constructor, so you can't deadlock by forgetting
 430         to initialize it.
 431
 432         * bmalloc.xcodeproj/project.pbxproj:
 433         * bmalloc/Allocator.cpp:
 434         (bmalloc::Allocator::processXSmallAllocatorLog):
 435         (bmalloc::Allocator::processSmallAllocatorLog):
 436         (bmalloc::Allocator::processMediumAllocatorLog):
 437         (bmalloc::Allocator::allocateLarge):
 438         (bmalloc::Allocator::allocateXLarge): Global replace Mutex => StaticMutex,
 439         since the Heap mutex is a static.
 440
 441         * bmalloc/AsyncTask.h:
 442         (bmalloc::Function>::AsyncTask): Use Mutex, since we're not static. No
 443         need for explicit initialization anymore.
 444
 445         * bmalloc/Deallocator.cpp:
 446         (bmalloc::Deallocator::scavenge):
 447         (bmalloc::Deallocator::deallocateLarge):
 448         (bmalloc::Deallocator::deallocateXLarge):
 449         (bmalloc::Deallocator::processObjectLog):
 450         (bmalloc::Deallocator::deallocateSmallLine):
 451         (bmalloc::Deallocator::deallocateXSmallLine):
 452         (bmalloc::Deallocator::allocateSmallLine):
 453         (bmalloc::Deallocator::allocateXSmallLine):
 454         (bmalloc::Deallocator::deallocateMediumLine):
 455         (bmalloc::Deallocator::allocateMediumLine):
 456         * bmalloc/Deallocator.h:
 457         * bmalloc/Heap.cpp:
 458         (bmalloc::sleep):
 459         (bmalloc::Heap::Heap):
 460         (bmalloc::Heap::concurrentScavenge):
 461         (bmalloc::Heap::scavenge):
 462         (bmalloc::Heap::scavengeSmallPages):
 463         (bmalloc::Heap::scavengeXSmallPages):
 464         (bmalloc::Heap::scavengeMediumPages):
 465         (bmalloc::Heap::scavengeLargeRanges):
 466         (bmalloc::Heap::allocateXSmallLineSlowCase):
 467         (bmalloc::Heap::allocateSmallLineSlowCase):
 468         (bmalloc::Heap::allocateMediumLineSlowCase):
 469         (bmalloc::Heap::allocateXLarge):
 470         (bmalloc::Heap::deallocateXLarge):
 471         (bmalloc::Heap::allocateLarge):
 472         (bmalloc::Heap::deallocateLarge):
 473         * bmalloc/Heap.h:
 474         (bmalloc::Heap::deallocateXSmallLine):
 475         (bmalloc::Heap::allocateXSmallLine):
 476         (bmalloc::Heap::deallocateSmallLine):
 477         (bmalloc::Heap::allocateSmallLine):
 478         (bmalloc::Heap::deallocateMediumLine):
 479         (bmalloc::Heap::allocateMediumLine):
 480         * bmalloc/Line.h:
 481         (bmalloc::Line<Traits>::deref):
 482         * bmalloc/Mutex.cpp: Removed.
 483         * bmalloc/Mutex.h:
 484         (bmalloc::Mutex::Mutex):
 485         (bmalloc::Mutex::init): Deleted.
 486         (bmalloc::Mutex::try_lock): Deleted.
 487         (bmalloc::Mutex::lock): Deleted.
 488         (bmalloc::Mutex::unlock): Deleted.
 489         * bmalloc/Page.h:
 490         (bmalloc::Page<Traits>::ref):
 491         (bmalloc::Page<Traits>::deref):
 492         (bmalloc::Page<Traits>::refCount):
 493         * bmalloc/PerProcess.h:
 494         (bmalloc::PerProcess::mutex):
 495         (bmalloc::PerProcess<T>::getSlowCase):
 496         * bmalloc/StaticMutex.cpp: Added.
 497         (bmalloc::StaticMutex::lockSlowCase):
 498         * bmalloc/StaticMutex.h: Added.
 499         (bmalloc::StaticMutex::init):
 500         (bmalloc::StaticMutex::try_lock):
 501         (bmalloc::StaticMutex::lock):
 502         (bmalloc::StaticMutex::unlock):
 503         * bmalloc/VMHeap.h:
 504         (bmalloc::VMHeap::deallocateXSmallPage):
 505         (bmalloc::VMHeap::deallocateSmallPage):
 506         (bmalloc::VMHeap::deallocateMediumPage):
 507         (bmalloc::VMHeap::deallocateLargeRange):
 508         * bmalloc/bmalloc.h:
 509         (bmalloc::api::scavenge): Global replace Mutex => StaticMutex,
 510         since the Heap mutex is a static.
 511
 512 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
 513
 514         bmalloc: AsyncTask should use Mutex instead of std::mutex
 515         https://bugs.webkit.org/show_bug.cgi?id=131865
 516
 517         Reviewed by Gavin Barraclough.
 518
 519         std::mutex is so slow that it makes parallelizing simple tasks through
 520         AsyncTask a net regression. Mutex fixes this.
 521
 522         * bmalloc/AsyncTask.h:
 523         (bmalloc::Function>::AsyncTask):
 524         (bmalloc::Function>::join):
 525         (bmalloc::Function>::runSlowCase):
 526         (bmalloc::Function>::entryPoint):
 527         * bmalloc/Mutex.h:
 528         (bmalloc::Mutex::init):
 529
 530 2014-04-18  Geoffrey Garen  <ggaren@apple.com>
 531
 532         bmalloc: Added an XSmall line size
 533         https://bugs.webkit.org/show_bug.cgi?id=131851
 534
 535         Reviewed by Sam Weinig.
 536
 537         Reduces malloc footprint on Membuster recordings by 10%.
 538
 539         This is a throughput regression, but we're still way ahead of TCMalloc.
 540         I have some ideas for how to recover the regression -- but I wanted to
 541         get this win in first.
 542
 543         Full set of benchmark results:
 544
 545                 bmalloc> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks --measure-heap nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
 546
 547                                                                nopatch                      patch                                Δ
 548                 Peak Memory:
 549                     reddit_memory_warning                      7,896kB                    7,532kB                  ^ 1.05x smaller
 550                     flickr_memory_warning                     12,968kB                   12,324kB                  ^ 1.05x smaller
 551                     theverge_memory_warning                   16,672kB                   15,200kB                   ^ 1.1x smaller
 552
 553                     <geometric mean>                          11,952kB                   11,216kB                  ^ 1.07x smaller
 554                     <arithmetic mean>                         12,512kB                   11,685kB                  ^ 1.07x smaller
 555                     <harmonic mean>                           11,375kB                   10,726kB                  ^ 1.06x smaller
 556
 557                 Memory at End:
 558                     reddit_memory_warning                      7,320kB                    6,856kB                  ^ 1.07x smaller
 559                     flickr_memory_warning                     10,848kB                    9,692kB                  ^ 1.12x smaller
 560                     theverge_memory_warning                   16,380kB                   14,872kB                   ^ 1.1x smaller
 561
 562                     <geometric mean>                          10,916kB                    9,961kB                   ^ 1.1x smaller
 563                     <arithmetic mean>                         11,516kB                   10,473kB                   ^ 1.1x smaller
 564                     <harmonic mean>                           10,350kB                    9,485kB                  ^ 1.09x smaller
 565
 566                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
 567
 568                                                    nopatch                patch                         Δ
 569                 Execution Time:
 570                     churn                            127ms                151ms            ! 1.19x slower
 571                     list_allocate                    130ms                164ms            ! 1.26x slower
 572                     tree_allocate                    109ms                127ms            ! 1.17x slower
 573                     tree_churn                       115ms                120ms            ! 1.04x slower
 574                     facebook                         240ms                259ms            ! 1.08x slower
 575                     fragment                          91ms                131ms            ! 1.44x slower
 576                     fragment_iterate                 105ms                106ms            ! 1.01x slower
 577                     message_one                      260ms                259ms             ^ 1.0x faster
 578                     message_many                     149ms                154ms            ! 1.03x slower
 579                     medium                           194ms                248ms            ! 1.28x slower
 580                     big                              157ms                160ms            ! 1.02x slower
 581
 582                     <geometric mean>                 144ms                163ms            ! 1.13x slower
 583                     <arithmetic mean>                152ms                171ms            ! 1.12x slower
 584                     <harmonic mean>                  137ms                156ms            ! 1.14x slower
 585
 586                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
 587
 588                                                                        nopatch                          patch                                     Δ
 589                 Execution Time:
 590                     churn                                                126ms                          148ms                        ! 1.17x slower
 591                     churn --parallel                                      62ms                           76ms                        ! 1.23x slower
 592                     list_allocate                                        130ms                          164ms                        ! 1.26x slower
 593                     list_allocate --parallel                             120ms                          175ms                        ! 1.46x slower
 594                     tree_allocate                                        111ms                          127ms                        ! 1.14x slower
 595                     tree_allocate --parallel                              95ms                          135ms                        ! 1.42x slower
 596                     tree_churn                                           115ms                          124ms                        ! 1.08x slower
 597                     tree_churn --parallel                                107ms                          126ms                        ! 1.18x slower
 598                     facebook                                             240ms                          276ms                        ! 1.15x slower
 599                     facebook --parallel                                  802ms                        1,088ms                        ! 1.36x slower
 600                     fragment                                              92ms                          130ms                        ! 1.41x slower
 601                     fragment --parallel                                   66ms                          124ms                        ! 1.88x slower
 602                     fragment_iterate                                     109ms                          127ms                        ! 1.17x slower
 603                     fragment_iterate --parallel                           55ms                           64ms                        ! 1.16x slower
 604                     message_one                                          260ms                          260ms
 605                     message_many                                         170ms                          238ms                         ! 1.4x slower
 606                     medium                                               185ms                          250ms                        ! 1.35x slower
 607                     medium --parallel                                    210ms                          334ms                        ! 1.59x slower
 608                     big                                                  150ms                          169ms                        ! 1.13x slower
 609                     big --parallel                                       138ms                          144ms                        ! 1.04x slower
 610
 611                     <geometric mean>                                     135ms                          170ms                        ! 1.26x slower
 612                     <arithmetic mean>                                    167ms                          214ms                        ! 1.28x slower
 613                     <harmonic mean>                                      117ms                          148ms                        ! 1.26x slower
 614
 615                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
 616
 617                                                                     TC                      patch                                Δ
 618                 Peak Memory:
 619                     reddit_memory_warning                     13,836kB                   13,436kB                  ^ 1.03x smaller
 620                     flickr_memory_warning                     24,868kB                   25,188kB                   ! 1.01x bigger
 621                     theverge_memory_warning                   24,504kB                   26,636kB                   ! 1.09x bigger
 622
 623                     <geometric mean>                          20,353kB                   20,812kB                   ! 1.02x bigger
 624                     <arithmetic mean>                         21,069kB                   21,753kB                   ! 1.03x bigger
 625                     <harmonic mean>                           19,570kB                   19,780kB                   ! 1.01x bigger
 626
 627                 Memory at End:
 628                     reddit_memory_warning                      8,656kB                   10,016kB                   ! 1.16x bigger
 629                     flickr_memory_warning                     11,844kB                   13,784kB                   ! 1.16x bigger
 630                     theverge_memory_warning                   18,516kB                   22,748kB                   ! 1.23x bigger
 631
 632                     <geometric mean>                          12,382kB                   14,644kB                   ! 1.18x bigger
 633                     <arithmetic mean>                         13,005kB                   15,516kB                   ! 1.19x bigger
 634                     <harmonic mean>                           11,813kB                   13,867kB                   ! 1.17x bigger
 635
 636                 MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
 637
 638                                                         TC                patch                         Δ
 639                 Execution Time:
 640                     churn                            416ms                148ms            ^ 2.81x faster
 641                     list_allocate                    463ms                164ms            ^ 2.82x faster
 642                     tree_allocate                    292ms                127ms             ^ 2.3x faster
 643                     tree_churn                       157ms                120ms            ^ 1.31x faster
 644                     facebook                         327ms                276ms            ^ 1.18x faster
 645                     fragment                         335ms                129ms             ^ 2.6x faster
 646                     fragment_iterate                 344ms                108ms            ^ 3.19x faster
 647                     message_one                      386ms                258ms             ^ 1.5x faster
 648                     message_many                     410ms                154ms            ^ 2.66x faster
 649                     medium                           391ms                245ms             ^ 1.6x faster
 650                     big                              261ms                167ms            ^ 1.56x faster
 651
 652                     <geometric mean>                 332ms                164ms            ^ 2.02x faster
 653                     <arithmetic mean>                344ms                172ms            ^ 1.99x faster
 654                     <harmonic mean>                  317ms                157ms            ^ 2.02x faster
 655
 656         * bmalloc.xcodeproj/project.pbxproj:
 657         * bmalloc/Allocator.cpp:
 658         (bmalloc::Allocator::Allocator): Don't assume that each allocator's
 659         index corresponds with its size. Instead, use the size selection function
 660         explicitly. Now that we have XSmall, some small allocator entries are
 661         unused.
 662
 663         (bmalloc::Allocator::scavenge):
 664         (bmalloc::Allocator::log):
 665         (bmalloc::Allocator::processXSmallAllocatorLog):
 666         (bmalloc::Allocator::allocateSlowCase):
 667         * bmalloc/Allocator.h:
 668         (bmalloc::Allocator::xSmallAllocatorFor):
 669         (bmalloc::Allocator::allocateFastCase):
 670         * bmalloc/Chunk.h:
 671         * bmalloc/Deallocator.cpp:
 672         (bmalloc::Deallocator::scavenge):
 673         (bmalloc::Deallocator::processObjectLog):
 674         (bmalloc::Deallocator::deallocateSlowCase):
 675         (bmalloc::Deallocator::deallocateXSmallLine):
 676         (bmalloc::Deallocator::allocateXSmallLine):
 677         * bmalloc/Deallocator.h:
 678         (bmalloc::Deallocator::deallocateFastCase):
 679         * bmalloc/Heap.cpp:
 680         (bmalloc::Heap::scavenge):
 681         (bmalloc::Heap::scavengeXSmallPages):
 682         (bmalloc::Heap::allocateXSmallLineSlowCase):
 683         * bmalloc/Heap.h:
 684         (bmalloc::Heap::deallocateXSmallLine):
 685         (bmalloc::Heap::allocateXSmallLine):
 686         * bmalloc/LargeChunk.h:
 687         (bmalloc::LargeChunk::get):
 688         (bmalloc::LargeChunk::endTag):
 689         * bmalloc/Line.h:
 690         * bmalloc/MediumAllocator.h:
 691         (bmalloc::MediumAllocator::allocate):
 692         (bmalloc::MediumAllocator::refill):
 693         * bmalloc/ObjectType.cpp:
 694         (bmalloc::objectType):
 695         * bmalloc/ObjectType.h:
 696         (bmalloc::isXSmall):
 697         (bmalloc::isSmall):
 698         (bmalloc::isMedium):
 699         (bmalloc::isLarge):
 700         (bmalloc::isSmallOrMedium): Deleted.
 701         * bmalloc/SegregatedFreeList.h: I boiler-plate copied existing code for
 702         handling small objects. There's probably a reasonable way to share this
 703         code in the future -- I'll look into that once it's stopped changing.
 704
 705         * bmalloc/Sizes.h: Tweaked size classes to make Membuster happy. This
 706         is the main reason things got slower.
 707
 708         * bmalloc/SmallAllocator.h:
 709         (bmalloc::SmallAllocator::allocate):
 710         * bmalloc/SmallTraits.h:
 711         * bmalloc/VMHeap.cpp:
 712         (bmalloc::VMHeap::allocateXSmallChunk):
 713         * bmalloc/VMHeap.h:
 714         (bmalloc::VMHeap::allocateXSmallPage):
 715         (bmalloc::VMHeap::deallocateXSmallPage):
 716         * bmalloc/XSmallAllocator.h: Added.
 717         (bmalloc::XSmallAllocator::isNull):
 718         (bmalloc::XSmallAllocator::canAllocate):
 719         (bmalloc::XSmallAllocator::XSmallAllocator):
 720         (bmalloc::XSmallAllocator::line):
 721         (bmalloc::XSmallAllocator::allocate):
 722         (bmalloc::XSmallAllocator::objectCount):
 723         (bmalloc::XSmallAllocator::derefCount):
 724         (bmalloc::XSmallAllocator::refill):
 725         * bmalloc/XSmallChunk.h: Added.
 726         * bmalloc/XSmallLine.h: Added.
 727         * bmalloc/XSmallPage.h: Added.
 728         * bmalloc/XSmallTraits.h: Added.
 729         * bmalloc/bmalloc.h:
 730         (bmalloc::api::realloc): Boiler-plate copy, as above.
 731
 732 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
 733
 734         MallocBench should scavenge explicitly instead of waiting
 735         https://bugs.webkit.org/show_bug.cgi?id=131661
 736
 737         Reviewed by Andreas Kling.
 738
 739         Added explicit scavenge support to bmalloc. This isn't a memory win,
 740         since bmalloc's per-thread cache is so small. But it makes testing
 741         simpler.
 742
 743         * bmalloc/Allocator.cpp:
 744         (bmalloc::Allocator::~Allocator):
 745         (bmalloc::Allocator::scavenge):
 746         * bmalloc/Allocator.h:
 747         * bmalloc/Cache.cpp:
 748         (bmalloc::Cache::operator new):
 749         (bmalloc::Cache::operator delete):
 750         (bmalloc::Cache::Cache):
 751         (bmalloc::Cache::scavenge):
 752         * bmalloc/Cache.h:
 753         * bmalloc/Deallocator.cpp:
 754         (bmalloc::Deallocator::~Deallocator):
 755         (bmalloc::Deallocator::scavenge):
 756         * bmalloc/Deallocator.h: Factored existing scavenging code into helper
 757         functions, for reuse.
 758
 759         * bmalloc/Heap.cpp:
 760         (bmalloc::sleep):
 761         (bmalloc::Heap::concurrentScavenge):
 762         (bmalloc::Heap::scavenge):
 763         (bmalloc::Heap::scavengeSmallPages):
 764         (bmalloc::Heap::scavengeMediumPages):
 765         (bmalloc::Heap::scavengeLargeRanges):
 766         * bmalloc/Heap.h: Made scavenge sleep duration a parameter. Forced
 767         scavenging -- in response to a benchmark or a low memory warning --
 768         wants to complete as soon as possible, so its sleep duration is 0.
 769
 770         * bmalloc/bmalloc.h:
 771         (bmalloc::api::scavenge):
 772         * bmalloc/mbmalloc.cpp: Exported the scavenge API for MallocBench's use.
 773
 774 2014-04-14  Geoffrey Garen  <ggaren@apple.com>
 775
 776         Use 4kB pages on Mac
 777         https://bugs.webkit.org/show_bug.cgi?id=131658
 778
 779         Reviewed by Sam Weinig.
 780
 781         This reduces memory use a lot on Membuster:
 782
 783                                                                   base                      patch                                Δ
 784                 Execution Time:
 785                     reddit_memory_warning                         18ms                       17ms                   ^ 1.06x faster
 786                     flickr_memory_warning                         34ms                       36ms                   ! 1.06x slower
 787                     theverge_memory_warning                       39ms                       41ms                   ! 1.05x slower
 788
 789                     <geometric mean>                              29ms                       29ms                   ! 1.02x slower
 790                     <arithmetic mean>                             30ms                       31ms                   ! 1.03x slower
 791                     <harmonic mean>                               27ms                       27ms                    ^ 1.0x faster
 792
 793                 Peak Memory:
 794                     reddit_memory_warning                     16,412kB                   16,436kB                    ! 1.0x bigger
 795                     flickr_memory_warning                     30,120kB                   30,184kB                    ! 1.0x bigger
 796                     theverge_memory_warning                   33,408kB                   33,420kB                    ! 1.0x bigger
 797
 798                     <geometric mean>                          25,466kB                   25,499kB                    ! 1.0x bigger
 799                     <arithmetic mean>                         26,647kB                   26,680kB                    ! 1.0x bigger
 800                     <harmonic mean>                           24,181kB                   24,214kB                    ! 1.0x bigger
 801
 802                 Memory at End:
 803                     reddit_memory_warning                      2,404kB                    1,920kB                  ^ 1.25x smaller
 804                     flickr_memory_warning                      3,764kB                    3,072kB                  ^ 1.23x smaller
 805                     theverge_memory_warning                    3,648kB                    3,132kB                  ^ 1.16x smaller
 806
 807                     <geometric mean>                           3,208kB                    2,644kB                  ^ 1.21x smaller
 808                     <arithmetic mean>                          3,272kB                    2,708kB                  ^ 1.21x smaller
 809                     <harmonic mean>                            3,139kB                    2,574kB                  ^ 1.22x smaller
 810
 811
 812         * bmalloc.xcodeproj/project.pbxproj:
 813         * bmalloc/BPlatform.h: Added.
 814         * bmalloc/VMAllocate.h: Only use 16kB pages on iOS because the page size
 815         is 4kB on Mac.
 816
 817 2014-04-14  Alexey Proskuryakov  <ap@apple.com>
 818
 819         Fixed svn:ignore on bmalloc.xcodeproj, it had erroneous leading spaces.
 820
 821         * bmalloc.xcodeproj: Modified property svn:ignore.
 822
 823 2014-04-13  Geoffrey Garen  <ggaren@apple.com>
 824
 825         Fixed some mbmalloc exports
 826         https://bugs.webkit.org/show_bug.cgi?id=131599
 827
 828         Reviewed by Ryosuke Niwa.
 829
 830         * bmalloc.xcodeproj/project.pbxproj: Made some headers a private part
 831         of the project, so we can call them from API.
 832
 833         * bmalloc/mbmalloc.cpp: Marked the mbmalloc functions with default
 834         visibility, so they show up as exported in the .dylib.
 835
 836 2014-04-09  Geoffrey Garen  <ggaren@apple.com>
 837
 838         Put bmalloc headers in the right place
 839         https://bugs.webkit.org/show_bug.cgi?id=131464
 840
 841         Reviewed by Mark Rowe.
 842
 843         * Configurations/bmalloc.xcconfig: Set PRIVATE_HEADERS_FOLDER_PATH to
 844         specify that we don't just want to dump all of our generically-named
 845         headers into /usr/local/include.
 846
 847 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
 848
 849         Made bmalloc more #include friendly
 850         https://bugs.webkit.org/show_bug.cgi?id=131386
 851
 852         Reviewed by Andreas Kling.
 853
 854         Marked a bunch of headers private so they can be used from client code
 855         that #includes bmalloc.h.
 856
 857         Renamed ASSERT macros to BASSERT. This matches their header, which already
 858         had to be renamed, and fixes conflicts with WTF's ASSERT macros.
 859
 860         * bmalloc.xcodeproj/project.pbxproj:
 861         * bmalloc/Allocator.cpp:
 862         (bmalloc::Allocator::allocateSlowCase):
 863         * bmalloc/AsyncTask.h:
 864         (bmalloc::Function>::runSlowCase):
 865         * bmalloc/BAssert.h:
 866         * bmalloc/BoundaryTag.h:
 867         (bmalloc::BoundaryTag::setSize):
 868         * bmalloc/BoundaryTagInlines.h:
 869         (bmalloc::validate):
 870         (bmalloc::BoundaryTag::init):
 871         (bmalloc::BoundaryTag::deallocate):
 872         (bmalloc::BoundaryTag::splitLarge):
 873         (bmalloc::BoundaryTag::allocate):
 874         * bmalloc/Chunk.h:
 875         * bmalloc/Deallocator.cpp:
 876         (bmalloc::Deallocator::processObjectLog):
 877         (bmalloc::Deallocator::deallocateSlowCase):
 878         * bmalloc/Deallocator.h:
 879         (bmalloc::Deallocator::deallocateFastCase):
 880         * bmalloc/FixedVector.h:
 881         (bmalloc::Capacity>::operator):
 882         (bmalloc::Capacity>::push):
 883         (bmalloc::Capacity>::pop):
 884         (bmalloc::Capacity>::shrink):
 885         * bmalloc/Heap.cpp:
 886         (bmalloc::Heap::allocateLarge):
 887         * bmalloc/LargeChunk.h:
 888         (bmalloc::LargeChunk::get):
 889         (bmalloc::LargeChunk::endTag):
 890         * bmalloc/Line.h:
 891         (bmalloc::Line<Traits>::concurrentRef):
 892         (bmalloc::Line<Traits>::deref):
 893         * bmalloc/MediumAllocator.h:
 894         (bmalloc::MediumAllocator::allocate):
 895         * bmalloc/ObjectType.h:
 896         (bmalloc::isSmall):
 897         * bmalloc/Page.h:
 898         (bmalloc::Page<Traits>::ref):
 899         (bmalloc::Page<Traits>::deref):
 900         * bmalloc/PerThread.h:
 901         (bmalloc::PerThread<T>::getSlowCase):
 902         * bmalloc/SegregatedFreeList.cpp:
 903         (bmalloc::SegregatedFreeList::SegregatedFreeList):
 904         (bmalloc::SegregatedFreeList::insert):
 905         * bmalloc/SmallAllocator.h:
 906         (bmalloc::SmallAllocator::allocate):
 907         (bmalloc::SmallAllocator::refill):
 908         * bmalloc/Syscall.h:
 909         * bmalloc/VMAllocate.h:
 910         (bmalloc::vmValidate):
 911         (bmalloc::vmAllocate):
 912         (bmalloc::vmDeallocatePhysicalPagesSloppy):
 913         * bmalloc/Vector.h:
 914         (bmalloc::Vector<T>::operator):
 915         (bmalloc::Vector<T>::pop):
 916         (bmalloc::Vector<T>::shrink):
 917         * bmalloc/XLargeChunk.h:
 918         (bmalloc::XLargeChunk::range):
 919         (bmalloc::XLargeChunk::size):
 920
 921 2014-04-08  Geoffrey Garen  <ggaren@apple.com>
 922
 923         Removed an unused file.
 924
 925         Unreviewed.
 926
 927         * bmalloc/AsyncTask.cpp: Removed.
 928
 929 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
 930
 931         Build bmalloc on Mac
 932         https://bugs.webkit.org/show_bug.cgi?id=131333
 933
 934         Reviewed by Mark Rowe.
 935
 936         * Makefile: Added. For make clients.
 937
 938         These files are required for building any project in WebKit. I copied
 939         them from WTF:
 940         * Configurations: Added.
 941         * Configurations/Base.xcconfig: Added.
 942         * Configurations/DebugRelease.xcconfig: Added.
 943         * Configurations/bmalloc.xcconfig: Added.
 944         * Configurations/iOS.xcconfig: Added.
 945         * Configurations/mbmalloc.xcconfig: Added.
 946
 947         * bmalloc.xcodeproj/project.pbxproj: I removed per-project-file stuff
 948         from here because everything is in .xcconfig files now.
 949
 950         I had to fix a bunch of minor warnings, since they're enabled in our
 951         .xcconfig files:
 952
 953         * bmalloc/AsyncTask.h:
 954         (bmalloc::Function>::AsyncTask):
 955         * bmalloc/BAssert.h:
 956         * bmalloc/BoundaryTagInlines.h:
 957         (bmalloc::validate):
 958         * bmalloc/Heap.cpp:
 959         (bmalloc::Heap::Heap):
 960         (bmalloc::Heap::allocateLarge):
 961         (bmalloc::Heap::deallocateLarge):
 962         * bmalloc/Mutex.h:
 963         (bmalloc::Mutex::Mutex): Deleted.
 964         * bmalloc/VMAllocate.h:
 965         (bmalloc::vmValidate):
 966         * bmalloc/mbmalloc.cpp:
 967
 968 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
 969
 970         bmalloc: Fixed a leak in the per-thread cache
 971         https://bugs.webkit.org/show_bug.cgi?id=131330
 972
 973         Reviewed by Andreas Kling.
 974
 975         Remember to deallocate our line caches upon thread exit.
 976
 977         * bmalloc/Deallocator.cpp:
 978         (bmalloc::Deallocator::~Deallocator):
 979
 980 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
 981
 982         bmalloc: rolled out the tryLock experiment
 983         https://bugs.webkit.org/show_bug.cgi?id=131328
 984
 985         Reviewed by Andreas Kling.
 986
 987         It wasn't a speedup.
 988
 989         * bmalloc.xcodeproj/project.pbxproj:
 990         * bmalloc/Allocator.cpp:
 991         (bmalloc::Allocator::processSmallAllocatorLog):
 992         (bmalloc::Allocator::processMediumAllocatorLog):
 993         * bmalloc/Deallocator.cpp:
 994         (bmalloc::Deallocator::processObjectLog):
 995         (bmalloc::Deallocator::deallocateSlowCase):
 996         (bmalloc::Deallocator::deallocateSmallLine):
 997         (bmalloc::Deallocator::deallocateMediumLine):
 998         * bmalloc/Deallocator.h:
 999         (bmalloc::Deallocator::deallocateFastCase):
1000         * bmalloc/Heap.h:
1001         (bmalloc::Heap::deallocateSmallLine):
1002         (bmalloc::Heap::deallocateMediumLine):
1003         * bmalloc/Line.h:
1004         (bmalloc::Line<Traits>::deref):
1005         * bmalloc/Page.h:
1006         (bmalloc::Page<Traits>::deref):
1007
1008 2014-04-07  Geoffrey Garen  <ggaren@apple.com>
1009
1010         bmalloc
1011         https://bugs.webkit.org/show_bug.cgi?id=131170
1012
1013         Reviewed by Andreas Kling.
1014
1015         Initial commit.
1016
1017         * bmalloc: Added.
1018         * bmalloc.xcodeproj: Added.
1019         * bmalloc.xcodeproj/project.pbxproj: Added.
1020         * bmalloc/Algorithm.h: Added.
1021         (bmalloc::max):
1022         (bmalloc::min):
1023         (bmalloc::mask):
1024         (bmalloc::test):
1025         (bmalloc::roundUpToMultipleOf):
1026         (bmalloc::roundDownToMultipleOf):
1027         (bmalloc::sizeOf):
1028         (bmalloc::bitCount):
1029         (bmalloc::isPowerOfTwo):
1030         * bmalloc/Allocator.cpp: Added.
1031         (bmalloc::Allocator::Allocator):
1032         (bmalloc::Allocator::~Allocator):
1033         (bmalloc::Allocator::log):
1034         (bmalloc::Allocator::processSmallAllocatorLog):
1035         (bmalloc::Allocator::processMediumAllocatorLog):
1036         (bmalloc::Allocator::allocateLarge):
1037         (bmalloc::Allocator::allocateXLarge):
1038         (bmalloc::Allocator::allocateMedium):
1039         (bmalloc::Allocator::allocateSlowCase):
1040         * bmalloc/Allocator.h: Added.
1041         (bmalloc::Allocator::smallAllocatorFor):
1042         (bmalloc::Allocator::allocateFastCase):
1043         (bmalloc::Allocator::allocate):
1044         * bmalloc/AsyncTask.cpp: Added.
1045         (bmalloc::AsyncTask<Function>::runSlowCase):
1046         (bmalloc::AsyncTask<Function>::pthreadEntryPoint):
1047         (bmalloc::AsyncTask<Function>::entryPoint):
1048         * bmalloc/AsyncTask.h: Added.
1049         (bmalloc::Function>::AsyncTask):
1050         (bmalloc::Function>::join):
1051         (bmalloc::Function>::run):
1052         (bmalloc::Function>::runSlowCase):
1053         (bmalloc::Function>::pthreadEntryPoint):
1054         (bmalloc::Function>::entryPoint):
1055         * bmalloc/BAssert.h: Added.
1056         * bmalloc/BeginTag.h: Added.
1057         (bmalloc::BeginTag::isInFreeList):
1058         * bmalloc/BoundaryTag.h: Added.
1059         (bmalloc::BoundaryTag::isXLarge):
1060         (bmalloc::BoundaryTag::setXLarge):
1061         (bmalloc::BoundaryTag::isFree):
1062         (bmalloc::BoundaryTag::setFree):
1063         (bmalloc::BoundaryTag::isEnd):
1064         (bmalloc::BoundaryTag::setEnd):
1065         (bmalloc::BoundaryTag::hasPhysicalPages):
1066         (bmalloc::BoundaryTag::setHasPhysicalPages):
1067         (bmalloc::BoundaryTag::isNull):
1068         (bmalloc::BoundaryTag::clear):
1069         (bmalloc::BoundaryTag::size):
1070         (bmalloc::BoundaryTag::setSize):
1071         (bmalloc::BoundaryTag::prev):
1072         (bmalloc::BoundaryTag::next):
1073         * bmalloc/BoundaryTagInlines.h: Added.
1074         (bmalloc::validate):
1075         (bmalloc::validatePrev):
1076         (bmalloc::validateNext):
1077         (bmalloc::BoundaryTag::init):
1078         (bmalloc::BoundaryTag::mergeLargeLeft):
1079         (bmalloc::BoundaryTag::mergeLargeRight):
1080         (bmalloc::BoundaryTag::mergeLarge):
1081         (bmalloc::BoundaryTag::deallocate):
1082         (bmalloc::BoundaryTag::splitLarge):
1083         (bmalloc::BoundaryTag::allocate):
1084         * bmalloc/Cache.cpp: Added.
1085         (bmalloc::Cache::operator new):
1086         (bmalloc::Cache::operator delete):
1087         (bmalloc::Cache::Cache):
1088         (bmalloc::Cache::allocateSlowCase):
1089         (bmalloc::Cache::allocateSlowCaseNullCache):
1090         (bmalloc::Cache::deallocateSlowCase):
1091         (bmalloc::Cache::deallocateSlowCaseNullCache):
1092         * bmalloc/Cache.h: Added.
1093         (bmalloc::Cache::allocator):
1094         (bmalloc::Cache::deallocator):
1095         (bmalloc::Cache::allocateFastCase):
1096         (bmalloc::Cache::deallocateFastCase):
1097         (bmalloc::Cache::allocate):
1098         (bmalloc::Cache::deallocate):
1099         * bmalloc/Chunk.h: Added.
1100         (bmalloc::Chunk::begin):
1101         (bmalloc::Chunk::end):
1102         (bmalloc::Chunk::lines):
1103         (bmalloc::Chunk::pages):
1104         * bmalloc/Deallocator.cpp: Added.
1105         (bmalloc::Deallocator::Deallocator):
1106         (bmalloc::Deallocator::~Deallocator):
1107         (bmalloc::Deallocator::deallocateLarge):
1108         (bmalloc::Deallocator::deallocateXLarge):
1109         (bmalloc::Deallocator::processObjectLog):
1110         (bmalloc::Deallocator::deallocateSlowCase):
1111         (bmalloc::Deallocator::deallocateSmallLine):
1112         (bmalloc::Deallocator::allocateSmallLine):
1113         (bmalloc::Deallocator::deallocateMediumLine):
1114         (bmalloc::Deallocator::allocateMediumLine):
1115         * bmalloc/Deallocator.h: Added.
1116         (bmalloc::Deallocator::deallocateFastCase):
1117         (bmalloc::Deallocator::deallocate):
1118         * bmalloc/EndTag.h: Added.
1119         (bmalloc::EndTag::operator=):
1120         * bmalloc/FixedVector.h: Added.
1121         (bmalloc::FixedVector::begin):
1122         (bmalloc::FixedVector::end):
1123         (bmalloc::FixedVector::size):
1124         (bmalloc::FixedVector::capacity):
1125         (bmalloc::FixedVector::clear):
1126         (bmalloc::FixedVector::isEmpty):
1127         (bmalloc::Capacity>::FixedVector):
1128         (bmalloc::Capacity>::operator):
1129         (bmalloc::Capacity>::push):
1130         (bmalloc::Capacity>::pop):
1131         (bmalloc::Capacity>::shrink):
1132         * bmalloc/Heap.cpp: Added.
1133         (bmalloc::sleep):
1134         (bmalloc::Heap::Heap):
1135         (bmalloc::Heap::concurrentScavenge):
1136         (bmalloc::Heap::scavengeSmallPages):
1137         (bmalloc::Heap::scavengeMediumPages):
1138         (bmalloc::Heap::scavengeLargeRanges):
1139         (bmalloc::Heap::allocateSmallLineSlowCase):
1140         (bmalloc::Heap::allocateMediumLineSlowCase):
1141         (bmalloc::Heap::allocateXLarge):
1142         (bmalloc::Heap::deallocateXLarge):
1143         (bmalloc::Heap::allocateLarge):
1144         (bmalloc::Heap::deallocateLarge):
1145         * bmalloc/Heap.h: Added.
1146         (bmalloc::Heap::deallocateSmallLine):
1147         (bmalloc::Heap::allocateSmallLine):
1148         (bmalloc::Heap::deallocateMediumLine):
1149         (bmalloc::Heap::allocateMediumLine):
1150         * bmalloc/Inline.h: Added.
1151         * bmalloc/LargeChunk.h: Added.
1152         (bmalloc::LargeChunk::begin):
1153         (bmalloc::LargeChunk::end):
1154         (bmalloc::LargeChunk::create):
1155         (bmalloc::LargeChunk::get):
1156         (bmalloc::LargeChunk::beginTag):
1157         (bmalloc::LargeChunk::endTag):
1158         * bmalloc/Line.h: Added.
1159         (bmalloc::Line<Traits>::begin):
1160         (bmalloc::Line<Traits>::end):
1161         (bmalloc::Line<Traits>::concurrentRef):
1162         (bmalloc::Line<Traits>::deref):
1163         * bmalloc/MediumAllocator.h: Added.
1164         (bmalloc::MediumAllocator::isNull):
1165         (bmalloc::MediumAllocator::MediumAllocator):
1166         (bmalloc::MediumAllocator::line):
1167         (bmalloc::MediumAllocator::allocate):
1168         (bmalloc::MediumAllocator::derefCount):
1169         (bmalloc::MediumAllocator::refill):
1170         * bmalloc/MediumChunk.h: Added.
1171         * bmalloc/MediumLine.h: Added.
1172         * bmalloc/MediumPage.h: Added.
1173         * bmalloc/MediumTraits.h: Added.
1174         * bmalloc/Mutex.cpp: Added.
1175         (bmalloc::Mutex::lockSlowCase):
1176         * bmalloc/Mutex.h: Added.
1177         (bmalloc::Mutex::Mutex):
1178         (bmalloc::Mutex::try_lock):
1179         (bmalloc::Mutex::lock):
1180         (bmalloc::Mutex::unlock):
1181         * bmalloc/ObjectType.cpp: Added.
1182         (bmalloc::objectType):
1183         * bmalloc/ObjectType.h: Added.
1184         (bmalloc::isSmallOrMedium):
1185         (bmalloc::isSmall):
1186         * bmalloc/Page.h: Added.
1187         (bmalloc::Page<Traits>::ref):
1188         (bmalloc::Page<Traits>::deref):
1189         (bmalloc::Page<Traits>::refCount):
1190         * bmalloc/PerProcess.h: Added.
1191         (bmalloc::PerProcess::mutex):
1192         (bmalloc::PerProcess<T>::getFastCase):
1193         (bmalloc::PerProcess<T>::get):
1194         (bmalloc::PerProcess<T>::getSlowCase):
1195         * bmalloc/PerThread.h: Added.
1196         (bmalloc::PerThreadStorage<Cache>::get):
1197         (bmalloc::PerThreadStorage<Cache>::init):
1198         (bmalloc::PerThreadStorage::get):
1199         (bmalloc::PerThreadStorage::init):
1200         (bmalloc::PerThread<T>::getFastCase):
1201         (bmalloc::PerThread<T>::get):
1202         (bmalloc::PerThread<T>::destructor):
1203         (bmalloc::PerThread<T>::getSlowCase):
1204         * bmalloc/Range.h: Added.
1205         (bmalloc::Range::Range):
1206         (bmalloc::Range::begin):
1207         (bmalloc::Range::end):
1208         (bmalloc::Range::size):
1209         (bmalloc::Range::operator!):
1210         (bmalloc::Range::operator<):
1211         * bmalloc/SegregatedFreeList.cpp: Added.
1212         (bmalloc::SegregatedFreeList::SegregatedFreeList):
1213         (bmalloc::SegregatedFreeList::insert):
1214         (bmalloc::SegregatedFreeList::takeGreedy):
1215         (bmalloc::SegregatedFreeList::take):
1216         * bmalloc/SegregatedFreeList.h: Added.
1217         * bmalloc/Sizes.h: Added.
1218         * bmalloc/SmallAllocator.h: Added.
1219         (bmalloc::SmallAllocator::isNull):
1220         (bmalloc::SmallAllocator::canAllocate):
1221         (bmalloc::SmallAllocator::SmallAllocator):
1222         (bmalloc::SmallAllocator::line):
1223         (bmalloc::SmallAllocator::allocate):
1224         (bmalloc::SmallAllocator::objectCount):
1225         (bmalloc::SmallAllocator::derefCount):
1226         (bmalloc::SmallAllocator::refill):
1227         * bmalloc/SmallChunk.h: Added.
1228         * bmalloc/SmallLine.h: Added.
1229         * bmalloc/SmallPage.h: Added.
1230         * bmalloc/SmallTraits.h: Added.
1231         * bmalloc/Syscall.h: Added.
1232         * bmalloc/VMAllocate.h: Added.
1233         (bmalloc::vmSize):
1234         (bmalloc::vmValidate):
1235         (bmalloc::vmAllocate):
1236         (bmalloc::vmDeallocate):
1237         (bmalloc::vmDeallocatePhysicalPages):
1238         (bmalloc::vmAllocatePhysicalPages):
1239         (bmalloc::vmDeallocatePhysicalPagesSloppy):
1240         (bmalloc::vmAllocatePhysicalPagesSloppy):
1241         * bmalloc/VMHeap.cpp: Added.
1242         (bmalloc::VMHeap::VMHeap):
1243         (bmalloc::VMHeap::allocateSmallChunk):
1244         (bmalloc::VMHeap::allocateMediumChunk):
1245         (bmalloc::VMHeap::allocateLargeChunk):
1246         * bmalloc/VMHeap.h: Added.
1247         (bmalloc::VMHeap::allocateSmallPage):
1248         (bmalloc::VMHeap::allocateMediumPage):
1249         (bmalloc::VMHeap::allocateLargeRange):
1250         (bmalloc::VMHeap::deallocateSmallPage):
1251         (bmalloc::VMHeap::deallocateMediumPage):
1252         (bmalloc::VMHeap::deallocateLargeRange):
1253         * bmalloc/Vector.h: Added.
1254         (bmalloc::Vector::begin):
1255         (bmalloc::Vector::end):
1256         (bmalloc::Vector::size):
1257         (bmalloc::Vector::capacity):
1258         (bmalloc::Vector::last):
1259         (bmalloc::Vector::pop):
1260         (bmalloc::Vector<T>::Vector):
1261         (bmalloc::Vector<T>::~Vector):
1262         (bmalloc::Vector<T>::operator):
1263         (bmalloc::Vector<T>::push):
1264         (bmalloc::Vector<T>::pop):
1265         (bmalloc::Vector<T>::shrink):
1266         (bmalloc::Vector<T>::reallocateBuffer):
1267         (bmalloc::Vector<T>::shrinkCapacity):
1268         (bmalloc::Vector<T>::growCapacity):
1269         * bmalloc/XLargeChunk.h: Added.
1270         (bmalloc::XLargeChunk::get):
1271         (bmalloc::XLargeChunk::begin):
1272         (bmalloc::XLargeChunk::XLargeChunk):
1273         (bmalloc::XLargeChunk::create):
1274         (bmalloc::XLargeChunk::destroy):
1275         (bmalloc::XLargeChunk::range):
1276         (bmalloc::XLargeChunk::size):
1277         * bmalloc/bmalloc.h: Added.
1278         (bmalloc::api::malloc):
1279         (bmalloc::api::free):
1280         (bmalloc::api::realloc):
1281         * bmalloc/mbmalloc.cpp: Added.
1282