PACCage should first cage leaving PAC bits intact then authenticate

[WebKit-https.git] / Source / bmalloc / ChangeLog

2019-07-02  Keith Miller  <keith_miller@apple.com>

        PACCage should first cage leaving PAC bits intact then authenticate
        https://bugs.webkit.org/show_bug.cgi?id=199372

        Reviewed by Saam Barati.

        * bmalloc/ProcessCheck.mm:
        (bmalloc::shouldProcessUnconditionallyUseBmalloc):

2019-06-27  Beth Dakin  <bdakin@apple.com>

        Upstream use of MACCATALYST
        https://bugs.webkit.org/show_bug.cgi?id=199245
        rdar://problem/51687723

        Reviewed by Tim Horton.

        * Configurations/SDKVariant.xcconfig:

2019-06-24  Yusuke Suzuki  <ysuzuki@apple.com>

        [bmalloc] IsoHeap shared tier threshold is small
        https://bugs.webkit.org/show_bug.cgi?id=199145

        Reviewed by Saam Barati.

        We accidentally picked 3 for the shared tier threshold. While this is OK because the important part of IsoHeap shared tier is putting
        small # of shared objects before tiering up to the page-based allocation, increasing this threshold can potentially improve memory footprint.
        This patch increases this threshold from 3 to 8. A/B test says that this offers stable 0.25% improvement in PLUM2.

        * bmalloc/IsoHeapImpl.h:

2019-06-21  Yusuke Suzuki  <ysuzuki@apple.com>

        [bmalloc] IsoTLS Layout extension initializes one IsoTLSEntry twice
        https://bugs.webkit.org/show_bug.cgi?id=199077

        Reviewed by Saam Barati.

        Found that IsoTLS::ensureEntries can construct the same IsoTLSEntry many times, it can leak memory because the construction clears previous fields including freelist.

        1. We have oldLastEntry.
        2. In that case, startEntry is oldLastEntry.
        3. We find some targetEntry.
        4. Finally, if startEntry exists, we newly construct [startEntry, targetEntry]
        5. In the above sequence, oldLastEntry (== startEntry) is constructed again, while oldLastEntry is already constructed previously.

        We fix this issue by changing the startEntry. We already have `RELEASE_BASSERT(!oldLastEntry || oldLastEntry->offset() < offset);`
        assertion. This means that `oldLastEntry->m_next` must exist, otherwise the following loop would not find a `targetEntry`. And `layout.head()`
        must return non nullptr at `IsoTLS::ensureEntries` because `IsoTLS::ensureEntries` requires that `IsoHeap<>` is initialized, and `IsoHeap<>`
        must add at least one TLS entry to the IsoTLSLayout.

        * bmalloc/IsoTLS.cpp:
        (bmalloc::IsoTLS::ensureEntries):

2019-06-19  Yusuke Suzuki  <ysuzuki@apple.com>

        [bmalloc] IsoHeap's initialization is racy with IsoHeap::isInitialized
        https://bugs.webkit.org/show_bug.cgi?id=199053

        Reviewed by Saam Barati.

        IsoHeap's initialization code is racy. Let's see the `isInitialized` and the initialization code.

        isInitialized:

            template<typename Type>
            bool IsoHeap<Type>::isInitialized()
            {
                std::atomic<unsigned>* atomic =
                    reinterpret_cast<std::atomic<unsigned>*>(&m_allocatorOffsetPlusOne);
                return !!atomic->load(std::memory_order_acquire);
            }

        initialization:

            if (!handle.isInitialized()) {
                std::lock_guard<Mutex> locker(handle.m_initializationLock);
                if (!handle.isInitialized()) {
                    auto* heap = new IsoHeapImpl<typename api::IsoHeap<Type>::Config>();
                    std::atomic_thread_fence(std::memory_order_seq_cst);
                    handle.setAllocatorOffset(heap->allocatorOffset());                  // <================= (1)
                    handle.setDeallocatorOffset(heap->deallocatorOffset());              // (2)
                    handle.m_impl = heap;
                }
            }

        IsoHeap::isInitialized is loading m_allocatorOffsetPlusOne with acquire fence. On the other hand, the initialization
        code configures m_allocatorOffsetPlusOne (1) before configuring m_deallocatorOffsetPlusOne (2). Let's consider the following
        case.

            1. Thread A is at (1)
            2. Thread B calls handle.isInitialized(). Then B think that handle is already initialized while it lacks m_deallocatorOffsetPlusOne and m_impl pointer.
            3. Thread B uses this handle, and does `std::max(handle.allocatorOffset(), handle.deallocatorOffset())`. But m_deallocatorOffsetPlusOne is not configured
               yet. As a result, deallocatorOffset() returns 0xffffffff (b/c it calculates m_deallocatorOffsetPlusOne - 1, and m_deallocatorOffsetPlusOne is first
               zero-initialized before IsoHeap initialization happens).
            4. std::max returns 0xffffffff as an offset. Of course, this is wrong, and leading to the release assertion.

        This patch fixes the above issue by,

            1. Add IsoHeap::initialize() function instead of initializing it in IsoTLS
            2. Change `isInitialized()` function to load m_impl pointer instead of m_allocatorOffsetPlusOne with acquire fence.
            3. In initialize() function, we store m_heap with release fence at last.

        * bmalloc/IsoHeap.h:
        * bmalloc/IsoHeapInlines.h:
        (bmalloc::api::IsoHeap<Type>::isInitialized):
        (bmalloc::api::IsoHeap<Type>::initialize):
        * bmalloc/IsoTLSInlines.h:
        (bmalloc::IsoTLS::ensureHeap):

2019-06-14  Keith Miller  <keith_miller@apple.com>

        Restore PAC based cage.
        https://bugs.webkit.org/show_bug.cgi?id=198872

        Rubber-stamped by Saam Barati.

        * bmalloc/Gigacage.h:

2019-06-12  Commit Queue  <commit-queue@webkit.org>

        Unreviewed, rolling out r246322.
        https://bugs.webkit.org/show_bug.cgi?id=198796

        "It's a huge page load regression on iOS" (Requested by
        saamyjoon on #webkit).

        Reverted changeset:

        "Roll out PAC cage"
        https://bugs.webkit.org/show_bug.cgi?id=198726
        https://trac.webkit.org/changeset/246322

2019-06-11  Saam Barati  <sbarati@apple.com>

        Roll out PAC cage
        https://bugs.webkit.org/show_bug.cgi?id=198726

        Reviewed by Keith Miller.

        * bmalloc/Gigacage.h:
        (Gigacage::isEnabled):
        (Gigacage::caged):
        (Gigacage::cagedMayBeNull): Deleted.

2019-06-09  Commit Queue  <commit-queue@webkit.org>

        Unreviewed, rolling out r246150, r246160, and r246166.
        https://bugs.webkit.org/show_bug.cgi?id=198698

        Regresses page loading time on iOS 13 (Requested by keith_m__
        on #webkit).

        Reverted changesets:

        "Reenable Gigacage on ARM64."
        https://bugs.webkit.org/show_bug.cgi?id=198453
        https://trac.webkit.org/changeset/246150

        "Unrevied build fix for FTL without Gigacage."
        https://trac.webkit.org/changeset/246160

        "Fix typo in cageWithoutUntagging"
        https://bugs.webkit.org/show_bug.cgi?id=198617
        https://trac.webkit.org/changeset/246166

2019-06-06  Keith Miller  <keith_miller@apple.com>

        Reenable Gigacage on ARM64.
        https://bugs.webkit.org/show_bug.cgi?id=198453

        Reviewed by Michael Saboff.

        * bmalloc/Gigacage.h:

2019-06-03  Commit Queue  <commit-queue@webkit.org>

        Unreviewed, rolling out r246022.
        https://bugs.webkit.org/show_bug.cgi?id=198486

        Causing Internal build failures and JSC test failures
        (Requested by ShawnRoberts on #webkit).

        Reverted changeset:

        "Reenable Gigacage on ARM64."
        https://bugs.webkit.org/show_bug.cgi?id=198453
        https://trac.webkit.org/changeset/246022

2019-06-02  Keith Miller  <keith_miller@apple.com>

        Reenable Gigacage on ARM64.
        https://bugs.webkit.org/show_bug.cgi?id=198453

        Reviewed by Filip Pizlo.

        * bmalloc/Gigacage.h:

2019-05-30  Don Olmstead  <don.olmstead@sony.com>

        [CMake] Add WEBKIT_FRAMEWORK_TARGET macro
        https://bugs.webkit.org/show_bug.cgi?id=198396

        Reviewed by Konstantin Tokarev.

        Use WEBKIT_FRAMEWORK_TARGET.

        * CMakeLists.txt:

2019-05-30  Keith Miller  <keith_miller@apple.com>

        IsoHeaps don't notice uncommitted VA becoming the first eligible.
        https://bugs.webkit.org/show_bug.cgi?id=198301

        Reviewed by Yusuke Suzuki.

        IsoDirectory has a firstEligible member that is used as an
        optimization to help find the first fit. However if the scavenger
        decommitted a page before firstEligible then we wouldn't move
        firstEligible. Thus, if no space is ever freed below firstEligible
        we will never reused the decommitted memory (e.g. if the VA page
        is decommitted). The fix is to make IsoDirectory::didDecommit move
        the firstEligible page back if the decommitted page is smaller
        than the current firstEligible. As such, this patch renames
        firstEligible to firstEligibleOrDecommitted.

        Also, this patch changes gigacageEnabledForProcess to check if the
        process starts with Test rather than just test as TestWTF does.

        Lastly, unbeknownst to me IsoHeaps are dependent on gigacage, so
        by removing gigacage from arm64 I accidentally disabled
        IsoHeaps...

        * bmalloc.xcodeproj/project.pbxproj:
        * bmalloc/IsoDirectory.h:
        * bmalloc/IsoDirectoryInlines.h:
        (bmalloc::passedNumPages>::takeFirstEligible):
        (bmalloc::passedNumPages>::didBecome):
        (bmalloc::passedNumPages>::didDecommit):
        * bmalloc/IsoHeapImpl.h:
        * bmalloc/IsoHeapImplInlines.h:
        (bmalloc::IsoHeapImpl<Config>::takeFirstEligible):
        (bmalloc::IsoHeapImpl<Config>::didBecomeEligibleOrDecommited):
        (bmalloc::IsoHeapImpl<Config>::didCommit):
        (bmalloc::IsoHeapImpl<Config>::didBecomeEligible): Deleted.
        * bmalloc/IsoTLS.cpp:
        (bmalloc::IsoTLS::determineMallocFallbackState):
        * bmalloc/ProcessCheck.mm:
        (bmalloc::gigacageEnabledForProcess):

2019-05-23  Don Olmstead  <don.olmstead@sony.com>

        [CMake] Use target oriented design for bmalloc
        https://bugs.webkit.org/show_bug.cgi?id=198046

        Reviewed by Konstantin Tokarev.

        Switch to a target oriented dsign for bmalloc. Use target_include_directories directly
        instead of include_directories.

        List the headers for bmalloc and copy them using WEBKIT_COPY_FILES.

        Add an intermediate target bmalloc_PostBuild which depends on bmalloc and the headers
        being copied. Then alias that to WebKit::bmalloc.

        * CMakeLists.txt:
        * PlatformMac.cmake:

2019-05-16  Keith Miller  <keith_miller@apple.com>

        Wasm should cage the memory base pointers in structs
        https://bugs.webkit.org/show_bug.cgi?id=197620

        Reviewed by Saam Barati.

        Fix signature to take Gigacage::Kind, which matches GIGACAGE_ENABLED build.

        * bmalloc/Gigacage.h:
        (Gigacage::isEnabled):

2019-05-08  Keith Miller  <keith_miller@apple.com>

        Remove Gigacage from arm64 and use PAC for arm64e instead
        https://bugs.webkit.org/show_bug.cgi?id=197110

        Reviewed by Saam Barati.

        Stop using gigacage on arm64 and add a new cage function cagedMayBeNull that is the same as
        cage but returns a nullptr if the incoming pointer is already null.

        * bmalloc/Gigacage.h:
        (Gigacage::cagedMayBeNull):

2019-04-29  Alex Christensen  <achristensen@webkit.org>

        <rdar://problem/50299396> Fix internal High Sierra build
        https://bugs.webkit.org/show_bug.cgi?id=197388

        * Configurations/Base.xcconfig:

2019-04-25  Yusuke Suzuki  <ysuzuki@apple.com>

        [bmalloc] Follow-up and fixing bug after r244481
        https://bugs.webkit.org/show_bug.cgi?id=197294

        Reviewed by Saam Barati.

        This patch includes follow-up after r244481 and bug fixes which is introduced in the refactoring.

        * bmalloc/IsoAllocator.h: Remove unused function.
        * bmalloc/IsoAllocatorInlines.h:
        (bmalloc::IsoAllocator<Config>::allocateSlow):
        * bmalloc/IsoDeallocatorInlines.h:
        (bmalloc::IsoDeallocator<Config>::deallocate):
        * bmalloc/IsoHeapImpl.h: Rename m_usableBits to m_availableShared and add static_assert.
        * bmalloc/IsoHeapImplInlines.h: Do not clear m_numberOfAllocationsFromSharedInOneCycle etc. in scavenge since IsoHeapImpl::scavenge
        is not related to thread-local IsoAllocator's status.
        (bmalloc::IsoHeapImpl<Config>::scavenge):
        (bmalloc::IsoHeapImpl<Config>::forEachLiveObject):
        (bmalloc::IsoHeapImpl<Config>::updateAllocationMode): Update m_allocationMode correctly.
        (bmalloc::IsoHeapImpl<Config>::allocateFromShared):
        * bmalloc/IsoSharedHeapInlines.h:
        (bmalloc::computeObjectSizeForSharedCell):
        (bmalloc::IsoSharedHeap::allocateNew):
        (bmalloc::IsoSharedHeap::allocateSlow): Add computeObjectSizeForSharedCell.
        * bmalloc/IsoSharedPage.h:
        * bmalloc/IsoSharedPageInlines.h:
        (bmalloc::IsoSharedPage::free): Pass `const std::lock_guard<Mutex>&` in its parameter.

2019-04-25  Alex Christensen  <achristensen@webkit.org>

        Start using C++17
        https://bugs.webkit.org/show_bug.cgi?id=197131

        Reviewed by Darin Adler.

        * Configurations/Base.xcconfig:

2019-04-24  Yusuke Suzuki  <ysuzuki@apple.com>

        Unreviewed, fix typo in r244481
        https://bugs.webkit.org/show_bug.cgi?id=196837

        * bmalloc/IsoHeapImplInlines.h:
        (bmalloc::IsoHeapImpl<Config>::allocateFromShared):

2019-04-21  Yusuke Suzuki  <ysuzuki@apple.com>

        [bmalloc] Use StaticPerProcess' mutex as bmalloc::Heap does with PerProcess
        https://bugs.webkit.org/show_bug.cgi?id=197135

        Reviewed by Darin Adler.

        This patch leverages StaticPerProcess::mutex() for per process instance's lock in various classes,
        as Heap does with PerProcess::mutex().

        * bmalloc/AllIsoHeaps.cpp:
        (bmalloc::AllIsoHeaps::add):
        (bmalloc::AllIsoHeaps::head):
        * bmalloc/AllIsoHeaps.h:
        * bmalloc/CryptoRandom.cpp:
        (bmalloc::ARC4RandomNumberGenerator::randomValues):
        * bmalloc/DebugHeap.cpp:
        (bmalloc::DebugHeap::memalignLarge):
        (bmalloc::DebugHeap::freeLarge):
        * bmalloc/DebugHeap.h:
        * bmalloc/Scavenger.cpp:
        (bmalloc::Scavenger::run):
        (bmalloc::Scavenger::runSoon):
        (bmalloc::Scavenger::scheduleIfUnderMemoryPressure):
        (bmalloc::Scavenger::schedule):
        (bmalloc::Scavenger::timeSinceLastFullScavenge):
        (bmalloc::Scavenger::scavenge):
        (bmalloc::Scavenger::threadRunLoop):
        * bmalloc/Scavenger.h:

2019-04-19  Yusuke Suzuki  <ysuzuki@apple.com>

        [bmalloc] IsoHeap should have lower tier using shared IsoPage
        https://bugs.webkit.org/show_bug.cgi?id=196837

        Reviewed by Filip Pizlo.

        IsoHeap had a scalability problem. Once one instance is allocated from IsoHeap, it immediately allocates 16KB page for this type.
        But some types allocate only a few instances. It leads to memory wastage, and it also limits the scalability of IsoHeap since
        we need to carefully select classes which will be confined in IsoHeap due to this characteristics. If we can remove this wastage,
        we can apply IsoHeap more aggressively without causing memory regression, this is the goal of this patch.

        In this patch, we introduce a slow tier to IsoHeap allocation. Initially, the allocator for a certain type allocates instances from
        a shared page with the other allocators, and eventually, the allocator tiers up and gets dedicated pages if instances of the type
        are allocated a lot. This "shared" tier is slow, but it is totally OK because we will tier up to the normal fast tier if allocation
        frequently happens. Even the instance is allocated from pages shared with the other allocators, we still make the allocated memory
        region dedicated to the specific type: once a memory region is allocated for a certain type from a shared page, this region continues
        being used only for this type even after this memory is freed. To summarize the changes:

        1. We introduce "shared" tier to IsoHeap allocation. Up to N (N = 8 for now, but we can pick any power-of-two numbers up to 32) allocations,
           we continue using this tier. We allocate memory from shared pages so that we do not waste 16KB pages for types which only allocates a few instances.

        2. We eventually tier up to the "fast" tier, and eventually tier down to the "shared" tier too. We measure the period between slow paths,
           and switch the appropriate tier for the type. Currently, we use 1 seconds as heuristics. We also count # of allocations per cycle to
           avoid pathological slow downs.

        3. Shared page mechanism must keep the characteristics of IsoHeap. Once a memory region is allocated for a certain type, this memory region
           must be dedicated to this type. We keep track the allocated memory regions from shared pages in IsoHeapImpl, and ensure that we never
           reuse a memory region for a different type.

        This patch improves PLUM2 by 1.4% (128.4MB v.s. 126.62MB), and early Speedometer2 results are performance-neutral.

        * CMakeLists.txt:
        * bmalloc.xcodeproj/project.pbxproj:
        * bmalloc/Algorithm.h:
        (bmalloc::roundUpToMultipleOfImpl):
        (bmalloc::roundUpToMultipleOf):
        * bmalloc/BCompiler.h:
        * bmalloc/BExport.h:
        * bmalloc/FreeList.h:
        * bmalloc/IsoAllocator.h:
        * bmalloc/IsoAllocatorInlines.h:
        (bmalloc::IsoAllocator<Config>::allocateSlow):
        * bmalloc/IsoDeallocator.h:
        * bmalloc/IsoDeallocatorInlines.h:
        (bmalloc::IsoDeallocator<Config>::deallocate):
        * bmalloc/IsoHeapImpl.h:
        * bmalloc/IsoHeapImplInlines.h:
        (bmalloc::IsoHeapImpl<Config>::scavenge):
        (bmalloc::IsoHeapImpl<Config>::forEachLiveObject):
        (bmalloc::IsoHeapImpl<Config>::updateAllocationMode):
        (bmalloc::IsoHeapImpl<Config>::allocateFromShared):
        * bmalloc/IsoPage.h:
        (bmalloc::IsoPageBase::IsoPageBase):
        (bmalloc::IsoPageBase::isShared const):
        * bmalloc/IsoPageInlines.h:
        (bmalloc::IsoPage<Config>::IsoPage):
        (bmalloc::IsoPageBase::pageFor):
        (bmalloc::IsoPage<Config>::pageFor):
        (bmalloc::IsoPage<Config>::free):
        * bmalloc/IsoSharedConfig.h: Copied from Source/bmalloc/bmalloc/BExport.h.
        * bmalloc/IsoSharedHeap.cpp: Copied from Source/bmalloc/bmalloc/BExport.h.
        * bmalloc/IsoSharedHeap.h: Copied from Source/bmalloc/bmalloc/IsoAllocator.h.
        (bmalloc::VariadicBumpAllocator::VariadicBumpAllocator):
        (bmalloc::IsoSharedHeap::IsoSharedHeap):
        * bmalloc/IsoSharedHeapInlines.h: Added.
        (bmalloc::VariadicBumpAllocator::allocate):
        (bmalloc::IsoSharedHeap::allocateNew):
        (bmalloc::IsoSharedHeap::allocateSlow):
        * bmalloc/IsoSharedPage.cpp: Copied from Source/bmalloc/bmalloc/BExport.h.
        (bmalloc::IsoSharedPage::tryCreate):
        * bmalloc/IsoSharedPage.h: Copied from Source/bmalloc/bmalloc/IsoDeallocator.h.
        (bmalloc::IsoSharedPage::IsoSharedPage):
        (bmalloc::indexSlotFor):
        * bmalloc/IsoSharedPageInlines.h: Added.
        (bmalloc::IsoSharedPage::free):
        (bmalloc::IsoSharedPage::startAllocating):
        (bmalloc::IsoSharedPage::stopAllocating):
        * bmalloc/IsoTLS.h:
        * bmalloc/IsoTLSInlines.h:
        (bmalloc::IsoTLS::deallocateImpl):
        (bmalloc::IsoTLS::deallocateFast):
        (bmalloc::IsoTLS::deallocateSlow):
        * bmalloc/StdLibExtras.h:
        (bmalloc::bitwise_cast):
        * test/testbmalloc.cpp:
        (testIsoMallocAndFreeFast):
        (run):

2019-04-18  Yusuke Suzuki  <ysuzuki@apple.com>

        Unreviewed, fix build failure
        https://bugs.webkit.org/show_bug.cgi?id=195938

        Including <array>.

        * bmalloc/AvailableMemory.cpp:

2019-04-15  Yoshiaki Jitsukawa  <yoshiaki.jitsukawa@sony.com>

        Unreviewed. Build fix after r244244.

        * bmalloc/AvailableMemory.cpp:

2019-04-13  Zan Dobersek  <zdobersek@igalia.com>

        [bmalloc][Linux] Add support for memory status calculation
        https://bugs.webkit.org/show_bug.cgi?id=195938

        Reviewed by Carlos Garcia Campos.

        Memory status and under-memory-pressure capabilities in bmalloc can be
        implemented on Linux by reading and parsing the statm file under the
        proc filesystem.

        We retrieve the resident set size from the statm file and multiply it
        with the page size. This gives an upper-bound estimate of the memory
        that's being consumed by the process.

        The statm-based estimate seems preferable to other alternatives. One
        such alternative would be reading and parsing more-detailed smaps file,
        also exposed under the proc filesystem. This is at the moment being done
        in WTF's MemoryFootprint implementation for Linux systems, but on Linux
        ports this operation is being throttled to only execute once per second
        because of the big computing expense required to read and parse out the
        data. A future MemoryFootprint implementation could simply retrieve the
        memory footprint value from bmalloc.

        Another alternative is the Linux taskstats interface. This one would
        require utilizing a netlink socket to retrieve the necessary statistics,
        but it requires the process to have elevated privileges, which is a
        blocker.

        * bmalloc/AvailableMemory.cpp:
        (bmalloc::LinuxMemory::singleton):
        (bmalloc::LinuxMemory::footprint const):
        (bmalloc::computeAvailableMemory):
        (bmalloc::memoryStatus):
        * bmalloc/AvailableMemory.h:
        (bmalloc::isUnderMemoryPressure):
        * bmalloc/bmalloc.h:

2019-04-04  Yusuke Suzuki  <ysuzuki@apple.com>

        [WebCore] Put most of derived classes of ScriptWrappable into IsoHeap
        https://bugs.webkit.org/show_bug.cgi?id=196475

        Reviewed by Saam Barati.

        Add MAKE_BISO_MALLOCED_IMPL_TEMPLATE, which can be used for explicit specialization for template classes.

        * bmalloc/IsoHeap.h:
        * bmalloc/IsoHeapInlines.h:

2019-03-22  Keith Rollin  <krollin@apple.com>

        Enable ThinLTO support in Production builds
        https://bugs.webkit.org/show_bug.cgi?id=190758
        <rdar://problem/45413233>

        Reviewed by Daniel Bates.

        Enable building with Thin LTO in Production when using Xcode 10.2 or
        later. This change results in a 1.45% progression in PLT5. Full
        Production build times increase about 2-3%. Incremental build times
        are more severely affected, and so LTO is not enabled for local
        engineering builds.

        LTO is enabled only on macOS for now, until rdar://problem/49013399,
        which affects ARM builds, is fixed.

        To change the LTO setting when building locally:

        - If building with `make`, specify WK_LTO_MODE={none,thin,full} on the
          command line.
        - If building with `build-webkit`, specify --lto-mode={none,thin,full}
          on the command line.
        - If building with `build-root`, specify --lto={none,thin,full} on the
          command line.
        - If building with Xcode, create a LocalOverrides.xcconfig file at the
          top level of your repository directory (if needed) and define
          WK_LTO_MODE to full, thin, or none.

        * Configurations/Base.xcconfig:

2019-03-21  Michael Saboff  <msaboff@apple.com>

        [BMalloc] No need to delay deallocating chunks based on recent use
        https://bugs.webkit.org/show_bug.cgi?id=196121

        Reviewed by Mark Lam.

        The "used since last scavenge" logic is not needed for small chunks since their memory isn't decommitted directly.
        We can deallocate small chunks immediately as that adds them to the LargeRange free list.  That free list employs the
        "used since last scavenge" logic before the scavenger decommits the backing memory.

        * bmalloc/Chunk.h:
        (bmalloc::Chunk::usedSinceLastScavenge): Deleted.
        (bmalloc::Chunk::clearUsedSinceLastScavenge): Deleted.
        (bmalloc::Chunk::setUsedSinceLastScavenge): Deleted.
        * bmalloc/Heap.cpp:
        (bmalloc::Heap::scavenge):
        (bmalloc::Heap::allocateSmallPage):

2019-03-21  Brady Eidson  <beidson@apple.com>

        Certain WebProcesses should opt-out of the freezer.
        <rdar://problem/42846139> and https://bugs.webkit.org/show_bug.cgi?id=196062

        Reviewed by Andy Estes.

        * bmalloc.xcodeproj/project.pbxproj:
        * bmalloc/darwin/MemoryStatusSPI.h:

2019-03-19  Michael Catanzaro  <mcatanzaro@igalia.com>

        Unreviewed, fix -Wformat warning
        https://bugs.webkit.org/show_bug.cgi?id=195895
        <rdar://problem/48517629>

        * bmalloc/Scavenger.cpp:
        (bmalloc::Scavenger::threadRunLoop):

2019-03-18  Michael Saboff  <msaboff@apple.com>

        [BMalloc] Scavenger should react to recent memory activity
        https://bugs.webkit.org/show_bug.cgi?id=195895

        Reviewed by Geoffrey Garen.

        This change adds a recently used bit to objects that are scavenged.  When an object is allocated, that bit is set.
        When we scavenge, if the bit is set, we clear it.  If the bit was already clear, we decommit the object.  The timing
        to scavenging has been changed as well.  We perform our first scavne almost immediately after bmalloc is initialized
        (10ms later).  Subsequent scavenging is done as a multiple of the time it took to scavenge.  We bound this computed
        time between a minimum and maximum.  Through empirical testing, the multiplier, minimum and maximum are
        150x, 100ms and 10,000ms respectively.  For mini-mode, when the JIT is disabled, we use much more aggressive values of
        50x, 25ms and 500ms.

        Eliminated partial scavenging since this change allows for any scavenge to be partial or full based on recent use of
        the objects on the various free lists.

        * bmalloc/Chunk.h:
        (bmalloc::Chunk::usedSinceLastScavenge):
        (bmalloc::Chunk::clearUsedSinceLastScavenge):
        (bmalloc::Chunk::setUsedSinceLastScavenge):
        * bmalloc/Heap.cpp:
        (bmalloc::Heap::scavenge):
        (bmalloc::Heap::allocateSmallChunk):
        (bmalloc::Heap::allocateSmallPage):
        (bmalloc::Heap::splitAndAllocate):
        (bmalloc::Heap::tryAllocateLarge):
        (bmalloc::Heap::scavengeToHighWatermark): Deleted.
        * bmalloc/Heap.h:
        * bmalloc/IsoDirectory.h:
        * bmalloc/IsoDirectoryInlines.h:
        (bmalloc::passedNumPages>::takeFirstEligible):
        (bmalloc::passedNumPages>::scavenge):
        (bmalloc::passedNumPages>::scavengeToHighWatermark): Deleted.
        * bmalloc/IsoHeapImpl.h:
        * bmalloc/IsoHeapImplInlines.h:
        (bmalloc::IsoHeapImpl<Config>::scavengeToHighWatermark): Deleted.
        * bmalloc/LargeRange.h:
        (bmalloc::LargeRange::LargeRange):
        (bmalloc::LargeRange::usedSinceLastScavenge):
        (bmalloc::LargeRange::clearUsedSinceLastScavenge):
        (bmalloc::LargeRange::setUsedSinceLastScavenge):
        (): Deleted.
        * bmalloc/Scavenger.cpp:
        (bmalloc::Scavenger::Scavenger):
        (bmalloc::Scavenger::threadRunLoop):
        (bmalloc::Scavenger::timeSinceLastPartialScavenge): Deleted.
        (bmalloc::Scavenger::partialScavenge): Deleted.
        * bmalloc/Scavenger.h:
        * bmalloc/SmallPage.h:
        (bmalloc::SmallPage::usedSinceLastScavenge):
        (bmalloc::SmallPage::clearUsedSinceLastScavenge):
        (bmalloc::SmallPage::setUsedSinceLastScavenge):

2019-03-14  Yusuke Suzuki  <ysuzuki@apple.com>

        [bmalloc] Add StaticPerProcess for known types to save pages
        https://bugs.webkit.org/show_bug.cgi?id=195691

        Reviewed by Mark Lam.

        As initial memory footprint of VM + JSGlobalObject becomes 488KB dirty size in fast malloc memory (w/ JSC_useJIT=0 and Malloc=1), pages for PerProcess is costly.
        For example, under Malloc=1 mode, we still need to allocate PerProcess<DebugHeap> and PerProcess<Environment>. And sizeof(Environment) is only 1 (bool flag), and
        sizeof(DebugHeap) is 120. But we are allocating 1 pages for them. Since page size in iOS is 16KB, this 121B consumes 16KB dirty memory, and it is not negligible
        size if we keep in mind that the current fast malloc heap size is 488KB. Putting them into the __DATA section, close to the other mutable data, we can avoid allocating
        this page.

        This patch revives the SafePerProcess concept in r228107. We add "StaticPerProcess<T>", which allocates underlying storage statically in the __DATA section instead of
        allocating it at runtime. And we use this StaticPerProcess<T> for types where (1) T is known a priori, and (2) sizeof(T) is not huge.

        * bmalloc.xcodeproj/project.pbxproj:
        * bmalloc/AllIsoHeaps.cpp:
        * bmalloc/AllIsoHeaps.h:
        * bmalloc/Allocator.cpp:
        (bmalloc::Allocator::Allocator):
        * bmalloc/Cache.cpp:
        (bmalloc::Cache::Cache):
        * bmalloc/CryptoRandom.cpp:
        (bmalloc::cryptoRandom):
        * bmalloc/Deallocator.cpp:
        (bmalloc::Deallocator::Deallocator):
        * bmalloc/DebugHeap.cpp:
        * bmalloc/DebugHeap.h:
        (bmalloc::DebugHeap::tryGet):
        * bmalloc/Environment.cpp:
        * bmalloc/Environment.h:
        * bmalloc/Gigacage.cpp:
        (Gigacage::Callback::Callback):
        (Gigacage::Callback::function):
        (bmalloc::PrimitiveDisableCallbacks::PrimitiveDisableCallbacks):
        (Gigacage::disablePrimitiveGigacage):
        (Gigacage::addPrimitiveDisableCallback):
        (Gigacage::removePrimitiveDisableCallback):
        (Gigacage::shouldBeEnabled):
        (Gigacage::bmalloc::Callback::Callback): Deleted.
        (Gigacage::bmalloc::Callback::function): Deleted.
        (Gigacage::bmalloc::PrimitiveDisableCallbacks::PrimitiveDisableCallbacks): Deleted.
        * bmalloc/Heap.cpp:
        (bmalloc::Heap::Heap):
        (bmalloc::Heap::tryAllocateLarge):
        * bmalloc/IsoDirectoryInlines.h:
        (bmalloc::passedNumPages>::takeFirstEligible):
        (bmalloc::passedNumPages>::didBecome):
        * bmalloc/IsoHeapImpl.cpp:
        (bmalloc::IsoHeapImplBase::addToAllIsoHeaps):
        * bmalloc/IsoPage.cpp:
        (bmalloc::IsoPageBase::allocatePageMemory):
        * bmalloc/IsoTLS.cpp:
        (bmalloc::IsoTLS::IsoTLS):
        (bmalloc::IsoTLS::ensureEntries):
        (bmalloc::IsoTLS::forEachEntry):
        * bmalloc/IsoTLSEntry.cpp:
        (bmalloc::IsoTLSEntry::IsoTLSEntry):
        * bmalloc/IsoTLSInlines.h:
        (bmalloc::IsoTLS::allocateSlow):
        (bmalloc::IsoTLS::deallocateSlow):
        * bmalloc/IsoTLSLayout.cpp:
        * bmalloc/IsoTLSLayout.h:
        * bmalloc/Scavenger.cpp:
        (bmalloc::Scavenger::Scavenger):
        (bmalloc::dumpStats):
        (bmalloc::Scavenger::scavenge):
        (bmalloc::Scavenger::partialScavenge):
        (bmalloc::Scavenger::freeableMemory):
        (bmalloc::Scavenger::footprint):
        * bmalloc/Scavenger.h:
        * bmalloc/StaticPerProcess.h: Added.
        * bmalloc/VMHeap.cpp:
        * bmalloc/VMHeap.h:
        * bmalloc/Zone.h:
        * bmalloc/bmalloc.cpp:
        (bmalloc::api::scavenge):
        (bmalloc::api::isEnabled):
        (bmalloc::api::setScavengerThreadQOSClass):
        (bmalloc::api::enableMiniMode):
        * test/testbmalloc.cpp:
        (assertEmptyPointerSet):
        (assertHasObjects):
        (assertHasOnlyObjects):
        (assertClean):

2019-03-13  Yoshiaki Jitsukawa  <yoshiaki.jitsukawa@sony.com>

        [bmalloc] Use MADV_FREE on FreeBSD
        https://bugs.webkit.org/show_bug.cgi?id=195665

        Reviewed by Geoffrey Garen.

        * bmalloc/BPlatform.h:

        Introduce BOS_FREEBSD, which is equivalent to WTF_OS_FREEBSD

        * bmalloc/VMAllocate.h:
        (bmalloc::vmDeallocatePhysicalPages):

        Use MADV_FREE instead of MADV_DONTNEED if BOS(FREEBSD), since on FreeBSD,
        unlike on Linux, MADV_DONTNEED doesn't let the OS discard the contents of
        the pages.

2019-03-13  Sam Weinig  <sam@webkit.org>

        Fix testbmalloc build
        https://bugs.webkit.org/show_bug.cgi?id=195660

        Reviewed by Geoffrey Garen.

        * bmalloc.xcodeproj/project.pbxproj:
        Link Foundation in when building testbmalloc. Since bmalloc requires Foundation, and is a static
        library, all clients of bmalloc are required to link it themselves.
        
        * bmalloc/IsoPageInlines.h:
        * bmalloc/StdLibExtras.h: Added.
        (bmalloc::bitwise_cast):
        Add bitwise_cast implementation, and use it in IsoPageInlines.h. It is a layering violation
        to expect the one from WTF to be available, as seems to have been the case.

2019-03-12  Robin Morisset  <rmorisset@apple.com>

        A lot more classes have padding that can be reduced by reordering their fields
        https://bugs.webkit.org/show_bug.cgi?id=195579

        Reviewed by Mark Lam.

        * bmalloc/Heap.h:
        * bmalloc/Scavenger.h:

2019-03-05  Yusuke Suzuki  <ysuzuki@apple.com>

        [bmalloc] Bmalloc DebugHeap should have dump and scavenge features
        https://bugs.webkit.org/show_bug.cgi?id=195305

        Reviewed by Saam Barati.

        As the same to bmalloc, bmalloc::DebugHeap should have scavenge feature to make it scavengable if we want.
        We also add DebugHeap::dump feature which dumps system malloc data in the WebKit Malloc zone.

        * bmalloc/DebugHeap.cpp:
        (bmalloc::DebugHeap::scavenge):
        (bmalloc::DebugHeap::dump):
        * bmalloc/DebugHeap.h:
        * bmalloc/bmalloc.cpp:
        (bmalloc::api::scavenge):

2019-02-23  Keith Miller  <keith_miller@apple.com>

        Add new mac target numbers
        https://bugs.webkit.org/show_bug.cgi?id=194955

        Reviewed by Tim Horton.

        * Configurations/Base.xcconfig:
        * Configurations/DebugRelease.xcconfig:

2019-02-19  Yusuke Suzuki  <ysuzuki@apple.com>

        [bmalloc] bmalloc::Heap is allocated even though we use system malloc mode
        https://bugs.webkit.org/show_bug.cgi?id=194836

        Reviewed by Mark Lam.

        Previously, bmalloc::Heap holds DebugHeap, and delegates allocation and deallocation to debug heap.
        However, bmalloc::Heap is large. We would like to avoid initialization of bmalloc::Heap under the
        system malloc mode.

        This patch extracts out DebugHeap from bmalloc::Heap, and logically puts this in a boundary of
        bmalloc::api. bmalloc::api delegates allocation and deallocation to DebugHeap if DebugHeap is enabled.
        Otherwise, using bmalloc's usual mechanism. The challenge is that we would like to keep bmalloc fast
        path fast.

        1. For IsoHeaps, we use the similar techniques done in Cache. If the debug mode is enabled, we always go
           to the slow path of the IsoHeap allocation, and keep IsoTLS::get() returning nullptr. In the slow path,
           we just fallback to the usual bmalloc::api::tryMalloc implementation. This is efficient because bmalloc
           continues using the fast path.

        2. For the other APIs, like freeLargeVirtual, we just put DebugHeap check because this API itself takes fair
           amount of time. Then debug heap check does not matter.

        * bmalloc/Allocator.cpp:
        (bmalloc::Allocator::reallocateImpl):
        * bmalloc/Cache.cpp:
        (bmalloc::Cache::tryAllocateSlowCaseNullCache):
        (bmalloc::Cache::allocateSlowCaseNullCache):
        (bmalloc::Cache::deallocateSlowCaseNullCache):
        (bmalloc::Cache::tryReallocateSlowCaseNullCache):
        (bmalloc::Cache::reallocateSlowCaseNullCache):
        (): Deleted.
        (bmalloc::debugHeap): Deleted.
        * bmalloc/DebugHeap.cpp:
        * bmalloc/DebugHeap.h:
        (bmalloc::DebugHeap::tryGet):
        * bmalloc/Heap.cpp:
        (bmalloc::Heap::Heap):
        (bmalloc::Heap::footprint):
        (bmalloc::Heap::tryAllocateLarge):
        (bmalloc::Heap::deallocateLarge):
        * bmalloc/Heap.h:
        (bmalloc::Heap::debugHeap): Deleted.
        * bmalloc/IsoTLS.cpp:
        (bmalloc::IsoTLS::IsoTLS):
        (bmalloc::IsoTLS::isUsingDebugHeap): Deleted.
        (bmalloc::IsoTLS::debugMalloc): Deleted.
        (bmalloc::IsoTLS::debugFree): Deleted.
        * bmalloc/IsoTLS.h:
        * bmalloc/IsoTLSInlines.h:
        (bmalloc::IsoTLS::allocateSlow):
        (bmalloc::IsoTLS::deallocateSlow):
        * bmalloc/ObjectType.cpp:
        (bmalloc::objectType):
        * bmalloc/ObjectType.h:
        * bmalloc/Scavenger.cpp:
        (bmalloc::Scavenger::Scavenger):
        * bmalloc/bmalloc.cpp:
        (bmalloc::api::tryLargeZeroedMemalignVirtual):
        (bmalloc::api::freeLargeVirtual):
        (bmalloc::api::scavenge):
        (bmalloc::api::isEnabled):
        (bmalloc::api::setScavengerThreadQOSClass):
        (bmalloc::api::commitAlignedPhysical):
        (bmalloc::api::decommitAlignedPhysical):
        (bmalloc::api::enableMiniMode):

2019-02-20  Andy Estes  <aestes@apple.com>

        [Xcode] Add SDKVariant.xcconfig to various Xcode projects
        https://bugs.webkit.org/show_bug.cgi?id=194869

        Rubber-stamped by Jer Noble.

        * bmalloc.xcodeproj/project.pbxproj:

2019-02-20  Yusuke Suzuki  <ysuzuki@apple.com>

        [bmalloc] DebugHeap::malloc does not have "try" version.
        https://bugs.webkit.org/show_bug.cgi?id=194837

        Reviewed by Mark Lam.

        Since DebugHeap::malloc does not have "try" version, our tryAllocate implementation does not work well with DebugHeap.
        This patch adds crashOnFailure flag to DebugHeap::malloc.

        * bmalloc/Cache.cpp:
        (bmalloc::Cache::tryAllocateSlowCaseNullCache):
        (bmalloc::Cache::allocateSlowCaseNullCache):
        * bmalloc/DebugHeap.cpp:
        (bmalloc::DebugHeap::malloc):
        * bmalloc/DebugHeap.h:
        * bmalloc/IsoTLS.cpp:
        (bmalloc::IsoTLS::debugMalloc):

2019-02-20  Yusuke Suzuki  <ysuzuki@apple.com>

        [bmalloc] bmalloc::Cache should not be instantiated if we are using system malloc
        https://bugs.webkit.org/show_bug.cgi?id=194811

        Reviewed by Mark Lam.

        bmalloc::Cache is very large. It is 13KB. Since it exists per HeapKind, it takes 40KB.
        But this is meaningless if we are under the system malloc mode by using "Malloc=1". We
        found that it continues using so much dirty memory region even under the system malloc mode.
        This patch avoids instantiation of bmalloc::Cache under the system malloc mode.

        * bmalloc/Allocator.cpp:
        (bmalloc::Allocator::Allocator):
        (bmalloc::Allocator::tryAllocate):
        (bmalloc::Allocator::allocateImpl):
        (bmalloc::Allocator::reallocateImpl):
        (bmalloc::Allocator::allocateSlowCase):
        Allocator is a per Cache object. So we no longer need to keep m_debugHeap. If debug heap is enabled,
        Allocator is never created.

        * bmalloc/Allocator.h:
        * bmalloc/Cache.cpp:
        (bmalloc::debugHeap):
        (bmalloc::Cache::Cache):
        (bmalloc::Cache::tryAllocateSlowCaseNullCache):
        (bmalloc::Cache::allocateSlowCaseNullCache):
        (bmalloc::Cache::deallocateSlowCaseNullCache):
        (bmalloc::Cache::tryReallocateSlowCaseNullCache):
        (bmalloc::Cache::reallocateSlowCaseNullCache):
        * bmalloc/Cache.h:
        (bmalloc::Cache::tryAllocate):
        (bmalloc::Cache::tryReallocate):
        If the debug heap mode is enabled, we keep Cache::getFast() returning nullptr. And in the slow path case, we use debugHeap.
        This makes bmalloc fast path fast, while we avoid Cache instantiation.

        * bmalloc/Deallocator.cpp:
        (bmalloc::Deallocator::Deallocator):
        (bmalloc::Deallocator::scavenge):
        (bmalloc::Deallocator::deallocateSlowCase):
        * bmalloc/Deallocator.h:
        Ditto for Deallocator.

        * bmalloc/bmalloc.cpp:
        (bmalloc::api::isEnabled):
        We used `getFastCase()` for Heap. But it is basically wrong since we do not have any guarantee that someone already initializes
        Heap when this is called. Previously, luckily, Cache is initialized, and Cache initialized Heap. But Cache initialization is removed
        for system malloc mode and now PerProcess<PerHeapKind<Heap>>::getFastCase() returns nullptr at an early phase. This patch just uses
        Environment::isDebugHeapEnabled() instead.

2019-02-20  Commit Queue  <commit-queue@webkit.org>

        Unreviewed, rolling out r241789.
        https://bugs.webkit.org/show_bug.cgi?id=194856

        GuardMalloc crashes (Requested by yusukesuzuki on #webkit).

        Reverted changeset:

        "[bmalloc] bmalloc::Cache should not be instantiated if we are
        using system malloc"
        https://bugs.webkit.org/show_bug.cgi?id=194811
        https://trac.webkit.org/changeset/241789

2019-02-19  Yusuke Suzuki  <ysuzuki@apple.com>

        [bmalloc] bmalloc::Cache should not be instantiated if we are using system malloc
        https://bugs.webkit.org/show_bug.cgi?id=194811

        Reviewed by Mark Lam.

        bmalloc::Cache is very large. It is 13KB. Since it exists per HeapKind, it takes 40KB.
        But this is meaningless if we are under the system malloc mode by using "Malloc=1". We
        found that it continues using so much dirty memory region even under the system malloc mode.
        This patch avoids instantiation of bmalloc::Cache under the system malloc mode.

        * bmalloc/Allocator.cpp:
        (bmalloc::Allocator::Allocator):
        (bmalloc::Allocator::tryAllocate):
        (bmalloc::Allocator::allocateImpl):
        (bmalloc::Allocator::reallocateImpl):
        (bmalloc::Allocator::allocateSlowCase):
        Allocator is a per Cache object. So we no longer need to keep m_debugHeap. If debug heap is enabled,
        Allocator is never created.

        * bmalloc/Allocator.h:
        * bmalloc/Cache.cpp:
        (bmalloc::debugHeap):
        (bmalloc::Cache::Cache):
        (bmalloc::Cache::tryAllocateSlowCaseNullCache):
        (bmalloc::Cache::allocateSlowCaseNullCache):
        (bmalloc::Cache::deallocateSlowCaseNullCache):
        (bmalloc::Cache::tryReallocateSlowCaseNullCache):
        (bmalloc::Cache::reallocateSlowCaseNullCache):
        * bmalloc/Cache.h:
        (bmalloc::Cache::tryAllocate):
        (bmalloc::Cache::tryReallocate):
        If the debug heap mode is enabled, we keep Cache::getFast() returning nullptr. And in the slow path case, we use debugHeap.
        This makes bmalloc fast path fast, while we avoid Cache instantiation.

        * bmalloc/Deallocator.cpp:
        (bmalloc::Deallocator::Deallocator):
        (bmalloc::Deallocator::scavenge):
        (bmalloc::Deallocator::deallocateSlowCase):
        * bmalloc/Deallocator.h:
        Ditto for Deallocator.

2019-02-15  Yusuke Suzuki  <ysuzuki@apple.com>

        [bmalloc] NSBundle-based application name check should be executed after debug-heap environment variable check
        https://bugs.webkit.org/show_bug.cgi?id=194694

        Reviewed by Mark Lam.

        Interestingly, NSBundle allocates fair amount of memory and keeps it for a process-long time. For example, it
        allocates global NSConcreteHashTable, which takes 2.5KB. This patch changes the order of gigacage-check, we
        first check "Malloc=1" status, and then check the process name through NSBundle. This allows us to remove NSBundle
        related allocation in JSC initialization in the system malloc mode.

        * bmalloc/Gigacage.cpp:
        (Gigacage::shouldBeEnabled):

2019-02-15  Yusuke Suzuki  <ysuzuki@apple.com>

        [bmalloc] Do not start scavenger thread if we use system malloc
        https://bugs.webkit.org/show_bug.cgi?id=194674

        Reviewed by Mark Lam.

        We always start the scavenger thread even if system malloc is used by the environment variable like "Malloc=1".
        Because bmalloc allocation goes to the system malloc if "Malloc=1" is set, we do not need to scavenge. This patch
        changes it not to start the scavenger thread.

        * bmalloc/Scavenger.cpp:
        (bmalloc::Scavenger::Scavenger):

2019-02-12  Commit Queue  <commit-queue@webkit.org>

        Unreviewed, rolling out r241182.
        https://bugs.webkit.org/show_bug.cgi?id=194547

        causes a 2-3% Speedometer2 regression. (Requested by
        keith_miller on #webkit).

        Reverted changeset:

        "bmalloc uses more memory on iOS compared to macOS due to
        physical page size differences"
        https://bugs.webkit.org/show_bug.cgi?id=192389
        https://trac.webkit.org/changeset/241182

2019-02-07  Michael Saboff  <msaboff@apple.com>

        bmalloc uses more memory on iOS compared to macOS due to physical page size differences
        https://bugs.webkit.org/show_bug.cgi?id=192389

        Reviewed by Geoffrey Garen.

        Changed small line allocations to be in smallPageSize "virtual page" multiples instead of physical
        page size increments for sizes less that the physical page size.  This required changing the small
        page commit / decommit code to work in full physical page increments.  For page classes that are
        physical page size and larger, there isn't any functional change.

        When scavenging page classes smaller than the physical page size, we need to consider whether or
        not the adjacent small pages on the same physical page are also free before decommiting that
        containing page.  When we need to commit more memory, we commit the whole page, and add any
        adjacent virtual pages that were fully committed as well.

        * bmalloc/Chunk.h:
        (bmalloc::forEachPage):
        * bmalloc/Heap.cpp:
        (bmalloc::Heap::initializeLineMetadata):
        (bmalloc::Heap::initializePageMetadata):
        (bmalloc::Heap::scavenge):
        (bmalloc::__attribute__):
        (bmalloc::Heap::commitSmallPagesInPhysicalPage):
        (bmalloc::Heap::allocateSmallPage):
        (bmalloc::Heap::allocateSmallBumpRangesByMetadata):
        * bmalloc/Heap.h:
        * bmalloc/SmallPage.h:
        (bmalloc::SmallPage::refCount):

2019-01-18  Keith Miller  <keith_miller@apple.com>

        gigacage slide should randomize both start and end
        https://bugs.webkit.org/show_bug.cgi?id=193601

        Reviewed by Yusuke Suzuki.

        This patch makes it so that the gigacade slide has an arbitrary
        distance from the end as well as the start. This is done by
        picking a random size then based on that size picking an random
        starting offset.

        * bmalloc/Gigacage.h:
        * bmalloc/Heap.cpp:
        (bmalloc::Heap::Heap):

2019-01-18  Jer Noble  <jer.noble@apple.com>

        SDK_VARIANT build destinations should be separate from non-SDK_VARIANT builds
        https://bugs.webkit.org/show_bug.cgi?id=189553

        Reviewed by Tim Horton.

        * Configurations/Base.xcconfig:
        * Configurations/SDKVariant.xcconfig: Added.

2019-01-18  Keith Miller  <keith_miller@apple.com>

        Gigacages should start allocations from a slide
        https://bugs.webkit.org/show_bug.cgi?id=193523

        Reviewed by Mark Lam.

        This patch makes it so that Gigacage Heaps slide the start of the
        cage by some random amount. We still ensure that there is always
        at least 4/2GB, on MacOS/iOS respectively, of VA space available
        for allocation.

        Also, this patch changes some macros into constants since macros
        are the devil.

        * bmalloc/Gigacage.cpp:
        (Gigacage::bmalloc::protectGigacageBasePtrs):
        (Gigacage::bmalloc::unprotectGigacageBasePtrs):
        (Gigacage::bmalloc::runwaySize):
        (Gigacage::ensureGigacage):
        (Gigacage::shouldBeEnabled):
        * bmalloc/Gigacage.h:
        (Gigacage::name):
        (Gigacage::gigacageSizeToMask):
        (Gigacage::size):
        (Gigacage::mask):
        (Gigacage::basePtr):
        (Gigacage::ensureGigacage):
        (Gigacage::wasEnabled):
        (Gigacage::isCaged):
        (Gigacage::isEnabled):
        (Gigacage::caged):
        (Gigacage::disableDisablingPrimitiveGigacageIfShouldBeEnabled):
        (Gigacage::canPrimitiveGigacageBeDisabled):
        (Gigacage::disablePrimitiveGigacage):
        (Gigacage::addPrimitiveDisableCallback):
        (Gigacage::removePrimitiveDisableCallback):
        * bmalloc/Heap.cpp:
        (bmalloc::Heap::Heap):
        * bmalloc/Sizes.h:
        (bmalloc::Sizes::maskSizeClass):
        (bmalloc::Sizes::maskObjectSize):
        (bmalloc::Sizes::logSizeClass):
        (bmalloc::Sizes::logObjectSize):
        (bmalloc::Sizes::sizeClass):
        (bmalloc::Sizes::objectSize):
        (bmalloc::Sizes::pageSize):

2019-01-18  Matt Lewis  <jlewis3@apple.com>

        Unreviewed, rolling out r240160.

        This broke multiple internal builds.

        Reverted changeset:

        "Gigacages should start allocations from a slide"
        https://bugs.webkit.org/show_bug.cgi?id=193523
        https://trac.webkit.org/changeset/240160

2019-01-18  Keith Miller  <keith_miller@apple.com>

        Gigacages should start allocations from a slide
        https://bugs.webkit.org/show_bug.cgi?id=193523

        Reviewed by Mark Lam.

        This patch makes it so that Gigacage Heaps slide the start of the
        cage by some random amount. We still ensure that there is always
        at least 4/2GB, on MacOS/iOS respectively, of VA space available
        for allocation.

        Also, this patch changes some macros into constants since macros
        are the devil.

        * bmalloc/Gigacage.cpp:
        (Gigacage::bmalloc::protectGigacageBasePtrs):
        (Gigacage::bmalloc::unprotectGigacageBasePtrs):
        (Gigacage::bmalloc::runwaySize):
        (Gigacage::ensureGigacage):
        (Gigacage::shouldBeEnabled):
        * bmalloc/Gigacage.h:
        (Gigacage::name):
        (Gigacage::gigacageSizeToMask):
        (Gigacage::size):
        (Gigacage::mask):
        (Gigacage::basePtr):
        (Gigacage::ensureGigacage):
        (Gigacage::wasEnabled):
        (Gigacage::isCaged):
        (Gigacage::caged):
        (Gigacage::disableDisablingPrimitiveGigacageIfShouldBeEnabled):
        (Gigacage::disablePrimitiveGigacage):
        (Gigacage::addPrimitiveDisableCallback):
        (Gigacage::removePrimitiveDisableCallback):
        * bmalloc/Heap.cpp:
        (bmalloc::Heap::Heap):
        * bmalloc/Sizes.h:
        (bmalloc::Sizes::maskSizeClass):
        (bmalloc::Sizes::maskObjectSize):
        (bmalloc::Sizes::logSizeClass):
        (bmalloc::Sizes::logObjectSize):
        (bmalloc::Sizes::sizeClass):
        (bmalloc::Sizes::objectSize):
        (bmalloc::Sizes::pageSize):

2019-01-17  Truitt Savell  <tsavell@apple.com>

        Unreviewed, rolling out r240124.

        This commit broke an internal build.

        Reverted changeset:

        "SDK_VARIANT build destinations should be separate from non-
        SDK_VARIANT builds"
        https://bugs.webkit.org/show_bug.cgi?id=189553
        https://trac.webkit.org/changeset/240124

2019-01-17  Jer Noble  <jer.noble@apple.com>

        SDK_VARIANT build destinations should be separate from non-SDK_VARIANT builds
        https://bugs.webkit.org/show_bug.cgi?id=189553

        Reviewed by Tim Horton.

        * Configurations/Base.xcconfig:
        * Configurations/SDKVariant.xcconfig: Added.

2019-01-16  Keith Miller  <keith_miller@apple.com>

        bmalloc should use JSC VM tag for gigacage
        https://bugs.webkit.org/show_bug.cgi?id=193496

        Reviewed by Mark Lam.

        This patch moves the VMTag info from WTF to bmalloc so that we can
        tag gigacage memory with the unused JSC memory tag. The JSC memory
        tag was previously used for wasm but since wasm is now allocated
        out of the primitive cage it was unused.

        * bmalloc.xcodeproj/project.pbxproj:
        * bmalloc/BVMTags.h: Copied from Source/WTF/wtf/VMTags.h.
        * bmalloc/Gigacage.cpp:
        (Gigacage::ensureGigacage):
        * bmalloc/VMAllocate.h:
        (bmalloc::tryVMAllocate):
        (bmalloc::vmZeroAndPurge):

2019-01-09  Mark Lam  <mark.lam@apple.com>

        Gigacage disabling checks should handle the GIGACAGE_ALLOCATION_CAN_FAIL case properly.
        https://bugs.webkit.org/show_bug.cgi?id=193292
        <rdar://problem/46485450>

        Reviewed by Yusuke Suzuki.

        Previously, when GIGACAGE_ALLOCATION_CAN_FAIL is true, we allow the Gigacage to
        be disabled if we fail to allocate memory for it.  However, Gigacage::primitiveGigacageDisabled()
        still always assumes that the Gigacage is always enabled after ensureGigacage() is
        called.

        This patch updates Gigacage::primitiveGigacageDisabled() to allow the Gigacage to
        already be disabled if GIGACAGE_ALLOCATION_CAN_FAIL is true and wasEnabled() is
        false.

        In this patch, we also put the wasEnabled flag in the 0th slot of the
        g_gigacageBasePtrs buffer to ensure that it is also protected against writes just
        like the Gigacage base pointers.

        To achieve this, we do the following:
        1. Added a reservedForFlags field in struct BasePtrs.
        2. Added a ReservedForFlagsAndNotABasePtr Gigacage::Kind.
        3. Added assertions to ensure that the BasePtrs::primitive is at the offset
           matching the offset computed from Gigacage::Primitive.  Ditto for
           BasePtrs::jsValue and Gigacage::JSValue.
        4. Added assertions to ensure that Gigacage::ReservedForFlagsAndNotABasePtr is not
           used for fetching a Gigacage base pointer.
        5. Added RELEASE_BASSERT_NOT_REACHED() to implement such assertions in bmalloc.

        No test added because this issue requires Gigacage allocation to fail in order to
        manifest.  I've tested it manually by modifying the code locally to force an
        allocation failure.

        * bmalloc/BAssert.h:
        * bmalloc/Gigacage.cpp:
        (Gigacage::ensureGigacage):
        (Gigacage::primitiveGigacageDisabled):
        * bmalloc/Gigacage.h:
        (Gigacage::wasEnabled):
        (Gigacage::setWasEnabled):
        (Gigacage::name):
        (Gigacage::basePtr):
        (Gigacage::size):
        * bmalloc/HeapKind.h:
        (bmalloc::heapKind):

2018-12-15  Yusuke Suzuki  <yusukesuzuki@slowstart.org>

        Unreviewed, suppress warnings in Linux

        * bmalloc/Gigacage.cpp:

2018-12-14  Keith Miller  <keith_miller@apple.com>

        Gigacage runway should immediately follow the primitive cage
        https://bugs.webkit.org/show_bug.cgi?id=192733

        Reviewed by Saam Barati.

        This patch makes sure that the Gigacage runway is always
        immediately after the primitive cage. Since writing outside the
        primitive gigacage is likely to be more dangerous than the JSValue
        cage. The ordering of the cages is still random however.

        * bmalloc/Gigacage.cpp:
        (Gigacage::ensureGigacage):

2018-12-13  Mark Lam  <mark.lam@apple.com>

        Verify that tryLargeZeroedMemalignVirtual()'s aligned size and alignment values are valid.
        https://bugs.webkit.org/show_bug.cgi?id=192682
        <rdar://problem/37751522>

        Reviewed by Saam Barati.

        * bmalloc/bmalloc.cpp:
        (bmalloc::api::tryLargeZeroedMemalignVirtual):

2018-11-21  Dominik Infuehr  <dinfuehr@igalia.com>

        Enable JIT on ARM/Linux
        https://bugs.webkit.org/show_bug.cgi?id=191548

        Reviewed by Yusuke Suzuki.

        * bmalloc/IsoPageInlines.h:
        (bmalloc::IsoPage<Config>::startAllocating):

2018-11-01  Jiewen Tan  <jiewen_tan@apple.com>

        Replace CommonRandom SPI with API
        https://bugs.webkit.org/show_bug.cgi?id=191178
        <rdar://problem/45722391>

        Reviewed by Brent Fulgham.

        * bmalloc/CryptoRandom.cpp:
        (bmalloc::ARC4RandomNumberGenerator::stir):

2018-10-29  Mark Lam  <mark.lam@apple.com>

        Correctly detect string overflow when using the 'Function' constructor.
        https://bugs.webkit.org/show_bug.cgi?id=184883
        <rdar://problem/36320331>

        Reviewed by Saam Barati.

        * bmalloc/Allocator.cpp:
        (bmalloc::Allocator::reallocate):
        (bmalloc::Allocator::tryReallocate):
        (bmalloc::Allocator::reallocateImpl):
        * bmalloc/Allocator.h:
        * bmalloc/Cache.h:
        (bmalloc::Cache::tryReallocate):
        * bmalloc/DebugHeap.cpp:
        (bmalloc::DebugHeap::realloc):
        * bmalloc/DebugHeap.h:
        * bmalloc/bmalloc.h:
        (bmalloc::api::tryRealloc):

2018-10-25  Ross Kirsling  <ross.kirsling@sony.com>

        Cleanup: inline constexpr is redundant as constexpr implies inline
        https://bugs.webkit.org/show_bug.cgi?id=190819

        Reviewed by Mark Lam.

        * bmalloc/Algorithm.h:
        (bmalloc::max):
        (bmalloc::min):
        (bmalloc::mask):
        (bmalloc::test):
        (bmalloc::isPowerOfTwo):
        (bmalloc::roundDownToMultipleOf):
        (bmalloc::sizeOf):
        (bmalloc::bitCount):
        (bmalloc::log2):
        * bmalloc/Bits.h:
        (bmalloc::bitsArrayLength):
        * bmalloc/Sizes.h:
        (bmalloc::Sizes::maskSizeClass):

2018-10-24  Alexey Proskuryakov  <ap@apple.com>

        Add BPLATFORM(IOS_FAMILY)
        https://bugs.webkit.org/show_bug.cgi?id=190878

        Reviewed by Saam Barati.

        * bmalloc/AvailableMemory.cpp:
        (bmalloc::memorySizeAccordingToKernel):
        (bmalloc::computeAvailableMemory):
        * bmalloc/AvailableMemory.h:
        (bmalloc::isUnderMemoryPressure):
        * bmalloc/BPlatform.h:
        * bmalloc/Gigacage.h:
        * bmalloc/Logging.cpp:
        (bmalloc::logVMFailure):
        * bmalloc/VMAllocate.h:
        (bmalloc::vmPageSizePhysical):
        * bmalloc/bmalloc.h:
        * bmalloc/darwin/MemoryStatusSPI.h:

2018-10-12  Ryan Haddad  <ryanhaddad@apple.com>

        Unreviewed, rolling out r237063.

        Caused layout test fast/dom/Window/window-postmessage-clone-
        deep-array.html to fail on macOS and iOS Debug bots.

        Reverted changeset:

        "[JSC] Remove gcc warnings on mips and armv7"
        https://bugs.webkit.org/show_bug.cgi?id=188598
        https://trac.webkit.org/changeset/237063

2018-10-11  Guillaume Emont  <guijemont@igalia.com>

        [JSC] Remove gcc warnings on mips and armv7
        https://bugs.webkit.org/show_bug.cgi?id=188598

        Reviewed by Mark Lam.

        Add bitwise_cast (from WTF) and use it instead of reinterpret_cast in
        a couple places where reinterpret_cast triggers a warning about
        alignment even though we know that alignment is correct.

        * bmalloc/Algorithm.h:
        (bmalloc::bitwise_cast): Copied from WTF/wtf/StdLibextras.h
        * bmalloc/IsoDirectoryPageInlines.h:
        (bmalloc::IsoDirectoryPage<Config>::pageFor):
        * bmalloc/IsoPageInlines.h:
        (bmalloc::IsoPage<Config>::startAllocating):

2018-10-03  Dan Bernstein  <mitz@apple.com>

        bmalloc part of [Xcode] Update some build settings as recommended by Xcode 10
        https://bugs.webkit.org/show_bug.cgi?id=190250

        Reviewed by Alex Christensen.

        * Configurations/Base.xcconfig: Enabled CLANG_WARN_COMMA, CLANG_WARN_DEPRECATED_OBJC_IMPLEMENTATIONS,
          and CLANG_WARN_OBJC_IMPLICIT_RETAIN_SELF.

        * bmalloc.xcodeproj/project.pbxproj: Let Xcode update LastUpgradeCheck.

2018-09-25  Alex Christensen  <achristensen@webkit.org>

        Allow for suffixes to com.apple.WebKit.WebContent
        https://bugs.webkit.org/show_bug.cgi?id=189972

        Reviewed by Chris Dumez.

        * bmalloc/ProcessCheck.mm:
        (bmalloc::gigacageEnabledForProcess):

2018-09-24  Fujii Hironori  <Hironori.Fujii@sony.com>

        Rename WTF_COMPILER_GCC_OR_CLANG to WTF_COMPILER_GCC_COMPATIBLE
        https://bugs.webkit.org/show_bug.cgi?id=189733

        Reviewed by Michael Catanzaro.

        * bmalloc/BCompiler.h:

2018-08-27  Keith Rollin  <krollin@apple.com>

        Unreviewed build fix -- disable LTO for production builds

        * Configurations/Base.xcconfig:

2018-08-27  Keith Rollin  <krollin@apple.com>

        Build system support for LTO
        https://bugs.webkit.org/show_bug.cgi?id=187785
        <rdar://problem/42353132>

        Reviewed by Dan Bernstein.

        Update Base.xcconfig and DebugRelease.xcconfig to optionally enable
        LTO.

        * Configurations/Base.xcconfig:
        * Configurations/DebugRelease.xcconfig:

2018-08-16  Tomas Popela  <tpopela@redhat.com>

        bmalloc: Coverity scan issues
        https://bugs.webkit.org/show_bug.cgi?id=186763

        Reviewed by Saam Barati.

        * bmalloc/DebugHeap.h: Initialize the m_pageSize variable.
        * bmalloc/IsoTLS.cpp:
        (bmalloc::IsoTLS::ensureEntries): Check the return value of
        pthread_key_create return().
        * bmalloc/VMAllocate.h:
        (bmalloc::vmPageSize): Correctly check the return value of sysconf().

2018-07-27  Mark Lam  <mark.lam@apple.com>

        Initialize bmalloc::DebugHeap::m_pageSize for non-Darwin builds.
        https://bugs.webkit.org/show_bug.cgi?id=188132
        <rdar://problem/40401599>

        Reviewed by Saam Barati.

        * bmalloc/DebugHeap.cpp:
        (bmalloc::DebugHeap::DebugHeap):

2018-07-27  Saam Barati  <sbarati@apple.com>

        Explicitly handle memlimit_active < 0
        https://bugs.webkit.org/show_bug.cgi?id=188125

        Reviewed by Mark Lam.

        This may come up during development when someone wants the limit
        to be "infinite".

        * bmalloc/AvailableMemory.cpp:
        (bmalloc::jetsamLimit):

2018-07-27  Saam Barati  <sbarati@apple.com>

        Use SPI to compute the jetsam limit on iOS instead of hardcoding 840MB
        https://bugs.webkit.org/show_bug.cgi?id=188091
        <rdar://problem/42647697>

        Reviewed by Simon Fraser.

        We want bmalloc to dynamically adapt to the jetsam limit of the process
        it's running in. WTF::ramSize() is based off bmalloc's availableMemory,
        so it will now reflect the result of the real jetsam limit when we can
        read it.
        
        Reading the jetsam limit requires an entitlement, so this patch opts in
        the WebContent/Storage/Network processes. We fall back to 840MB (the
        old hard coded value) when the SPI call fails (e.g, when we're in a
        process without the proper entitlement).

        * bmalloc.xcodeproj/project.pbxproj:
        * bmalloc/AvailableMemory.cpp:
        (bmalloc::jetsamLimit):
        (bmalloc::computeAvailableMemory):
        * bmalloc/darwin/MemoryStatusSPI.h: Added.

2018-07-24  Saam Barati  <sbarati@apple.com>

        Revert back to using phys_footprint to calculate isUnderMemoryPressure()
        https://bugs.webkit.org/show_bug.cgi?id=187919
        <rdar://problem/42552888>

        Reviewed by Simon Fraser.

        Currently on iOS, bmalloc will run the scavenger more frequently when it detects
        that the process is under memory pressure. However, it only uses bmalloc's
        own footprint as a percentage of the HW available memory to determine if
        the process is under memory pressure. This is a change I recently made
        in an effort to run the scavenger less when bmalloc wasn't contributing
        to the dirty footprint in the process. However, this fails to run the
        scavenger eagerly when the process in question has a heap split
        between a lot of dirty bmalloc memory as well as a lot of dirty memory
        from elsewhere. We also have evidence that we may have increased jetsams
        in the Web Content process. Since my original change was not a measurable
        speedup, this patch reverts isUnderMemoryPressure() to its previous
        behavior of using phys_footprint to determine if 75% of the available 
        HW memory is being used.

        * bmalloc/AvailableMemory.cpp:
        (bmalloc::memoryStatus):

2019-07-12  Michael Saboff  <msaboff@apple.com>

        Disable IsoHeaps when Gigacage is off
        https://bugs.webkit.org/show_bug.cgi?id=187160

        Reviewed by Saam Barati.

        Relanding change sets 233547 and 233550 with the added fix that Gigacage is also
        enabled for DumpRenderTree.

        Updated determineMallocFallbackState to base enabling of Iso Heaps on Gigacage 
        being enabled.  We do this because if Gigacage is disabled, it may be due to lack
        of address space.

        To work around a compiler issue uncovered by the change above, I added explicit
        instantiation of PerThread's static variables.  Defined the same explicit
        instantiated static variables with export scope in the new file PerThread.cpp
        to eliminate separate variables allocations in each linked framework / library.

        * CMakeLists.txt:
        * bmalloc.xcodeproj/project.pbxproj:
        * bmalloc/IsoTLS.cpp:
        (bmalloc::IsoTLS::determineMallocFallbackState):
        * bmalloc/PerThread.cpp: Added.
        * bmalloc/PerThread.h:
        * bmalloc/ProcessCheck.mm:
        (bmalloc::gigacageEnabledForProcess):

2018-07-09  Commit Queue  <commit-queue@webkit.org>

        Unreviewed, rolling out r233547 and r233550.
        https://bugs.webkit.org/show_bug.cgi?id=187497

        Introduced flakiness for media/fullscreen-* tests on mac-wk1
        (Requested by ryanhaddad on #webkit).

        Reverted changesets:

        "Disable IsoHeaps when Gigacage is off"
        https://bugs.webkit.org/show_bug.cgi?id=187160
        https://trac.webkit.org/changeset/233547

        "Build fix (r233547): Disable IsoHeaps when Gigacage is off"
        https://bugs.webkit.org/show_bug.cgi?id=187160
        https://trac.webkit.org/changeset/233550

2018-07-05  David Kilzer  <ddkilzer@apple.com>

        Build fix (r233547): Disable IsoHeaps when Gigacage is off
        <https://webkit.org/b/187160>

        * bmalloc/PerThread.cpp: Add #if !HAVE_PTHREAD_MACHDEP_H/#endif
        around variables only used when that macro is 0.  Include what
        you use: Cache.h and Heap.h.
        * bmalloc/PerThread.h: Include <memory> for std::once_flag.

2018-07-05  Michael Saboff  <msaboff@apple.com>

        Disable IsoHeaps when Gigacage is off
        https://bugs.webkit.org/show_bug.cgi?id=187160

        Reviewed by Saam Barati.

        Updated determineMallocFallbackState to base enabling of Iso Heaps on Gigacage 
        being enabled.  We do this because if Gigacage is disabled, it may be due to lack
        of address space.

        To work around a compiler issue uncovered by the change above, I added explicit
        instantiation of PerThread's static variables.  Defined the same explicit
        instantiated static variables with export scope in the new file PerThread.cpp
        to eliminate separate variables allocations in each linked framework / library.

        * CMakeLists.txt:
        * bmalloc.xcodeproj/project.pbxproj:
        * bmalloc/IsoTLS.cpp:
        (bmalloc::IsoTLS::determineMallocFallbackState):
        * bmalloc/PerThread.cpp: Added.
        * bmalloc/PerThread.h:

2018-07-04  Tim Horton  <timothy_horton@apple.com>

        Introduce PLATFORM(IOSMAC)
        https://bugs.webkit.org/show_bug.cgi?id=187315

        Reviewed by Dan Bernstein.

        * Configurations/Base.xcconfig:

2018-06-29  Ryan Haddad  <ryanhaddad@apple.com>

        Unreviewed, rolling out r233347.

        Causes crashes during WK1 tests.

        Reverted changeset:

        "Disable IsoHeaps when Gigacage is off"
        https://bugs.webkit.org/show_bug.cgi?id=187160
        https://trac.webkit.org/changeset/233347

2018-06-28  Michael Saboff  <msaboff@apple.com>

        Disable IsoHeaps when Gigacage is off
        https://bugs.webkit.org/show_bug.cgi?id=187160

        Reviewed by Saam Barati.

        If Gigacage is disabled, it may be due to lack of address space.
        Therefore we should also turn off IsoHeaps since it uses more virtual
        address space as well.

        * bmalloc/IsoTLS.cpp:
        (bmalloc::IsoTLS::determineMallocFallbackState):

2018-06-27  Simon Fraser  <simon.fraser@apple.com>

        https://hackernoon.com/ uses lots of layer backing store
        https://bugs.webkit.org/show_bug.cgi?id=186909
        rdar://problem/40257540

        Reviewed by Tim Horton.
        
        Drive-by typo fix.

        * bmalloc/Scavenger.cpp:
        (bmalloc::dumpStats):

2018-06-26  Saam Barati  <sbarati@apple.com>

        Unreviewed followup. Fix the watchos build after r233192.

        This patch also correct the changelog entry below to have the correct
        bug and title info.

        * bmalloc/ProcessCheck.mm:

2018-06-26  Saam Barati  <sbarati@apple.com>

        Switch to system malloc on iOS when nano malloc is disabled
        https://bugs.webkit.org/show_bug.cgi?id=186322
        <rdar://problem/41140257>

        Reviewed by Keith Miller.

        We have evidence showing that processes with small heaps using the
        JS API are more space efficient when using system malloc. Our main
        hypothesis as to why this is, is that when dealing with small heaps,
        one malloc can be more efficient at optimizing memory usage than
        two mallocs.

        * bmalloc/BPlatform.h:
        * bmalloc/Environment.cpp:
        (bmalloc::isNanoMallocEnabled):
        (bmalloc::Environment::computeIsDebugHeapEnabled):
        * bmalloc/ProcessCheck.h:
        (bmalloc::shouldProcessUnconditionallyUseBmalloc):
        * bmalloc/ProcessCheck.mm:
        (bmalloc::shouldProcessUnconditionallyUseBmalloc):

2018-06-24  Yusuke Suzuki  <utatane.tea@gmail.com>

        [bmalloc][Linux] Remove static initializers for PerProcess<>::s_object
        https://bugs.webkit.org/show_bug.cgi?id=186966

        Reviewed by Anders Carlsson.

        chrome/tools/linux/dump-static-initializers.py can dump static initializers
        in the binary and we found that PerProcess<>::s_object initialization is done
        by static initializers in GCC + Linux environments. The example is the following.

        Scavenger.cpp (initializer offset 0x38c210 size 0x3e)
            _GLOBAL__sub_I_Scavenger.cpp+0x1e
            _GLOBAL__sub_I_Scavenger.cpp+0x2d
            _GLOBAL__sub_I_Scavenger.cpp+0x3c
            _GLOBAL__sub_I_Scavenger.cpp+0xf
            guard variable for bmalloc::PerProcess<bmalloc::AllIsoHeaps>::s_object@@Base-0x3f0d8
            guard variable for bmalloc::PerProcess<bmalloc::Environment>::s_object@@Base-0x3f0e8
            guard variable for bmalloc::PerProcess<bmalloc::PerHeapKind<bmalloc::Heap> >::s_object@@Base-0x3c600
            guard variable for bmalloc::PerProcess<bmalloc::Scavenger>::s_object@@Base-0x38ce8

        We can remove this by initializing `nullptr`, which leads to constexpr initialization.
        After this change, Linux JSCOnly libJavaScriptCore.so has no static initializers.

        * bmalloc/PerProcess.h:

2018-06-09  Dan Bernstein  <mitz@apple.com>

        [Xcode] Clean up and modernize some build setting definitions
        https://bugs.webkit.org/show_bug.cgi?id=186463

        Reviewed by Sam Weinig.

        * Configurations/Base.xcconfig: Removed definition for macOS 10.11.
        * Configurations/DebugRelease.xcconfig: Ditto.

2018-06-07  Darin Adler  <darin@apple.com>

        [Cocoa] Turn on ARC for the single Objective-C++ source file in bmalloc
        https://bugs.webkit.org/show_bug.cgi?id=186398

        Reviewed by Saam Barati.

        * Configurations/Base.xcconfig: Turn on ARC.
        * bmalloc/ProcessCheck.mm:
        (bmalloc::gigacageEnabledForProcess): Removed the globals from this function,
        since it's only called once. If it was called more than once, we could optimize
        that with a single boolean global rather than two strings and two booleans.

2018-06-07  David Kilzer  <ddkilzer@apple.com>

        bmalloc: Fix 'noreturn' warnings when compiling with -std=gnu++17
        <https://webkit.org/b/186400>

        Reviewed by Saam Barati.

        Fixes the following warnings when compiling with gnu++17:

            Source/bmalloc/bmalloc/Scavenger.cpp:363:1: error: function 'threadRunLoop' could be declared with attribute 'noreturn' [-Werror,-Wmissing-noreturn]
            {
            ^
            Source/bmalloc/bmalloc/Scavenger.cpp:358:1: error: function 'threadEntryPoint' could be declared with attribute 'noreturn' [-Werror,-Wmissing-noreturn]
            {
            ^

        * bmalloc/BCompiler.h:
        (BCOMPILER): Add support for the BCOMPILER() macro, then add
        BCOMPILER(GCC_OR_CLANG).  Taken from Source/WTF/wtf/Compiler.h.
        (BNO_RETURN): Implement 'norerturn' support using the new
        BCOMPILER() macros.  Taken from Source/WTF/wtf/Compiler.h.
        * bmalloc/Scavenger.cpp:
        (bmalloc::Scavenger::threadRunLoop): Remove the workaround that
        tricked older compilers into thinking the while() loop wasn't
        infinite.
        * bmalloc/Scavenger.h:
        (bmalloc::Scavenger::threadEntryPoint): Add BNO_RETURN attribute.
        (bmalloc::Scavenger::threadRunLoop): Ditto.

2018-05-29  Saam Barati  <sbarati@apple.com>

        JSC should put bmalloc's scavenger into mini mode
        https://bugs.webkit.org/show_bug.cgi?id=185988

        Reviewed by Michael Saboff.

        We expose an API for putting bmalloc into mini mode. All that means now
        is that we'll run the scavenger more aggressively.

        * bmalloc/Scavenger.cpp:
        (bmalloc::Scavenger::enableMiniMode):
        (bmalloc::Scavenger::threadRunLoop):
        * bmalloc/Scavenger.h:
        * bmalloc/Sizes.h:
        * bmalloc/bmalloc.cpp:
        (bmalloc::api::enableMiniMode):
        * bmalloc/bmalloc.h:

2018-05-29  Geoffrey Garen  <ggaren@apple.com>

        Fixed the bmalloc build
        https://bugs.webkit.org/show_bug.cgi?id=186025

        Reviewed by Sam Weinig.

        * bmalloc.xcodeproj/project.pbxproj: Link Foundation because the 
        gigacage check needs it.

2018-05-23  Antti Koivisto  <antti@apple.com>

        Increase the simulated memory size on PLATFORM(IOS_SIMULATOR) from 512MB to 1024MB
        https://bugs.webkit.org/show_bug.cgi?id=185908

        Reviewed by Geoffrey Garen.

        We don't support 512MB devices anymore. This will make the simulator behave more
        like a real device.

        * bmalloc/AvailableMemory.cpp:
        (bmalloc::memorySizeAccordingToKernel):

        Factor to a function.
        Don't use availableMemoryGuess for the simulator value as it is not a guess.

        (bmalloc::computeAvailableMemory):

        Apply the same adjustments to the simulated value too.

2018-05-22  Ryan Haddad  <ryanhaddad@apple.com>

        Unreviewed, rolling out r232052.

        Breaks internal builds.

        Reverted changeset:

        "Use more C++17"
        https://bugs.webkit.org/show_bug.cgi?id=185176
        https://trac.webkit.org/changeset/232052

2018-05-22  Yusuke Suzuki  <utatane.tea@gmail.com>

        Define GIGACAGE_ALLOCATION_CAN_FAIL on Linux
        https://bugs.webkit.org/show_bug.cgi?id=183329

        Reviewed by Michael Catanzaro.

        We specify `GIGACAGE_ALLOCATION_CAN_FAIL 1` in Linux since
        Linux can fail to `mmap` if `vm.overcommit_memory = 2`.
        Users can enable Gigacage if users enable overcommit_memory.

        * bmalloc/Gigacage.h:

2018-05-21  Yusuke Suzuki  <utatane.tea@gmail.com>

        Use more C++17
        https://bugs.webkit.org/show_bug.cgi?id=185176

        Reviewed by JF Bastien.

        Add BNO_RETURN.

        * Configurations/Base.xcconfig:
        * bmalloc/BCompiler.h:
        * bmalloc/Scavenger.h:

2018-05-06  Yusuke Suzuki  <utatane.tea@gmail.com>

        [JSC] Remove "using namespace std;" from JSC, bmalloc, WTF
        https://bugs.webkit.org/show_bug.cgi?id=185362

        Reviewed by Sam Weinig.

        * bmalloc/Allocator.cpp:
        * bmalloc/Deallocator.cpp:

2018-05-03  Filip Pizlo  <fpizlo@apple.com>

        Strings should not be allocated in a gigacage
        https://bugs.webkit.org/show_bug.cgi?id=185218

        Reviewed by Saam Barati.
        
        This removes the string gigacage.
        
        Putting strings in a gigacage prevents read gadgets. The other things that get to be in gigacages
        are there to prevent read-write gadgets.
        
        Also, putting strings in a gigacage seems to have been a bigger regression than putting other
        things in gigacages.
        
        Therefore, to maximize the benefit/cost ratio of gigacages, we should evict strings from them. If
        we want to throw away perf for security, there are more beneficial things to sacrifice.

        * bmalloc/Gigacage.h:
        (Gigacage::name):
        (Gigacage::basePtr):
        (Gigacage::size):
        (Gigacage::forEachKind):
        * bmalloc/HeapKind.h:
        (bmalloc::isGigacage):
        (bmalloc::gigacageKind):
        (bmalloc::heapKind):
        (bmalloc::isActiveHeapKindAfterEnsuringGigacage):
        (bmalloc::mapToActiveHeapKindAfterEnsuringGigacage):

2018-04-30  Yusuke Suzuki  <utatane.tea@gmail.com>

        Use WordLock instead of std::mutex for Threading
        https://bugs.webkit.org/show_bug.cgi?id=185121

        Reviewed by Geoffrey Garen.

        Add constexpr to explicitly describe that bmalloc::Mutex constructor is constexpr.

        * bmalloc/Mutex.h:

2018-04-23  Ting-Wei Lan  <lantw44@gmail.com>

        Include stdio.h before using stderr
        https://bugs.webkit.org/show_bug.cgi?id=184872

        Reviewed by Yusuke Suzuki.

        * bmalloc/PerProcess.cpp:
        * bmalloc/Scavenger.cpp:

2018-04-21  Yusuke Suzuki  <utatane.tea@gmail.com>

        Unreviewed, follow-up patch after r230474
        https://bugs.webkit.org/show_bug.cgi?id=166684

        Add "JavaScriptCore" to Darwin name. And use short name "BMScavenger"
        for Linux since adding "JavaScriptCore" makes the name too long for Linux.

        * bmalloc/Scavenger.cpp:
        (bmalloc::Scavenger::threadRunLoop):

2018-04-18  Jer Noble  <jer.noble@apple.com>

        Don't put build products into WK_ALTERNATE_WEBKIT_SDK_PATH for engineering builds
        https://bugs.webkit.org/show_bug.cgi?id=184762

        Reviewed by Dan Bernstein.

        * Configurations/Base.xcconfig:

2018-04-20  Daniel Bates  <dabates@apple.com>

        Remove code for compilers that did not support NSDMI for aggregates
        https://bugs.webkit.org/show_bug.cgi?id=184599

        Reviewed by Per Arne Vollan.

        Remove workaround for earlier Visual Studio versions that did not support non-static data
        member initializers (NSDMI) for aggregates. We have since updated all the build.webkit.org
        and EWS bots to a newer version that supports this feature.

        * bmalloc/BPlatform.h:
        * bmalloc/List.h:
        (bmalloc::ListNode::ListNode): Deleted.
        (bmalloc::List::iterator::iterator): Deleted.

2018-04-19  David Kilzer  <ddkilzer@apple.com>

        Enable Objective-C weak references
        <https://webkit.org/b/184789>
        <rdar://problem/39571716>

        Reviewed by Dan Bernstein.

        * Configurations/Base.xcconfig:
        (CLANG_ENABLE_OBJC_WEAK): Enable.

2018-04-12  Saam Barati  <sbarati@apple.com>

        Lessen partial scavenge interval on x86-64
        https://bugs.webkit.org/show_bug.cgi?id=184577

        Rubber-stamped by Filip Pizlo.

        I initially made the scavenge interval longer because I had thought the
        shorter interval caused a JetStream regression. I was mistaken though.
        I was looking at the wrong commit range when analyzing perf data.
        
        This patch shortens the interval, but still keeps x86-64 50% longer than
        other architectures. We know that scavenging frequently on Mac is less
        important to overall system performance than it is on iOS.

        * bmalloc/Scavenger.cpp:
        (bmalloc::Scavenger::threadRunLoop):

2018-04-12  Saam Barati  <sbarati@apple.com>

        Raise the partial scavenge interval even more on x86-64
        https://bugs.webkit.org/show_bug.cgi?id=184551

        Rubber-stamped by Filip Pizlo.

        The JetStream regression didn't recover from my previous patch.
        This is another attempt to get it to recover perf.

        * bmalloc/Scavenger.cpp:
        (bmalloc::Scavenger::threadRunLoop):

2018-04-11  Saam Barati  <sbarati@apple.com>

        raise partial scavenge interval on x86-64
        https://bugs.webkit.org/show_bug.cgi?id=184521

        Rubber-stamped by Filip Pizlo.

        This patch is an attempt to recover the 1-3% JetStream regression
        my initial partial scavenging patch introduced on some Macs.

        * bmalloc/Scavenger.cpp:
        (bmalloc::Scavenger::threadRunLoop):

2018-04-10  Saam Barati  <sbarati@apple.com>

        IsoHeapImpl::scavenge* needs to grab the lock
        https://bugs.webkit.org/show_bug.cgi?id=184461

        Reviewed by Filip Pizlo.

        Another thread could be modifying the linked list that the scavenge* methods traverse.

        * bmalloc/IsoHeapImplInlines.h:
        (bmalloc::IsoHeapImpl<Config>::scavenge):
        (bmalloc::IsoHeapImpl<Config>::scavengeToHighWatermark):

2018-04-10  Saam Barati  <sbarati@apple.com>

        bmalloc should do partial scavenges more frequently
        https://bugs.webkit.org/show_bug.cgi?id=184176

        Reviewed by Filip Pizlo.

        This patch adds the ability for bmalloc to do a partial scavenge.
        bmalloc will now do a partial scavenge with some frequency even
        when the heap is growing.
        
        For Heap, this means tracking the high water mark of where the Heap
        has allocated since the last scavenge. Partial scavenging is just
        decommitting entries in the LargeMap that are past this high water
        mark. Because we allocate in first fit order out of LargeMap, tracking
        the high water mark is a good heuristic of how much memory a partial
        scavenge should decommit.
        
        For IsoHeaps, each IsoDirectory also keeps track of its high water mark
        for the furthest page it allocates into. Similar to Heap, we scavenge pages
        past that high water mark. IsoHeapImpl then tracks the high water mark
        for the IsoDirectory it allocates into. We then scavenge all directories 
        including and past the directory high water mark. This includes scavenging
        the inline directory when its the only thing we allocate out of since
        the last scavenge.
        
        This patch also adds some other capabilities to bmalloc:
        
        Heaps and IsoHeaps now track how much memory is freeable. Querying
        this number is now cheap.
        
        Heaps no longer hold the global lock when decommitting large ranges.
        Instead, that range is just marked as non eligible to be allocated.
        Then, without the lock held, the scavenger will decommit those ranges.
        Once this is done, the scavenger will then reacquire the lock and mark
        these ranges as eligible. This lessens lock contention between the
        scavenger and the allocation slow path since threads that are taking an
        allocation slow path can now allocate concurrently to the scavenger's
        decommits. The main consideration in adding this functionality is that
        a large allocation may fail while the scavenger is in the process of
        decommitting memory. When the Heap fails to allocate a large range when
        the scavenger is in the middle of a decommit, Heap will wait for the
        Scavenger to finish and then it will try to allocate a large range again.
        
        Decommitting from Heap now aggregates the ranges to decommit and tries to
        merge them together to lower the number of calls to vmDeallocatePhysicalPages.
        This is analogous to what IsoHeaps already do.

        * bmalloc.xcodeproj/project.pbxproj:
        * bmalloc/Allocator.cpp:
        (bmalloc::Allocator::tryAllocate):
        (bmalloc::Allocator::allocateImpl):
        (bmalloc::Allocator::reallocate):
        (bmalloc::Allocator::refillAllocatorSlowCase):
        (bmalloc::Allocator::allocateLarge):
        * bmalloc/BulkDecommit.h: Added.
        (bmalloc::BulkDecommit::addEager):
        (bmalloc::BulkDecommit::addLazy):
        (bmalloc::BulkDecommit::processEager):
        (bmalloc::BulkDecommit::processLazy):
        (bmalloc::BulkDecommit::add):
        (bmalloc::BulkDecommit::process):
        * bmalloc/Deallocator.cpp:
        (bmalloc::Deallocator::scavenge):
        (bmalloc::Deallocator::processObjectLog):
        (bmalloc::Deallocator::deallocateSlowCase):
        * bmalloc/Deallocator.h:
        (bmalloc::Deallocator::lineCache):
        * bmalloc/Heap.cpp:
        (bmalloc::Heap::freeableMemory):
        (bmalloc::Heap::markAllLargeAsEligibile):
        (bmalloc::Heap::decommitLargeRange):
        (bmalloc::Heap::scavenge):
        (bmalloc::Heap::scavengeToHighWatermark):
        (bmalloc::Heap::deallocateLineCache):
        (bmalloc::Heap::allocateSmallChunk):
        (bmalloc::Heap::deallocateSmallChunk):
        (bmalloc::Heap::allocateSmallPage):
        (bmalloc::Heap::deallocateSmallLine):
        (bmalloc::Heap::allocateSmallBumpRangesByMetadata):
        (bmalloc::Heap::allocateSmallBumpRangesByObject):
        (bmalloc::Heap::splitAndAllocate):
        (bmalloc::Heap::tryAllocateLarge):
        (bmalloc::Heap::allocateLarge):
        (bmalloc::Heap::isLarge):
        (bmalloc::Heap::largeSize):
        (bmalloc::Heap::shrinkLarge):
        (bmalloc::Heap::deallocateLarge):
        (bmalloc::Heap::externalCommit):
        (bmalloc::Heap::externalDecommit):
        * bmalloc/Heap.h:
        (bmalloc::Heap::allocateSmallBumpRanges):
        (bmalloc::Heap::derefSmallLine):
        * bmalloc/IsoDirectory.h:
        * bmalloc/IsoDirectoryInlines.h:
        (bmalloc::passedNumPages>::takeFirstEligible):
        (bmalloc::passedNumPages>::didBecome):
        (bmalloc::passedNumPages>::didDecommit):
        (bmalloc::passedNumPages>::scavengePage):
        (bmalloc::passedNumPages>::scavenge):
        (bmalloc::passedNumPages>::scavengeToHighWatermark):
        (bmalloc::passedNumPages>::freeableMemory): Deleted.
        * bmalloc/IsoHeapImpl.h:
        * bmalloc/IsoHeapImplInlines.h:
        (bmalloc::IsoHeapImpl<Config>::takeFirstEligible):
        (bmalloc::IsoHeapImpl<Config>::scavenge):
        (bmalloc::IsoHeapImpl<Config>::scavengeToHighWatermark):
        (bmalloc::IsoHeapImpl<Config>::freeableMemory):
        (bmalloc::IsoHeapImpl<Config>::isNowFreeable):
        (bmalloc::IsoHeapImpl<Config>::isNoLongerFreeable):
        * bmalloc/LargeMap.cpp:
        (bmalloc::LargeMap::remove):
        (bmalloc::LargeMap::markAllAsEligibile):
        * bmalloc/LargeMap.h:
        (bmalloc::LargeMap::size):
        (bmalloc::LargeMap::at):
        * bmalloc/LargeRange.h:
        (bmalloc::LargeRange::setEligible):
        (bmalloc::LargeRange::isEligibile const):
        (bmalloc::canMerge):
        * bmalloc/ObjectType.cpp:
        (bmalloc::objectType):
        * bmalloc/Scavenger.cpp:
        (bmalloc::PrintTime::PrintTime):
        (bmalloc::PrintTime::~PrintTime):
        (bmalloc::PrintTime::print):
        (bmalloc::Scavenger::timeSinceLastFullScavenge):
        (bmalloc::Scavenger::timeSinceLastPartialScavenge):
        (bmalloc::Scavenger::scavenge):
        (bmalloc::Scavenger::partialScavenge):
        (bmalloc::Scavenger::freeableMemory):
        (bmalloc::Scavenger::threadRunLoop):
        * bmalloc/Scavenger.h:
        * bmalloc/SmallLine.h:
        (bmalloc::SmallLine::refCount):
        (bmalloc::SmallLine::ref):
        (bmalloc::SmallLine::deref):
        * bmalloc/SmallPage.h:
        (bmalloc::SmallPage::refCount):
        (bmalloc::SmallPage::hasFreeLines const):
        (bmalloc::SmallPage::setHasFreeLines):
        (bmalloc::SmallPage::ref):
        (bmalloc::SmallPage::deref):
        * bmalloc/bmalloc.cpp:
        (bmalloc::api::tryLargeZeroedMemalignVirtual):
        (bmalloc::api::freeLargeVirtual):

2018-04-09  Yusuke Suzuki  <utatane.tea@gmail.com>

        [bmalloc] Name Scavenger thread "bmalloc scavenger"
        https://bugs.webkit.org/show_bug.cgi?id=166684

        Reviewed by Saam Barati.

        We name the thread for bmalloc Scavenger "bmalloc scavenger".
        It is useful for debugging. In Linux environment, it will be
        shown in GDB.

        * bmalloc/Scavenger.cpp:
        (bmalloc::Scavenger::threadRunLoop):
        (bmalloc::Scavenger::setName):
        * bmalloc/Scavenger.h:

2018-04-09  Michael Catanzaro  <mcatanzaro@igalia.com>

        Rename UNUSED to BUNUSED
        https://bugs.webkit.org/show_bug.cgi?id=184093

        Reviewed by Yusuke Suzuki.

        * bmalloc/BAssert.h:
        * bmalloc/VMAllocate.h:
        (bmalloc::vmValidate):
        (bmalloc::vmValidatePhysical):

2018-04-08  Yusuke Suzuki  <utatane.tea@gmail.com>

        Use alignas instead of compiler-specific attributes
        https://bugs.webkit.org/show_bug.cgi?id=183508

        Reviewed by Mark Lam.

        Use alignas for g_gigacageBasePtr. We also add reinterpret_cast to fix
        compile errors in ARMv7 and MIPS JSCOnly ports.

        * bmalloc/Gigacage.cpp:
        * bmalloc/Gigacage.h:
        (Gigacage::basePtrs):

2018-04-06  Saam Barati  <sbarati@apple.com>

        bmalloc virtual allocation API should not treat memory it vends as dirty with respect to how it drives the scavenger
        https://bugs.webkit.org/show_bug.cgi?id=184342

        Reviewed by Mark Lam.

        Currently, the only user of this API is Wasm. Ideally, Wasm would tell
        us exactly which page is dirtied. We should really do that at some point:
        https://bugs.webkit.org/show_bug.cgi?id=184207
        
        However, until we do that, it's better to treat none of the virtual memory
        we vend as dirty, versus what we do now, which is treat it all as dirty.
        This dirty memory tracking helps drive the scavenger, so on iOS, having the
        scavenger think its under memory pressure because of memory it can't free isn't
        useful.

        * bmalloc/bmalloc.cpp:
        (bmalloc::api::tryLargeZeroedMemalignVirtual):
        (bmalloc::api::freeLargeVirtual):
        * bmalloc/bmalloc.h:

2018-04-05  Saam Barati  <sbarati@apple.com>

        IsoHeapImpl not IsoHeapImplBase should add itself to AllIsoHeaps
        https://bugs.webkit.org/show_bug.cgi?id=184174

        Reviewed by Filip Pizlo.

        Otherwise, another thread may see a non-fully formed IsoHeapImpl.

        * bmalloc/IsoHeapImpl.cpp:
        (bmalloc::IsoHeapImplBase::IsoHeapImplBase):
        (bmalloc::IsoHeapImplBase::addToAllIsoHeaps):
        * bmalloc/IsoHeapImpl.h:
        * bmalloc/IsoHeapImplInlines.h:
        (bmalloc::IsoHeapImpl<Config>::IsoHeapImpl):

2018-04-05  Yusuke Suzuki  <utatane.tea@gmail.com>

        bmalloc StaticMutex's constructor should be constexpr
        https://bugs.webkit.org/show_bug.cgi?id=180600

        Reviewed by Mark Lam.

        StaticMutex and Mutex can be unified. This patch changes std::atomic_flag in StaticMutex
        to std::atomic<bool> to add constexpr constructor to StaticMutex. Then, StaticMutex can
        be initialized in static storage without calling any static initializers.
        And we also rename StaticMutex to Mutex simply.

        * CMakeLists.txt:
        * bmalloc.xcodeproj/project.pbxproj:
        * bmalloc/AllIsoHeaps.cpp:
        (bmalloc::AllIsoHeaps::AllIsoHeaps):
        * bmalloc/AllIsoHeaps.h:
        * bmalloc/Allocator.cpp:
        (bmalloc::Allocator::tryAllocate):
        (bmalloc::Allocator::allocateImpl):
        (bmalloc::Allocator::reallocate):
        (bmalloc::Allocator::refillAllocatorSlowCase):
        (bmalloc::Allocator::allocateLarge):
        * bmalloc/CryptoRandom.cpp:
        (bmalloc::ARC4RandomNumberGenerator::ARC4RandomNumberGenerator):
        * bmalloc/Deallocator.cpp:
        (bmalloc::Deallocator::scavenge):
        (bmalloc::Deallocator::processObjectLog):
        (bmalloc::Deallocator::deallocateSlowCase):
        * bmalloc/Deallocator.h:
        (bmalloc::Deallocator::lineCache):
        * bmalloc/DebugHeap.cpp:
        (bmalloc::DebugHeap::DebugHeap):
        * bmalloc/DebugHeap.h:
        * bmalloc/Environment.cpp:
        (bmalloc::Environment::Environment):
        * bmalloc/Environment.h:
        * bmalloc/Gigacage.cpp:
        (Gigacage::disablePrimitiveGigacage):
        (Gigacage::addPrimitiveDisableCallback):
        (Gigacage::removePrimitiveDisableCallback):
        * bmalloc/Heap.cpp:
        (bmalloc::Heap::Heap):
        (bmalloc::Heap::freeableMemory):
        (bmalloc::Heap::scavenge):
        (bmalloc::Heap::deallocateLineCache):
        (bmalloc::Heap::allocateSmallChunk):
        (bmalloc::Heap::allocateSmallPage):
        (bmalloc::Heap::deallocateSmallLine):
        (bmalloc::Heap::allocateSmallBumpRangesByMetadata):
        (bmalloc::Heap::allocateSmallBumpRangesByObject):
        (bmalloc::Heap::splitAndAllocate):
        (bmalloc::Heap::tryAllocateLarge):
        (bmalloc::Heap::allocateLarge):
        (bmalloc::Heap::isLarge):
        (bmalloc::Heap::largeSize):
        (bmalloc::Heap::shrinkLarge):
        (bmalloc::Heap::deallocateLarge):
        (bmalloc::Heap::externalCommit):
        (bmalloc::Heap::externalDecommit):
        * bmalloc/Heap.h:
        (bmalloc::Heap::mutex):
        (bmalloc::Heap::allocateSmallBumpRanges):
        (bmalloc::Heap::derefSmallLine):
        * bmalloc/IsoDeallocator.h:
        * bmalloc/IsoHeap.h:
        * bmalloc/IsoTLSDeallocatorEntry.h:
        * bmalloc/IsoTLSDeallocatorEntryInlines.h:
        (bmalloc::IsoTLSDeallocatorEntry<Config>::IsoTLSDeallocatorEntry):
        * bmalloc/IsoTLSInlines.h:
        (bmalloc::IsoTLS::ensureHeap):
        * bmalloc/IsoTLSLayout.cpp:
        (bmalloc::IsoTLSLayout::IsoTLSLayout):
        (bmalloc::IsoTLSLayout::add):
        * bmalloc/IsoTLSLayout.h:
        * bmalloc/Mutex.cpp: Renamed from Source/bmalloc/bmalloc/StaticMutex.cpp.
        (bmalloc::Mutex::lockSlowCase):
        * bmalloc/Mutex.h:
        (bmalloc::sleep):
        (bmalloc::waitUntilFalse):
        (bmalloc::Mutex::try_lock):
        (bmalloc::Mutex::lock):
        (bmalloc::Mutex::unlock):
        (bmalloc::Mutex::Mutex): Deleted.
        * bmalloc/ObjectType.cpp:
        (bmalloc::objectType):
        * bmalloc/PerProcess.cpp:
        (bmalloc::getPerProcessData):
        * bmalloc/PerProcess.h:
        (bmalloc::PerProcess::mutex):
        (bmalloc::PerProcess::getSlowCase):
        * bmalloc/Scavenger.cpp:
        (bmalloc::Scavenger::Scavenger):
        (bmalloc::Scavenger::scavenge):
        (bmalloc::Scavenger::freeableMemory):
        * bmalloc/Scavenger.h:
        * bmalloc/SmallLine.h:
        (bmalloc::SmallLine::refCount):
        (bmalloc::SmallLine::ref):
        (bmalloc::SmallLine::deref):
        * bmalloc/SmallPage.h:
        (bmalloc::SmallPage::refCount):
        (bmalloc::SmallPage::hasFreeLines const):
        (bmalloc::SmallPage::setHasFreeLines):
        (bmalloc::SmallPage::ref):
        (bmalloc::SmallPage::deref):
        * bmalloc/StaticMutex.h: Removed.
        * bmalloc/VMHeap.cpp:
        (bmalloc::VMHeap::VMHeap):
        * bmalloc/VMHeap.h:
        * bmalloc/Zone.cpp:
        (bmalloc::Zone::Zone):
        * bmalloc/Zone.h:
        * bmalloc/bmalloc.cpp:
        (bmalloc::api::tryLargeZeroedMemalignVirtual):
        (bmalloc::api::freeLargeVirtual):
        (bmalloc::api::isEnabled):
        (bmalloc::api::setScavengerThreadQOSClass):
        * bmalloc/bmalloc.h:

2018-04-04  Konstantin Tokarev  <annulen@yandex.ru>

        Enable Gigacage unconditionally when building JSCOnly on macOS (build fix)
        https://bugs.webkit.org/show_bug.cgi?id=184301

        Reviewed by Yusuke Suzuki.

        bmalloc/ProcessCheck.mm implements specific behavior for Mac and iOS ports,
        which is guarded with BPLATFORM(COCOA). if we don't enable BPLATFORM(MAC)
        or BPLATFORM(IOS) in JSCOnly, then BPLATFORM(COCOA) won't be defined
        as well, and code path from ProcessCheck.mm will not be taken.

        * CMakeLists.txt: Exclude ProcessCheck.mm from port-independent file
        list.
        * PlatformMac.cmake: Build ProcessCheck.mm for Mac port.
        * bmalloc/BPlatform.h: Don't enable BPLATFORM(MAC) or BPLATFORM(IOS)
        when building JSCOnly port.

2018-04-03  Saam Barati  <sbarati@apple.com>

        totalPhysicalSize calculation when splitting a range must account for double rounding effects
        https://bugs.webkit.org/show_bug.cgi?id=184275

        Reviewed by Mark Lam.

        The rounding error could happen when we split a range where the
        range's total physical size equals the range's total size. The
        rounding may cause the left size to lose a byte, and the right
        size to gain a byte. This caused the right side to be a byte
        large than its size.

        * bmalloc/LargeRange.h:
        (bmalloc::LargeRange::LargeRange):
        (bmalloc::LargeRange::split const):

2018-04-02  Saam Barati  <sbarati@apple.com>

        bmalloc should compute its own estimate of its footprint
        https://bugs.webkit.org/show_bug.cgi?id=184121

        Reviewed by Filip Pizlo.

        This patch makes it so that bmalloc keeps track of its own physical
        footprint.
        
        Doing this for IsoHeaps is trivial. It allocates/deallocates fixed
        page sizes at a time. IsoHeapImpl just updates a count every time
        a page is committed/decommitted.
        
        Making Heap keep its footprint was a bit trickier because of how
        LargeRange is constructed. Before this patch, LargeRange kept track
        of the amount of physical memory at the start of its range. This
        patch extends large range to also keep track of the total physical memory
        in the range just for footprint bookkeeping. This was needed to make
        Heap's footprint come close to resembling reality, because as we merge and split
        large ranges, the start physical size often becomes wildly inaccurate.
        The total physical size number stored in LargeRange is still just an
        estimate. It's possible that as ranges are split, that the total physical
        size split amongst the two ranges doesn't resemble reality. This can
        happen when the total physical size is really all in one end of the split,
        but we mark it as being proportionally split amongst the resulting two
        ranges. In practice, I did not notice this being a problem. The footprint
        estimate tracks reality very closely (in my testing, within less than 1MB for
        heaps with sizes upwards of 1GB). The other nice thing about total physical
        size is that even if it diverges from reality in terms of how memory is
        using up physical RAM, it stays internally consistent inside bmalloc's
        own data structures.
        
        The main oversight of this patch is how it deals with Wasm memory. All Wasm
        memory will be viewed by bmalloc as taking up physical space even when it
        may not be. Wasm memory starts off as taking up purely virtual pages. When a
        page is first accessed, only then will the OS page it in and cause it to use
        physical RAM. I opened a bug to come up with a solution to this problem:
        https://bugs.webkit.org/show_bug.cgi?id=184207

        * bmalloc.xcodeproj/project.pbxproj:
        * bmalloc/AvailableMemory.cpp:
        (bmalloc::memoryStatus):
        * bmalloc/BPlatform.h:
        * bmalloc/Heap.cpp:
        (bmalloc::Heap::Heap):
        (bmalloc::Heap::freeableMemory):
        (bmalloc::Heap::footprint):
        (bmalloc::Heap::scavenge):
        (bmalloc::Heap::deallocateSmallChunk):
        (bmalloc::Heap::allocateSmallPage):
        (bmalloc::Heap::splitAndAllocate):
        (bmalloc::Heap::tryAllocateLarge):
        (bmalloc::Heap::shrinkLarge):
        (bmalloc::Heap::deallocateLarge):
        (bmalloc::Heap::externalCommit):
        (bmalloc::Heap::externalDecommit):
        * bmalloc/Heap.h:
        * bmalloc/IsoDirectory.h:
        * bmalloc/IsoDirectoryInlines.h:
        (bmalloc::passedNumPages>::takeFirstEligible):
        (bmalloc::passedNumPages>::didDecommit):
        (bmalloc::passedNumPages>::freeableMemory):
        * bmalloc/IsoHeapImpl.h:
        * bmalloc/IsoHeapImplInlines.h:
        (bmalloc::IsoHeapImpl<Config>::freeableMemory):
        (bmalloc::IsoHeapImpl<Config>::footprint):
        (bmalloc::IsoHeapImpl<Config>::didCommit):
        (bmalloc::IsoHeapImpl<Config>::didDecommit):
        * bmalloc/LargeRange.h:
        (bmalloc::LargeRange::LargeRange):
        (bmalloc::LargeRange::startPhysicalSize const):
        (bmalloc::LargeRange::setStartPhysicalSize):
        (bmalloc::LargeRange::totalPhysicalSize const):
        (bmalloc::LargeRange::setTotalPhysicalSize):
        (bmalloc::merge):
        (bmalloc::LargeRange::split const):
        (bmalloc::LargeRange::physicalSize const): Deleted.
        (bmalloc::LargeRange::setPhysicalSize): Deleted.
        * bmalloc/PhysicalPageMap.h: Added.
        This class is added for debugging purposes. It's useful when hacking
        on the code that calculates the footprint to use this map as a sanity
        check. It's just a simple implementation that has a set of all the committed pages.

        (bmalloc::PhysicalPageMap::commit):
        (bmalloc::PhysicalPageMap::decommit):
        (bmalloc::PhysicalPageMap::footprint):
        (bmalloc::PhysicalPageMap::forEachPhysicalPage):
        * bmalloc/Scavenger.cpp:
        (bmalloc::dumpStats):
        (bmalloc::Scavenger::scavenge):
        (bmalloc::Scavenger::freeableMemory):
        This is here just for debugging for now. But we should implement an
        efficient version of this to use when driving when to run the
        scavenger.

        (bmalloc::Scavenger::footprint):
        (bmalloc::Scavenger::threadRunLoop):
        * bmalloc/Scavenger.h:
        * bmalloc/VMAllocate.h:
        (bmalloc::physicalPageSizeSloppy):
        * bmalloc/VMHeap.cpp:
        (bmalloc::VMHeap::tryAllocateLargeChunk):
        * bmalloc/bmalloc.cpp:
        (bmalloc::api::commitAlignedPhysical):
        (bmalloc::api::decommitAlignedPhysical):
        * bmalloc/bmalloc.h:

2018-03-28  Commit Queue  <commit-queue@webkit.org>

        Unreviewed, rolling out r230005.
        https://bugs.webkit.org/show_bug.cgi?id=184115

        "it caused a huge regression on iOS" (Requested by saamyjoon
        on #webkit).

        Reverted changeset:

        "memoryStatus() is wrong in certain testing scenarios on iOS"
        https://bugs.webkit.org/show_bug.cgi?id=184050
        https://trac.webkit.org/changeset/230005

2018-03-27  Saam Barati  <sbarati@apple.com>

        memoryStatus() is wrong in certain testing scenarios on iOS
        https://bugs.webkit.org/show_bug.cgi?id=184050
        <rdar://problem/37959258>

        Rubber-stamped by Mark Lam.

        This switches us from using "phys_footprint" to using "internal + compressed"
        when computing the dirty memory in the current process. There are iOS testing
        scenarios where phys_footprint doesn't give us a reliable answer. In my testing,
        "internal + compressed" tracks phys_footprint closely (when phys_footprint is
        working). They're usually within much less than 1% of each other. We're making
        this change to ensure testing in our iOS infrastructure is valid.
        
        I opened a bug to move back to phys_footprint when it's feasible:
        https://bugs.webkit.org/show_bug.cgi?id=184050

        * bmalloc/AvailableMemory.cpp:
        (bmalloc::memoryStatus):

2018-03-20  Tim Horton  <timothy_horton@apple.com>

        Add and adopt WK_PLATFORM_NAME and adjust default feature defines
        https://bugs.webkit.org/show_bug.cgi?id=183758
        <rdar://problem/38017644>

        Reviewed by Dan Bernstein.

        * Configurations/Base.xcconfig:

2018-03-16  Filip Pizlo  <fpizlo@apple.com>

        Put the DOM in IsoHeaps
        https://bugs.webkit.org/show_bug.cgi?id=183546

        Reviewed by Simon Fraser.
        
        Make it easy to runtime-disable IsoHeaps.

        * bmalloc/Allocator.h:
        * bmalloc/IsoTLS.cpp:
        (bmalloc::IsoTLS::determineMallocFallbackState):
        * bmalloc/IsoTLS.h:
        * bmalloc/IsoTLSInlines.h:
        (bmalloc::IsoTLS::allocateSlow):
        (bmalloc::IsoTLS::deallocateSlow):

2018-03-16  Michael Catanzaro  <mcatanzaro@igalia.com>

        Improve error message when Gigacage cannot allocate virtual memory
        https://bugs.webkit.org/show_bug.cgi?id=183329

        Reviewed by Filip Pizlo.

        We've discovered that Deja Dup monitor sets a virtual memory limit, breaking Gigacage. Since
        it runs in the background on a fresh out-of-the-box install of Ubuntu, this is not good.
        That will have to be fixed by Deja Dup, but there is concern that other applications might
        try this, or that users will set a virtual memory limit for the entire desktop session. Of
        particular concern is the possibility that users might have copypasted a ulimit line into
        a session startup script without understanding it. Let's try to make it slightly easier to
        understand what's going wrong.

        * bmalloc/Gigacage.cpp:
        (Gigacage::ensureGigacage):

2018-03-13  Tim Horton  <timothy_horton@apple.com>

        Add and adopt WK_ALTERNATE_FRAMEWORKS_DIR in WTF and bmalloc
        https://bugs.webkit.org/show_bug.cgi?id=183576
        <rdar://problem/38396766>

        Reviewed by Dan Bernstein.

        * Configurations/Base.xcconfig:
        * Configurations/bmalloc.xcconfig:
        * Configurations/mbmalloc.xcconfig:

2018-03-10  Filip Pizlo  <fpizlo@apple.com>

        PerProcess<> should be safe by default
        https://bugs.webkit.org/show_bug.cgi?id=183545

        Reviewed by Yusuke Suzuki.
        
        This makes PerProcess<> safe by default, so we don't need SafePerProcess<>.
        
        The new PerProcess<> design relies on a hash-consing mechanism for PerProcess<> storage based
        on the __PRETTY_FUNCTION__ from inside PerProcess<>, which captures the instantiated type in
        the string. Therefore, this can be used to runtime-coalesce PerProcess<> instances based on
        type.
        
        I expect this to be perf-neutral. It's an important prerequisite to more bmalloc work, since I
        don't want to have more PerProcess<> vs SafePerProcess<> bugs, and SafePerProcess<> can't be
        used for everything (I don't see how to use it for isoheaps).

        * CMakeLists.txt:
        * bmalloc.xcodeproj/project.pbxproj:
        * bmalloc/Heap.cpp:
        (bmalloc::Heap::Heap):
        * bmalloc/IsoDirectoryInlines.h:
        (bmalloc::passedNumPages>::takeFirstEligible):
        (bmalloc::passedNumPages>::didBecome):
        * bmalloc/PerProcess.cpp: Added.
        (bmalloc::stringHash):
        (bmalloc::allocate):
        (bmalloc::getPerProcessData):
        * bmalloc/PerProcess.h:
        (bmalloc::PerProcess::mutex):
        (bmalloc::PerProcess::coalesce):
        (bmalloc::PerProcess::getSlowCase):
        (): Deleted.
        * bmalloc/Scavenger.cpp:
        * bmalloc/Scavenger.h:
        * bmalloc/bmalloc.cpp:
        (bmalloc::api::scavenge):
        (bmalloc::api::setScavengerThreadQOSClass):

2018-03-10  Commit Queue  <commit-queue@webkit.org>

        Unreviewed, rolling out r229436.
        https://bugs.webkit.org/show_bug.cgi?id=183542

        seems to have regressed wasm compile times by 10% (Requested
        by pizlo-mbp on #webkit).

        Reverted changeset:

        "bmalloc mutex should be adaptive"
        https://bugs.webkit.org/show_bug.cgi?id=177839
        https://trac.webkit.org/changeset/229436

2018-03-08  Filip Pizlo  <fpizlo@apple.com>

        bmalloc mutex should be adaptive
        https://bugs.webkit.org/show_bug.cgi?id=177839

        Reviewed by Michael Saboff.
        
        This pulls the WordLock algorithm into bmalloc, mostly by copy-pasting the code. We need to
        copy paste because sometimes we build WTF without bmalloc, so WTF cannot rely on bmalloc for
        anything other than malloc.

        Reland after failing to reproduce the WasmBench crash that caused it to get rolled out. Maybe that fixed
        itself somehow?

        * bmalloc/Algorithm.h:
        (bmalloc::compareExchangeWeak):
        (bmalloc::compareExchangeStrong):
        * bmalloc/PerThread.h:
        * bmalloc/StaticMutex.cpp:
        (bmalloc::StaticMutex::lockSlow):
        (bmalloc::StaticMutex::unlockSlow):
        (bmalloc::StaticMutex::lockSlowCase): Deleted.
        * bmalloc/StaticMutex.h:
        (bmalloc::StaticMutex::try_lock):
        (bmalloc::StaticMutex::isLocked const):
        (bmalloc::StaticMutex::init):
        (bmalloc::StaticMutex::tryLock):
        (bmalloc::StaticMutex::lock):
        (bmalloc::StaticMutex::unlock):
        (bmalloc::sleep): Deleted.
        (bmalloc::waitUntilFalse): Deleted.

2018-02-16  Filip Pizlo  <fpizlo@apple.com>

        Unreviewed, roll out r228306 (custom memcpy/memset) because the bots say that it was not a
        progression.

        * bmalloc/Algorithm.h:
        (bmalloc::fastCopy): Deleted.
        (bmalloc::fastZeroFill): Deleted.
        * bmalloc/Allocator.cpp:
        (bmalloc::Allocator::reallocate):
        * bmalloc/Bits.h:
        (bmalloc::BitsWordOwner::operator=):
        (bmalloc::BitsWordOwner::clearAll):
        (bmalloc::BitsWordOwner::set):
        * bmalloc/IsoPageInlines.h:
        (bmalloc::IsoPage<Config>::IsoPage):
        * bmalloc/Vector.h:
        (bmalloc::Vector<T>::reallocateBuffer):

2018-02-09  Carlos Alberto Lopez Perez  <clopez@igalia.com>

        Improve of string.h include after r228317.
        https://bugs.webkit.org/show_bug.cgi?id=182642

        Reviewed by Mark Lam.

        * bmalloc/Algorithm.h: Avoid an architecture-specific #include.

2018-02-09  Carlos Alberto Lopez Perez  <clopez@igalia.com>

        Fix build for !BCPU(X86_64) after r228306
        https://bugs.webkit.org/show_bug.cgi?id=182563

        Unreviewed build fix.

        * bmalloc/Algorithm.h:

2018-02-08  Filip Pizlo  <fpizlo@apple.com>

        Experiment with alternative implementation of memcpy/memset
        https://bugs.webkit.org/show_bug.cgi?id=182563

        Reviewed by Michael Saboff and Mark Lam.
        
        Add a faster x86_64-specific implementation of memcpy and memset. Ideally, this would just be
        implemented in WTF, but we have to copy it into bmalloc since bmalloc sits below WTF on the
        stack.

        * bmalloc/Algorithm.h:
        (bmalloc::fastCopy):
        (bmalloc::fastZeroFill):
        * bmalloc/Allocator.cpp:
        (bmalloc::Allocator::reallocate):
        * bmalloc/Bits.h:
        (bmalloc::BitsWordOwner::operator=):
        (bmalloc::BitsWordOwner::clearAll):
        (bmalloc::BitsWordOwner::set):
        * bmalloc/IsoPageInlines.h:
        (bmalloc::IsoPage<Config>::IsoPage):
        * bmalloc/Vector.h:
        (bmalloc::Vector<T>::reallocateBuffer):

2018-02-05  JF Bastien  <jfbastien@apple.com>

        Gigacage: enable only for WebContent process and token executables
        https://bugs.webkit.org/show_bug.cgi?id=182457
        <rdar://problem/35875011>

        Reviewed by Keith Miller.

        Gigacage is a solid security improvement, but it's probably best
        to roll it out incrementally to the most valuable targets first
        and progressively try out more and more over time rather than
        outright enabling it everywhere. We've gotten some reports that it
        has some side-effects that weren't expected, so for now let's
        enable it for the WebContent process, JSC, and other executables
        we know, and then later we'll enable more gigacage uses.

        For now I've chosen the following bundles:

          - com.apple.WebKit.WebContent.Development
          - com.apple.WebKit.WebContent
          - com.apple.WebProcess

        And the following processes:

          - jsc
          - wasm
          - anything starting with "test", to match the JSC tests

        I tried a different approach first, where I add a function to turn
        gigacage on or off and crash if gigacage is initialized without
        having been told what to do. Doing this in ChildProcess and a
        bunch of the process initialization methods isn't sufficient. I
        got MiniBrowser working, but some other builds use static globals
        which themselves use hash and string which are allocate with
        bmalloc and therefore which initialize gigacage before main is
        called and before the process gets a chance to opt in our out. It
        gets tricky with API calls too, because we have to do the right
        thing in any entry an API user could plausibly use, even the
        private ones, so I endend up having to initialize gigacage in e.g.
        WebPreferencesExperimentalFeatures.cpp.erb.

        Another approach could be to create a free-for-all gigacage
        entitlement, and opt-in the processes we want..

        As a follow-up we can also check that gigacage allocation always
        succeeds if it was allowed for that process. With my change I
        expect it to always succeed.

        * CMakeLists.txt:
        * bmalloc.xcodeproj/project.pbxproj:
        * bmalloc/BPlatform.h:
        * bmalloc/Gigacage.cpp:
        (Gigacage::shouldBeEnabled):
        * bmalloc/ProcessCheck.h: Added.
        (bmalloc::gigacageEnabledForProcess):
        * bmalloc/ProcessCheck.mm: Added.
        (bmalloc::gigacageEnabledForProcess):

2018-02-05  Joseph Pecoraro  <pecoraro@apple.com>

        Multiple bmalloc scavenger threads is unexpected
        https://bugs.webkit.org/show_bug.cgi?id=182474
        <rdar://problem/37175526>

        Reviewed by Filip Pizlo.

        * bmalloc/Heap.cpp:
        (bmalloc::Heap::Heap):
        * bmalloc/IsoDirectoryInlines.h:
        (bmalloc::passedNumPages>::takeFirstEligible):
        (bmalloc::passedNumPages>::didBecome):
        * bmalloc/bmalloc.cpp:
        (bmalloc::api::scavenge):
        (bmalloc::api::setScavengerThreadQOSClass):
        Switch to SafePerProcess for Scavenger to ensure one instance
        for the entire process.

        * bmalloc/PerProcess.h:
        (bmalloc::PerProcess::get):
        (bmalloc::PerProcess::getFastCase):
        (bmalloc::PerProcess::getSlowCase):
        (bmalloc::SafePerProcess::get):
        (bmalloc::SafePerProcess::getFastCase):
        (bmalloc::SafePerProcess::getSlowCase):
        Duplicate the class with a version that can ensure
        single instances by requiring exporting symbols that
        can be created with macros.

        * bmalloc/Scavenger.cpp:
        * bmalloc/Scavenger.h:
        Export symbols to ensure all images get the same instance.

2018-01-31  Saam Barati  <sbarati@apple.com>

        Replace tryLargeMemalignVirtual with tryLargeZeroedMemalignVirtual and use it to allocate large zeroed memory in Wasm
        https://bugs.webkit.org/show_bug.cgi?id=182064
        <rdar://problem/36840132>

        Reviewed by Geoffrey Garen.

        This patch replaces the tryLargeMemalignVirtual API with tryLargeZeroedMemalignVirtual.
        By doing that, we're able to remove the AllocationKind enum. To zero the memory,
        tryLargeZeroedMemalignVirtual uses mmap(... MAP_ANON ...) over previously mmapped
        memory. This both purges the any resident memory for the virtual range and ensures
        that the pages in the range are zeroed. Most OSs should implement this by taking a
        page fault and zero filling on first access. Therefore, this API is returning pages
        that will result in page faults on first access. Hence, the name 'virtual' in the API.
        This API differs from the old API in that users of it need not call madvise themselves.
        The memory is ready to go.

        * bmalloc.xcodeproj/project.pbxproj:
        * bmalloc/AllocationKind.h: Removed.
        * bmalloc/DebugHeap.cpp:
        (bmalloc::DebugHeap::memalignLarge):
        (bmalloc::DebugHeap::freeLarge):
        * bmalloc/DebugHeap.h:
        * bmalloc/Heap.cpp:
        (bmalloc::Heap::splitAndAllocate):
        (bmalloc::Heap::tryAllocateLarge):
        (bmalloc::Heap::allocateLarge):
        (bmalloc::Heap::shrinkLarge):
        (bmalloc::Heap::deallocateLarge):
        * bmalloc/Heap.h:
        * bmalloc/IsoPage.cpp:
        (bmalloc::IsoPageBase::allocatePageMemory):
        * bmalloc/VMAllocate.h:
        (bmalloc::vmZeroAndPurge):
        * bmalloc/VMHeap.cpp:
        (bmalloc::VMHeap::tryAllocateLargeChunk):
        * bmalloc/VMHeap.h:
        * bmalloc/bmalloc.cpp:
        (bmalloc::api::tryLargeZeroedMemalignVirtual):
        (bmalloc::api::freeLargeVirtual):
        (bmalloc::api::tryLargeMemalignVirtual): Deleted.
        * bmalloc/bmalloc.h:

2018-01-19  Keith Miller  <keith_miller@apple.com>

        HaveInternalSDK includes should be "#include?"
        https://bugs.webkit.org/show_bug.cgi?id=179670

        Reviewed by Dan Bernstein.

        * Configurations/Base.xcconfig:

2018-01-18  Dan Bernstein  <mitz@apple.com>

        [Xcode] Streamline and future-proof target-macOS-version-dependent build setting definitions
        https://bugs.webkit.org/show_bug.cgi?id=181803

        Reviewed by Tim Horton.

        * Configurations/Base.xcconfig: Updated.
        * Configurations/DebugRelease.xcconfig: Ditto.

2018-01-16  Michael Catanzaro  <mcatanzaro@igalia.com>

        mbmalloc should only be built in developer mode
        https://bugs.webkit.org/show_bug.cgi?id=181654

        Reviewed by Carlos Garcia Campos.

        * CMakeLists.txt:

2018-01-15  Michael Catanzaro  <mcatanzaro@igalia.com>

        Improve use of ExportMacros
        https://bugs.webkit.org/show_bug.cgi?id=181652

        Reviewed by Konstantin Tokarev.

        Disable BEXPORT on Linux ports.

        * bmalloc/BExport.h: Check for BUSE(EXPORT_MACROS).
        * bmalloc/BPlatform.h: Add BUSE(EXPORT_MACROS) and define it on macOS and iOS.

2017-12-20  Ting-Wei Lan  <lantw44@gmail.com>

        Include stdio.h before using stderr and _IONBF
        https://bugs.webkit.org/show_bug.cgi?id=181046

        Reviewed by Alex Christensen.

        * bmalloc/IsoTLS.cpp:

2017-12-14  David Kilzer  <ddkilzer@apple.com>

        Enable -Wstrict-prototypes for WebKit
        <https://webkit.org/b/180757>
        <rdar://problem/36024132>

        Rubber-stamped by Joseph Pecoraro.

        * Configurations/Base.xcconfig:
        (CLANG_WARN_STRICT_PROTOTYPES): Add. Set to YES.

2017-12-14  Saam Barati  <sbarati@apple.com>

        logVMFailure should not simulate crash on iOS
        https://bugs.webkit.org/show_bug.cgi?id=180790

        Reviewed by JF Bastien.

        The Gigacage allocation on iOS is expected to fail in certain circumstances. 
        Let's not simulate a crash on failure because since this is expected behavior.

        * bmalloc/VMAllocate.h:
        (bmalloc::tryVMAllocate):

2017-12-11  Tim Horton  <timothy_horton@apple.com>

        Stop using deprecated target conditional for simulator builds
        https://bugs.webkit.org/show_bug.cgi?id=180662
        <rdar://problem/35136156>

        Reviewed by Simon Fraser.

        * bmalloc/BPlatform.h:

2017-12-08  Saam Barati  <sbarati@apple.com>

        Enable gigacage on iOS with a 32GB runway and ensure it doesn't break WasmBench
        https://bugs.webkit.org/show_bug.cgi?id=178557

        Reviewed by Mark Lam.

        * bmalloc/Algorithm.h:
        (bmalloc::isPowerOfTwo):
        * bmalloc/Gigacage.cpp:
        * bmalloc/Gigacage.h:

2017-12-05  Andy Estes  <aestes@apple.com>

        [Darwin] Simplify use of TargetConditionals
        https://bugs.webkit.org/show_bug.cgi?id=180455
        <rdar://problem/35142971>

        Reviewed by Tim Horton.

        There's no need to check if TARGET_* macros are defined on Darwin platforms, since
        TargetConditionals.h always defines them. Also, we can simplify
        (TARGET_OS_EMBEDDED || TARGET_OS_IPHONE || TARGET_IPHONE_SIMULATOR) to TARGET_OS_IPHONE.

        * bmalloc/BPlatform.h:

2017-12-05  Filip Pizlo  <fpizlo@apple.com>

        bmalloc IsoHeap needs to allow a thread to deallocate some size for the first time
        https://bugs.webkit.org/show_bug.cgi?id=180443

        Reviewed by Saam Barati.

        It's true that we can expect a heap to already be initialized if we try to deallocate in it.  But it
        may not have its deallocator initialized on this thread yet.
        
        This is easily fixed by adding a null check on the deallocate path. That's probably not going to
        change perf at all. But doing that allows me to get rid of a lot of weird stuff I previously did to
        avoid that null check, like creating a dummy TLS in the DebugHeap case.

        * bmalloc/IsoTLS.cpp:
        (bmalloc::IsoTLS::debugFree):
        (bmalloc::IsoTLS::deallocateSlow): Deleted.
        * bmalloc/IsoTLS.h:
        * bmalloc/IsoTLSInlines.h:
        (bmalloc::IsoTLS::allocateImpl):
        (bmalloc::IsoTLS::allocateSlow):
        (bmalloc::IsoTLS::deallocateImpl):
        (bmalloc::IsoTLS::deallocateSlow):
        (bmalloc::IsoTLS::ensureHeapAndEntries):

2017-12-01  Michael Saboff  <msaboff@apple.com>

        Gigacage should not be enabled for ARM64_32
        https://bugs.webkit.org/show_bug.cgi?id=180265

        Reviewed by Saam Barati.

        Disabled Gigacage for ARM64_32.
        In the process, restructured Gigacage::shouldBeEnabled() with GIGACAGE_ENABLED set
        to 0 to avoid a dead code compiler warning.

        * bmalloc/Gigacage.cpp:
        (Gigacage::shouldBeEnabled):
        * bmalloc/Gigacage.h:

2017-11-29  JF Bastien  <jfbastien@apple.com>

        WTF / bmalloc: don't write to 0xbbadbeef when ASAN is looking
        https://bugs.webkit.org/show_bug.cgi?id=180175

        Reviewed by Mark Lam.

        ASAN knows that 0xbbadbeef is a bbad aaddress, and tells us so
        when we write to it, say in an assert. That creates bbad error
        reports where ASAN thinks we write to an invalid address, instead
        of thinking that we hit an assertion. In some cases, tooling that
        use fuzzers aggregate similar issues, and think that we just have
        the one bug and not a bunch of different asserts.

        At the same time, bmalloc's version of CRASH just writes to
        0xbbadbeef and assumes that's invalid and will crash, which isn't
        necessarily true on non-Mac platforms. WTF's version then makes
        sure there's a crash, so bmalloc should do the same.

        * bmalloc.xcodeproj/project.pbxproj:
        * bmalloc/BAssert.h:
        * bmalloc/BCompiler.h: Added.
        * bmalloc/BPlatform.h:

2017-11-27  Filip Pizlo  <fpizlo@apple.com>

        Don't crash in forEachEntry when DebugHeap is enabled.

        Unreviewed, fixing crashes on leaks bots by removing an assertion.

        * bmalloc/IsoTLS.cpp:
        (bmalloc::IsoTLS::forEachEntry):
        * test/testbmalloc.cpp: Make this test work with DebugHeap so I can catch this kind of problem in the future.

2017-11-16  Filip Pizlo  <fpizlo@apple.com>

        Isolated Heaps caused an increase in reported leaks on the bots
        https://bugs.webkit.org/show_bug.cgi?id=179463

        Reviewed by Darin Adler.
        
        This fixes the way isoheaps interact with system tools:
        
        - Opts into the VMHeap API so that the leaks tool can find isoheap memory.
        
        - Opts into the DebugHeap/Environment APIs so that we turn off isoheap allocation if memory
          debugging options are in use.

        * bmalloc.xcodeproj/project.pbxproj:
        * bmalloc/DebugHeap.h:
        * bmalloc/IsoHeap.h:
        * bmalloc/IsoPage.cpp: Added.
        (bmalloc::IsoPageBase::allocatePageMemory):
        * bmalloc/IsoPage.h:
        * bmalloc/IsoPageInlines.h:
        (bmalloc::IsoPage<Config>::tryCreate):
        * bmalloc/IsoTLS.cpp:
        (bmalloc::IsoTLS::deallocateSlow):
        (bmalloc::IsoTLS::ensureEntries):
        (bmalloc::IsoTLS::isUsingDebugHeap):
        (bmalloc::IsoTLS::debugMalloc):
        * bmalloc/IsoTLS.h:
        * bmalloc/IsoTLSInlines.h:
        (bmalloc::IsoTLS::allocate):
        (bmalloc::IsoTLS::deallocate):
        (bmalloc::IsoTLS::allocateImpl):
        (bmalloc::IsoTLS::allocateFast):
        (bmalloc::IsoTLS::allocateSlow):
        (bmalloc::IsoTLS::deallocateImpl):
        (bmalloc::IsoTLS::deallocateFast):
        (bmalloc::IsoTLS::ensureHeapAndEntries):
        (bmalloc::IsoTLS::allocator): Deleted.
        (bmalloc::IsoTLS::deallocator): Deleted.
        * bmalloc/bmalloc.cpp:
        (bmalloc::api::tryLargeMemalignVirtual):
        (bmalloc::api::freeLargeVirtual):
        (bmalloc::api::scavenge):
        (bmalloc::api::isEnabled):
        (bmalloc::api::setScavengerThreadQOSClass):
        * bmalloc/bmalloc.h:
        (bmalloc::api::tryLargeMemalignVirtual): Deleted.
        (bmalloc::api::freeLargeVirtual): Deleted.
        (bmalloc::api::scavenge): Deleted.
        (bmalloc::api::isEnabled): Deleted.
        (bmalloc::api::setScavengerThreadQOSClass): Deleted.

2017-11-14  Saam Barati  <sbarati@apple.com>

        Make the gigacage runway 32GB
        https://bugs.webkit.org/show_bug.cgi?id=175062

        Reviewed by Mark Lam.

        Making the gigacage runway 32GB defends us against buffer overflows in the
        cage reaching memory outside the cage assuming indices are 32-bit unsigned
        integers and the type they're indexing into has size <= 8 bytes. This is
        exactly the case for many things in JSC. For example, butterfly access in
        JSC meet this criteria, as does typed array access.
        
        The 32GB comes from 8 * 2^32 = 32GB.

        * bmalloc/Gigacage.cpp:

2017-11-08  Michael Catanzaro  <mcatanzaro@igalia.com>

        Gigacage.cpp:44:46: warning: ‘*’ in boolean context, suggest ‘&&’ instead [-Wint-in-bool-context]
        https://bugs.webkit.org/show_bug.cgi?id=179427

        Reviewed by Saam Barati.

        Tweak the conditional to suppress the warning.

        * bmalloc/Gigacage.cpp:
        (Gigacage::ensureGigacage):

2017-11-07  Saam Barati  <sbarati@apple.com>

        We should PROT_NONE the Gigacage runway so OOB accesses crash
        https://bugs.webkit.org/show_bug.cgi?id=179392

        Reviewed by Mark Lam.

        If we assume that an attacker will exploit JSC and cause OOB accesses,
        we should make OOB accesses in the Gigacage runway crash.

        * bmalloc/Gigacage.cpp:
        (Gigacage::ensureGigacage):

2017-10-31  Filip Pizlo  <fpizlo@apple.com>

        bmalloc should support strictly type-segregated isolated heaps
        https://bugs.webkit.org/show_bug.cgi?id=178108

        Reviewed by Saam Barati, Simon Fraser, and Ryosuke Niwa.
        
        This introduces a new allocation API in bmalloc called IsoHeap. An IsoHeap is templatized by
        type and created in static storage. When unused, it takes only a few words. When you do use
        it, each IsoHeap gets a bag of virtual pages unique to it. This prevents use-after-free bugs
        in one IsoHeap from affecting any other memory. At worst, two pointers of the same type will
        point to the same object even though they should not have.
        
        IsoHeaps allocate using a first-fit discipline that combines ideas from bmalloc and Riptide
        (the JSC GC):
        
        Like Riptide, it uses a bump'n'pop allocator. What Riptide calls blocks, IsoHeaps calls
        pages. Pages are collected into directories. Directories track pages using bitvectors, so
        that it's easy to quickly find a completely free page or one that has at least one free
        object. I think that the bump'n'pop allocator is as fast as the bmalloc Immix-style (page and
        line) allocator, but is better at allocating in holes. It's guaranteed to follow a first-fit
        discipline. However, the real reason why I wrote it that was is that this is what I'm more
        familiar with. This is a part of the design I want to revisit (bug 179278).
        
        Like bmalloc, it uses a deallocation log. This means that the internal IsoHeap data
        structures can be locked with a coarse-grained lock, since the deallocator only grabs it when
        flushing the log. Similarly, the allocator only grabs it when refilling the bump'n'pop
        FreeList.
        
        This adds a unit test for IsoHeaps. In this change, IsoHeaps are adopted only by WebCore's
        RenderObject.
        
        Note that despite the use of GC concepts, it's not a goal to make this code directly sharable
        with GC. The GC will probably have to do isolated heaps its own way (likely a special
        Subspace or something like that).

        * bmalloc.xcodeproj/project.pbxproj:
        * bmalloc/Algorithm.h:
        (bmalloc::findBitInWord):
        * bmalloc/AllIsoHeaps.cpp: Added.
        (bmalloc::AllIsoHeaps::AllIsoHeaps):
        (bmalloc::AllIsoHeaps::add):
        (bmalloc::AllIsoHeaps::head):
        * bmalloc/AllIsoHeaps.h: Added.
        * bmalloc/AllIsoHeapsInlines.h: Added.
        (bmalloc::AllIsoHeaps::forEach):
        * bmalloc/BMalloced.h: Added.
        * bmalloc/Bits.h: Added.
        (bmalloc::bitsArrayLength):
        (bmalloc::BitsWordView::BitsWordView):
        (bmalloc::BitsWordView::numBits const):
        (bmalloc::BitsWordView::word const):
        (bmalloc::BitsWordOwner::BitsWordOwner):
        (bmalloc::BitsWordOwner::view const):
        (bmalloc::BitsWordOwner::operator=):
        (bmalloc::BitsWordOwner::setAll):
        (bmalloc::BitsWordOwner::clearAll):
        (bmalloc::BitsWordOwner::set):
        (bmalloc::BitsWordOwner::numBits const):
        (bmalloc::BitsWordOwner::arrayLength const):
        (bmalloc::BitsWordOwner::word const):
        (bmalloc::BitsWordOwner::word):
        (bmalloc::BitsWordOwner::words const):
        (bmalloc::BitsWordOwner::words):
        (bmalloc::BitsAndWords::BitsAndWords):
        (bmalloc::BitsAndWords::view const):
        (bmalloc::BitsAndWords::numBits const):
        (bmalloc::BitsAndWords::word const):
        (bmalloc::BitsOrWords::BitsOrWords):
        (bmalloc::BitsOrWords::view const):
        (bmalloc::BitsOrWords::numBits const):
        (bmalloc::BitsOrWords::word const):
        (bmalloc::BitsNotWords::BitsNotWords):
        (bmalloc::BitsNotWords::view const):
        (bmalloc::BitsNotWords::numBits const):
        (bmalloc::BitsNotWords::word const):
        (bmalloc::BitsImpl::BitsImpl):
        (bmalloc::BitsImpl::numBits const):
        (bmalloc::BitsImpl::size const):
        (bmalloc::BitsImpl::arrayLength const):
        (bmalloc::BitsImpl::operator== const):
        (bmalloc::BitsImpl::operator!= const):
        (bmalloc::BitsImpl::at const):
        (bmalloc::BitsImpl::operator[] const):
        (bmalloc::BitsImpl::isEmpty const):
        (bmalloc::BitsImpl::operator& const):
        (bmalloc::BitsImpl::operator| const):
        (bmalloc::BitsImpl::operator~ const):
        (bmalloc::BitsImpl::forEachSetBit const):
        (bmalloc::BitsImpl::forEachClearBit const):
        (bmalloc::BitsImpl::forEachBit const):
        (bmalloc::BitsImpl::findBit const):
        (bmalloc::BitsImpl::findSetBit const):
        (bmalloc::BitsImpl::findClearBit const):
        (bmalloc::BitsImpl::wordView const):
        (bmalloc::BitsImpl::atImpl const):
        (bmalloc::Bits::Bits):
        (bmalloc::Bits::operator=):
        (bmalloc::Bits::resize):
        (bmalloc::Bits::setAll):
        (bmalloc::Bits::clearAll):
        (bmalloc::Bits::setAndCheck):
        (bmalloc::Bits::operator|=):
        (bmalloc::Bits::operator&=):
        (bmalloc::Bits::at const):
        (bmalloc::Bits::operator[] const):
        (bmalloc::Bits::BitReference::BitReference):
        (bmalloc::Bits::BitReference::operator bool const):
        (bmalloc::Bits::BitReference::operator=):
        (bmalloc::Bits::at):
        (bmalloc::Bits::operator[]):
        * bmalloc/CryptoRandom.cpp: Replaced with Source/bmalloc/bmalloc/CryptoRandom.cpp.
        (bmalloc::cryptoRandom):
        * bmalloc/CryptoRandom.h: Replaced with Source/bmalloc/bmalloc/CryptoRandom.h.
        * bmalloc/DeferredDecommit.h: Added.
        * bmalloc/DeferredDecommitInlines.h: Added.
        (bmalloc::DeferredDecommit::DeferredDecommit):
        * bmalloc/DeferredTrigger.h: Added.
        (bmalloc::DeferredTrigger::DeferredTrigger):
        * bmalloc/DeferredTriggerInlines.h: Added.
        (bmalloc::DeferredTrigger<trigger>::didBecome):
        (bmalloc::DeferredTrigger<trigger>::handleDeferral):
        * bmalloc/EligibilityResult.h: Added.
        (bmalloc::EligibilityResult::EligibilityResult):
        * bmalloc/EligibilityResultInlines.h: Added.
        (bmalloc::EligibilityResult<Config>::EligibilityResult):
        * bmalloc/FixedVector.h:
        * bmalloc/FreeList.cpp: Added.
        (bmalloc::FreeList::FreeList):
        (bmalloc::FreeList::~FreeList):
        (bmalloc::FreeList::clear):
        (bmalloc::FreeList::initializeList):
        (bmalloc::FreeList::initializeBump):
        (bmalloc::FreeList::contains const):
        * bmalloc/FreeList.h: Added.
        (bmalloc::FreeCell::scramble):
        (bmalloc::FreeCell::descramble):
        (bmalloc::FreeCell::setNext):
        (bmalloc::FreeCell::next const):
        (bmalloc::FreeList::allocationWillFail const):
        (bmalloc::FreeList::allocationWillSucceed const):
        (bmalloc::FreeList::originalSize const):
        (bmalloc::FreeList::head const):
        * bmalloc/FreeListInlines.h: Added.
        (bmalloc::FreeList::allocate):
        (bmalloc::FreeList::forEach const):
        * bmalloc/IsoAllocator.h: Added.
        * bmalloc/IsoAllocatorInlines.h: Added.
        (bmalloc::IsoAllocator<Config>::IsoAllocator):
        (bmalloc::IsoAllocator<Config>::~IsoAllocator):
        (bmalloc::IsoAllocator<Config>::allocate):
        (bmalloc::IsoAllocator<Config>::allocateSlow):
        (bmalloc::IsoAllocator<Config>::scavenge):
        * bmalloc/IsoConfig.h: Added.
        * bmalloc/IsoDeallocator.h: Added.
        * bmalloc/IsoDeallocatorInlines.h: Added.
        (bmalloc::IsoDeallocator<Config>::IsoDeallocator):
        (bmalloc::IsoDeallocator<Config>::~IsoDeallocator):
        (bmalloc::IsoDeallocator<Config>::deallocate):
        (bmalloc::IsoDeallocator<Config>::scavenge):
        * bmalloc/IsoDirectory.h: Added.
        (bmalloc::IsoDirectoryBaseBase::IsoDirectoryBaseBase):
        (bmalloc::IsoDirectoryBaseBase::~IsoDirectoryBaseBase):
        (bmalloc::IsoDirectoryBase::heap):
        * bmalloc/IsoDirectoryInlines.h: Added.
        (bmalloc::IsoDirectoryBase<Config>::IsoDirectoryBase):
        (bmalloc::passedNumPages>::IsoDirectory):
        (bmalloc::passedNumPages>::takeFirstEligible):
        (bmalloc::passedNumPages>::didBecome):
        (bmalloc::passedNumPages>::didDecommit):
        (bmalloc::passedNumPages>::scavenge):
        (bmalloc::passedNumPages>::forEachCommittedPage):
        * bmalloc/IsoDirectoryPage.h: Added.
        (bmalloc::IsoDirectoryPage::index const):
        * bmalloc/IsoDirectoryPageInlines.h: Added.
        (bmalloc::IsoDirectoryPage<Config>::IsoDirectoryPage):
        (bmalloc::IsoDirectoryPage<Config>::pageFor):
        * bmalloc/IsoHeap.h: Added.
        (bmalloc::api::IsoHeap::allocatorOffset):
        (bmalloc::api::IsoHeap::setAllocatorOffset):
        (bmalloc::api::IsoHeap::deallocatorOffset):
        (bmalloc::api::IsoHeap::setDeallocatorOffset):
        * bmalloc/IsoHeapImpl.cpp: Added.
        (bmalloc::IsoHeapImplBase::IsoHeapImplBase):
        (bmalloc::IsoHeapImplBase::~IsoHeapImplBase):
        (bmalloc::IsoHeapImplBase::scavengeNow):
        (bmalloc::IsoHeapImplBase::finishScavenging):
        * bmalloc/IsoHeapImpl.h: Added.
        * bmalloc/IsoHeapImplInlines.h: Added.
        (bmalloc::IsoHeapImpl<Config>::IsoHeapImpl):
        (bmalloc::IsoHeapImpl<Config>::takeFirstEligible):
        (bmalloc::IsoHeapImpl<Config>::didBecomeEligible):
        (bmalloc::IsoHeapImpl<Config>::scavenge):
        (bmalloc::IsoHeapImpl<Config>::allocatorOffset):
        (bmalloc::IsoHeapImpl<Config>::deallocatorOffset):
        (bmalloc::IsoHeapImpl<Config>::numLiveObjects):
        (bmalloc::IsoHeapImpl<Config>::numCommittedPages):
        (bmalloc::IsoHeapImpl<Config>::forEachDirectory):
        (bmalloc::IsoHeapImpl<Config>::forEachCommittedPage):
        (bmalloc::IsoHeapImpl<Config>::forEachLiveObject):
        * bmalloc/IsoHeapInlines.h: Added.
        (bmalloc::api::IsoHeap<Type>::allocate):
        (bmalloc::api::IsoHeap<Type>::tryAllocate):
        (bmalloc::api::IsoHeap<Type>::deallocate):
        (bmalloc::api::IsoHeap<Type>::scavenge):
        (bmalloc::api::IsoHeap<Type>::isInitialized):
        (bmalloc::api::IsoHeap<Type>::impl):
        * bmalloc/IsoPage.h: Added.
        (bmalloc::IsoPage::index const):
        (bmalloc::IsoPage::directory):
        (bmalloc::IsoPage::isInUseForAllocation const):
        (bmalloc::IsoPage::indexOfFirstObject):
        * bmalloc/IsoPageInlines.h: Added.
        (bmalloc::IsoPage<Config>::tryCreate):
        (bmalloc::IsoPage<Config>::IsoPage):
        (bmalloc::IsoPage<Config>::free):
        (bmalloc::IsoPage<Config>::startAllocating):
        (bmalloc::IsoPage<Config>::stopAllocating):
        (bmalloc::IsoPage<Config>::forEachLiveObject):
        * bmalloc/IsoPageTrigger.h: Added.
        * bmalloc/IsoTLS.cpp: Added.
        (bmalloc::IsoTLS::scavenge):
        (bmalloc::IsoTLS::IsoTLS):
        (bmalloc::IsoTLS::ensureEntries):
        (bmalloc::IsoTLS::destructor):
        (bmalloc::IsoTLS::sizeForCapacity):
        (bmalloc::IsoTLS::capacityForSize):
        (bmalloc::IsoTLS::size):
        (bmalloc::IsoTLS::forEachEntry):
        * bmalloc/IsoTLS.h: Added.
        * bmalloc/IsoTLSAllocatorEntry.h: Added.
        * bmalloc/IsoTLSAllocatorEntryInlines.h: Added.
        (bmalloc::IsoTLSAllocatorEntry<Config>::IsoTLSAllocatorEntry):
        (bmalloc::IsoTLSAllocatorEntry<Config>::~IsoTLSAllocatorEntry):
        (bmalloc::IsoTLSAllocatorEntry<Config>::construct):
        * bmalloc/IsoTLSDeallocatorEntry.h: Added.
        * bmalloc/IsoTLSDeallocatorEntryInlines.h: Added.
        (bmalloc::IsoTLSDeallocatorEntry<Config>::IsoTLSDeallocatorEntry):
        (bmalloc::IsoTLSDeallocatorEntry<Config>::~IsoTLSDeallocatorEntry):
        (bmalloc::IsoTLSDeallocatorEntry<Config>::construct):
        * bmalloc/IsoTLSEntry.cpp: Added.
        (bmalloc::IsoTLSEntry::IsoTLSEntry):
        (bmalloc::IsoTLSEntry::~IsoTLSEntry):
        * bmalloc/IsoTLSEntry.h: Added.
        (bmalloc::IsoTLSEntry::offset const):
        (bmalloc::IsoTLSEntry::alignment const):
        (bmalloc::IsoTLSEntry::size const):
        (bmalloc::IsoTLSEntry::extent const):
        * bmalloc/IsoTLSEntryInlines.h: Added.
        (bmalloc::IsoTLSEntry::walkUpToInclusive):
        (bmalloc::DefaultIsoTLSEntry<EntryType>::DefaultIsoTLSEntry):
        (bmalloc::DefaultIsoTLSEntry<EntryType>::~DefaultIsoTLSEntry):
        (bmalloc::Defa