[WebAssembly] Create a Wasm interpreter
authortzagallo@apple.com <tzagallo@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 31 Oct 2019 22:32:52 +0000 (22:32 +0000)
committertzagallo@apple.com <tzagallo@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 31 Oct 2019 22:32:52 +0000 (22:32 +0000)
https://bugs.webkit.org/show_bug.cgi?id=194257
<rdar://problem/44186794>

Reviewed by Saam Barati.

Source/JavaScriptCore:

Add an interpreter tier to WebAssembly which reuses the LLInt infrastructure. The interpreter
currently tiers up straight to OMG and can OSR enter at the prologue and from loops. The initial
implementation of the interpreter is very naive, but despite the lack of optimizations it still
shows a 2x improvement on the WebAssembly subtests in JetStream2 and 2x improvement on the
PSPDFKit benchmark. It reduces "compilation" times by ~3x and it's neutral on throughput.

The interpreter follows the same calling conventions as the BBQ/OMG, this means that:
- We have to allocate locals for all argument registers and write all arguments registers to the
  stack in the prologue.
- Calls have to allocate space for at least as many arguments as the number of argument registers.
  Before each call, all argument registers must be loaded from the stack, and after we return from
  the call, all registers must be stored back to the stack, in case they contain return values. We
  carefully layout the stack so that the arguments that would already have to be passed in the stack
  end up in the right place. The stack layout for calls is:
    [ gprs ][ fprs ][ optional stack arguments ][ callee frame ]
                                                               ^ sp
- The return opcode has to load all registers from the stack, since they might need to contain
  results of the function.
- The calling convention requires that the callee should store itself in the callee slot of the call
  frame, which is impossible in the interpreter, since the code we execute is the same for all callees.
  In order to work around that, we generate an entry thunk to the wasm interpreter for each function.
  All this thunk does is store the callee in the call frame and tail call the interpreter.

* CMakeLists.txt:
* DerivedSources-input.xcfilelist:
* DerivedSources-output.xcfilelist:
* DerivedSources.make:
* JavaScriptCore.xcodeproj/project.pbxproj:
* Sources.txt:
* bytecode/BytecodeDumper.cpp:
(JSC::BytecodeDumper<Block>::constantName const):
(JSC::BytecodeDumper<Block>::dumpValue):
(JSC::BytecodeDumper<Block>::dumpBytecode):
(JSC::CodeBlockBytecodeDumper<Block>::vm const):
(JSC::CodeBlockBytecodeDumper<Block>::identifier const):
(JSC::CodeBlockBytecodeDumper<Block>::dumpIdentifiers):
(JSC::CodeBlockBytecodeDumper<Block>::dumpConstants):
(JSC::CodeBlockBytecodeDumper<Block>::dumpExceptionHandlers):
(JSC::CodeBlockBytecodeDumper<Block>::dumpSwitchJumpTables):
(JSC::CodeBlockBytecodeDumper<Block>::dumpStringSwitchJumpTables):
(JSC::CodeBlockBytecodeDumper<Block>::dumpBlock):
* bytecode/BytecodeDumper.h:
(JSC::BytecodeDumper::dumpValue):
(JSC::BytecodeDumper::BytecodeDumper):
* bytecode/BytecodeGeneratorification.cpp:
(JSC::performGeneratorification):
* bytecode/BytecodeList.rb:
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::dumpBytecode):
* bytecode/Fits.h:
* bytecode/Instruction.h:
(JSC::BaseInstruction::BaseInstruction):
(JSC::BaseInstruction::Impl::opcodeID const):
(JSC::BaseInstruction::opcodeID const):
(JSC::BaseInstruction::name const):
(JSC::BaseInstruction::isWide16 const):
(JSC::BaseInstruction::isWide32 const):
(JSC::BaseInstruction::hasMetadata const):
(JSC::BaseInstruction::sizeShiftAmount const):
(JSC::BaseInstruction::size const):
(JSC::BaseInstruction::is const):
(JSC::BaseInstruction::as const):
(JSC::BaseInstruction::cast):
(JSC::BaseInstruction::cast const):
(JSC::BaseInstruction::wide16 const):
(JSC::BaseInstruction::wide32 const):
* bytecode/InstructionStream.h:
(JSC::InstructionStream::iterator::operator+=):
(JSC::InstructionStream::iterator::operator++):
(JSC::InstructionStreamWriter::iterator::operator+=):
(JSC::InstructionStreamWriter::iterator::operator++):
* bytecode/Opcode.cpp:
* bytecode/Opcode.h:
* bytecode/PreciseJumpTargetsInlines.h:
* bytecode/UnlinkedCodeBlock.h:
* bytecode/VirtualRegister.cpp:
(JSC::VirtualRegister::VirtualRegister):
* bytecode/VirtualRegister.h:
* bytecompiler/BytecodeGenerator.cpp:
(JSC::GenericLabel<JSGeneratorTraits>::setLocation):
(JSC::BytecodeGenerator::BytecodeGenerator):
* bytecompiler/BytecodeGenerator.h:
* bytecompiler/BytecodeGeneratorBase.h: Added.
* bytecompiler/BytecodeGeneratorBaseInlines.h: Added.
(JSC::shrinkToFit):
(JSC::BytecodeGeneratorBase<Traits>::BytecodeGeneratorBase):
(JSC::BytecodeGeneratorBase<Traits>::newLabel):
(JSC::BytecodeGeneratorBase<Traits>::newEmittedLabel):
(JSC::BytecodeGeneratorBase<Traits>::reclaimFreeRegisters):
(JSC::BytecodeGeneratorBase<Traits>::emitLabel):
(JSC::BytecodeGeneratorBase<Traits>::recordOpcode):
(JSC::BytecodeGeneratorBase<Traits>::alignWideOpcode16):
(JSC::BytecodeGeneratorBase<Traits>::alignWideOpcode32):
(JSC::BytecodeGeneratorBase<Traits>::write):
(JSC::BytecodeGeneratorBase<Traits>::newRegister):
(JSC::BytecodeGeneratorBase<Traits>::newTemporary):
(JSC::BytecodeGeneratorBase<Traits>::addVar):
(JSC::BytecodeGeneratorBase<Traits>::allocateCalleeSaveSpace):
* bytecompiler/Label.h:
(JSC::GenericBoundLabel::GenericBoundLabel):
(JSC::GenericBoundLabel::target):
(JSC::GenericBoundLabel::saveTarget):
(JSC::GenericBoundLabel::commitTarget):
* dfg/DFGByteCodeParser.cpp:
* dfg/DFGCapabilities.cpp:
(JSC::DFG::capabilityLevel):
* dfg/DFGOperations.cpp:
* generator/Argument.rb:
* generator/DSL.rb:
* generator/GeneratedFile.rb:
* generator/Opcode.rb:
* generator/Options.rb:
* generator/Section.rb:
* generator/Wasm.rb: Added.
* interpreter/Register.h:
* interpreter/RegisterInlines.h:
(JSC::Register::operator=):
* jit/JITArithmetic.cpp:
* jit/JITOpcodes.cpp:
* llint/LLIntData.cpp:
(JSC::LLInt::initialize):
* llint/LLIntData.h:
(JSC::LLInt::wasmExceptionInstructions):
* llint/LLIntOfflineAsmConfig.h:
* llint/LLIntOffsetsExtractor.cpp:
* llint/LLIntSlowPaths.cpp:
* llint/LLIntThunks.cpp:
(JSC::LLInt::generateThunkWithJumpTo):
(JSC::LLInt::wasmFunctionEntryThunk):
* llint/LLIntThunks.h:
* llint/LowLevelInterpreter.asm:
* llint/LowLevelInterpreter32_64.asm:
* llint/LowLevelInterpreter64.asm:
* llint/WebAssembly.asm: Added.
* offlineasm/arm64.rb:
* offlineasm/instructions.rb:
* offlineasm/parser.rb:
* offlineasm/registers.rb:
* offlineasm/transform.rb:
* offlineasm/x86.rb:
* parser/Nodes.h:
* runtime/Error.cpp:
(JSC::FindFirstCallerFrameWithCodeblockFunctor::operator() const):
* runtime/ErrorInstance.cpp:
(JSC::ErrorInstance::finishCreation):
* runtime/Options.cpp:
(JSC::overrideDefaults):
* runtime/OptionsList.h:
* runtime/SamplingProfiler.cpp:
(JSC::FrameWalker::recordJITFrame):
(JSC::FrameWalker::resetAtMachineFrame):
* wasm/WasmAirIRGenerator.cpp:
(JSC::Wasm::AirIRGenerator::isControlTypeIf):
(JSC::Wasm::AirIRGenerator::emitLoopTierUpCheck):
* wasm/WasmB3IRGenerator.cpp:
(JSC::Wasm::B3IRGenerator::isControlTypeIf):
* wasm/WasmBBQPlan.cpp:
(JSC::Wasm::BBQPlan::prepareImpl):
(JSC::Wasm::BBQPlan::work):
(JSC::Wasm::BBQPlan::compileFunction):
(JSC::Wasm::BBQPlan::didCompleteCompilation):
(JSC::Wasm::BBQPlan::initializeCallees):
* wasm/WasmBBQPlan.h:
* wasm/WasmBBQPlanInlines.h: Removed.
* wasm/WasmCallee.cpp:
(JSC::Wasm::Callee::Callee):
(JSC::Wasm::Callee::dump const):
(JSC::Wasm::JITCallee::JITCallee):
(JSC::Wasm::LLIntCallee::setEntrypoint):
(JSC::Wasm::LLIntCallee::entrypoint const):
(JSC::Wasm::LLIntCallee::calleeSaveRegisters):
(JSC::Wasm:: const):
* wasm/WasmCallee.h:
(JSC::Wasm::Callee::setOSREntryCallee):
(JSC::Wasm::JITCallee::wasmToWasmCallsites):
(JSC::Wasm::JITCallee:: const):
* wasm/WasmCallingConvention.h:
* wasm/WasmCodeBlock.cpp:
(JSC::Wasm::CodeBlock::CodeBlock):
* wasm/WasmCodeBlock.h:
(JSC::Wasm::CodeBlock::wasmEntrypointCalleeFromFunctionIndexSpace):
(JSC::Wasm::CodeBlock::wasmBBQCalleeFromFunctionIndexSpace):
(JSC::Wasm::CodeBlock::wasmToWasmExitStub):
* wasm/WasmCompilationMode.cpp:
(JSC::Wasm::makeString):
* wasm/WasmCompilationMode.h:
* wasm/WasmEmbedder.h:
* wasm/WasmEntryPlan.cpp: Added.
(JSC::Wasm::EntryPlan::EntryPlan):
(JSC::Wasm::EntryPlan::stateString):
(JSC::Wasm::EntryPlan::moveToState):
(JSC::Wasm::EntryPlan::didReceiveFunctionData):
(JSC::Wasm::EntryPlan::parseAndValidateModule):
(JSC::Wasm::EntryPlan::prepare):
(JSC::Wasm::EntryPlan::ThreadCountHolder::ThreadCountHolder):
(JSC::Wasm::EntryPlan::ThreadCountHolder::~ThreadCountHolder):
(JSC::Wasm::EntryPlan::complete):
(JSC::Wasm::EntryPlan::compileFunctions):
(JSC::Wasm::EntryPlan::work):
* wasm/WasmEntryPlan.h: Copied from Source/JavaScriptCore/wasm/WasmBBQPlan.h.
(JSC::Wasm::EntryPlan::parseAndValidateModule):
(JSC::Wasm::EntryPlan::exports const):
(JSC::Wasm::EntryPlan::internalFunctionCount const):
(JSC::Wasm::EntryPlan::takeModuleInformation):
(JSC::Wasm::EntryPlan::takeWasmToWasmExitStubs):
(JSC::Wasm::EntryPlan::takeWasmToWasmCallsites):
(JSC::Wasm::EntryPlan::hasBeenPrepared const):
(JSC::Wasm::EntryPlan::tryReserveCapacity):
* wasm/WasmFunctionCodeBlock.cpp: Added.
(JSC::Wasm::FunctionCodeBlock::setInstructions):
(JSC::Wasm::FunctionCodeBlock::dumpBytecode):
(JSC::Wasm::FunctionCodeBlock::addOutOfLineJumpTarget):
(JSC::Wasm::FunctionCodeBlock::outOfLineJumpOffset):
(JSC::Wasm::FunctionCodeBlock::outOfLineJumpTarget):
(JSC::Wasm::FunctionCodeBlock::addSignature):
(JSC::Wasm::FunctionCodeBlock::signature const):
(JSC::Wasm::FunctionCodeBlock::addJumpTable):
(JSC::Wasm::FunctionCodeBlock::jumpTable const const):
(JSC::Wasm::FunctionCodeBlock::numberOfJumpTables const):
* wasm/WasmFunctionCodeBlock.h: Added.
(JSC::Wasm::FunctionCodeBlock::FunctionCodeBlock):
(JSC::Wasm::FunctionCodeBlock::getConstant const):
(JSC::Wasm::FunctionCodeBlock::functionIndex const):
(JSC::Wasm::FunctionCodeBlock::addJumpTarget):
(JSC::Wasm::FunctionCodeBlock::numberOfJumpTargets):
(JSC::Wasm::FunctionCodeBlock::lastJumpTarget):
(JSC::Wasm::FunctionCodeBlock::outOfLineJumpOffset):
(JSC::Wasm::FunctionCodeBlock::bytecodeOffset):
(JSC::Wasm::FunctionCodeBlock::tierUpCounter):
* wasm/WasmFunctionParser.h:
(JSC::Wasm::FunctionParser<Context>::parseExpression):
(JSC::Wasm::FunctionParser<Context>::parseUnreachableExpression):
* wasm/WasmInstance.h:
* wasm/WasmLLIntGenerator.cpp: Added.
(JSC::Wasm::LLIntGenerator::ControlType::ControlType):
(JSC::Wasm::LLIntGenerator::ControlType::loop):
(JSC::Wasm::LLIntGenerator::ControlType::topLevel):
(JSC::Wasm::LLIntGenerator::ControlType::block):
(JSC::Wasm::LLIntGenerator::ControlType::if_):
(JSC::Wasm::LLIntGenerator::ControlType::targetLabelForBranch const):
(JSC::Wasm::LLIntGenerator::fail const):
(JSC::Wasm::LLIntGenerator::unifyValuesWithBlock):
(JSC::Wasm::LLIntGenerator::emptyExpression):
(JSC::Wasm::LLIntGenerator::createStack):
(JSC::Wasm::LLIntGenerator::isControlTypeIf):
(JSC::Wasm::LLIntGenerator::addEndToUnreachable):
(JSC::Wasm::LLIntGenerator::setParser):
(JSC::Wasm::LLIntGenerator::dump):
(JSC::Wasm::LLIntGenerator::virtualRegisterForLocal):
(JSC::Wasm::LLIntGenerator::tmpsForSignature):
(JSC::Wasm::LLIntGenerator::jsNullConstant):
(JSC::Wasm::LLIntGenerator::isConstant):
(JSC::Wasm::parseAndCompileBytecode):
(JSC::Wasm::LLIntGenerator::LLIntGenerator):
(JSC::Wasm::LLIntGenerator::finalize):
(JSC::Wasm::LLIntGenerator::callInformationFor):
(JSC::Wasm::LLIntGenerator::addArguments):
(JSC::Wasm::LLIntGenerator::addLocal):
(JSC::Wasm::LLIntGenerator::addConstant):
(JSC::Wasm::LLIntGenerator::getLocal):
(JSC::Wasm::LLIntGenerator::setLocal):
(JSC::Wasm::LLIntGenerator::getGlobal):
(JSC::Wasm::LLIntGenerator::setGlobal):
(JSC::Wasm::LLIntGenerator::addLoop):
(JSC::Wasm::LLIntGenerator::addTopLevel):
(JSC::Wasm::LLIntGenerator::addBlock):
(JSC::Wasm::LLIntGenerator::addIf):
(JSC::Wasm::LLIntGenerator::addElse):
(JSC::Wasm::LLIntGenerator::addElseToUnreachable):
(JSC::Wasm::LLIntGenerator::addReturn):
(JSC::Wasm::LLIntGenerator::addBranch):
(JSC::Wasm::LLIntGenerator::addSwitch):
(JSC::Wasm::LLIntGenerator::endBlock):
(JSC::Wasm::LLIntGenerator::addCall):
(JSC::Wasm::LLIntGenerator::addCallIndirect):
(JSC::Wasm::LLIntGenerator::addRefIsNull):
(JSC::Wasm::LLIntGenerator::addRefFunc):
(JSC::Wasm::LLIntGenerator::addTableGet):
(JSC::Wasm::LLIntGenerator::addTableSet):
(JSC::Wasm::LLIntGenerator::addTableSize):
(JSC::Wasm::LLIntGenerator::addTableGrow):
(JSC::Wasm::LLIntGenerator::addTableFill):
(JSC::Wasm::LLIntGenerator::addUnreachable):
(JSC::Wasm::LLIntGenerator::addCurrentMemory):
(JSC::Wasm::LLIntGenerator::addGrowMemory):
(JSC::Wasm::LLIntGenerator::addSelect):
(JSC::Wasm::LLIntGenerator::load):
(JSC::Wasm::LLIntGenerator::store):
(JSC::GenericLabel<Wasm::GeneratorTraits>::setLocation):
* wasm/WasmLLIntGenerator.h: Copied from Source/JavaScriptCore/wasm/WasmCompilationMode.h.
* wasm/WasmLLIntPlan.cpp: Added.
(JSC::Wasm::LLIntPlan::prepareImpl):
(JSC::Wasm::LLIntPlan::compileFunction):
(JSC::Wasm::LLIntPlan::didCompleteCompilation):
(JSC::Wasm::LLIntPlan::initializeCallees):
* wasm/WasmLLIntPlan.h: Copied from Source/JavaScriptCore/wasm/WasmOMGForOSREntryPlan.h.
* wasm/WasmLLIntTierUpCounter.cpp: Copied from Source/JavaScriptCore/wasm/WasmCompilationMode.cpp.
(JSC::Wasm::LLIntTierUpCounter::addOSREntryDataForLoop):
(JSC::Wasm::LLIntTierUpCounter::osrEntryDataForLoop const const):
* wasm/WasmLLIntTierUpCounter.h: Copied from Source/JavaScriptCore/wasm/WasmOMGForOSREntryPlan.h.
(JSC::Wasm::LLIntTierUpCounter::LLIntTierUpCounter):
(JSC::Wasm::LLIntTierUpCounter::optimizeAfterWarmUp):
(JSC::Wasm::LLIntTierUpCounter::checkIfOptimizationThresholdReached):
(JSC::Wasm::LLIntTierUpCounter::optimizeSoon):
* wasm/WasmMemoryInformation.cpp:
(JSC::Wasm::PinnedRegisterInfo::get):
* wasm/WasmModule.cpp:
(JSC::Wasm::makeValidationResult):
(JSC::Wasm::makeValidationCallback):
(JSC::Wasm::Module::validateSync):
(JSC::Wasm::Module::validateAsync):
* wasm/WasmOMGForOSREntryPlan.cpp:
(JSC::Wasm::OMGForOSREntryPlan::OMGForOSREntryPlan):
(JSC::Wasm::OMGForOSREntryPlan::work):
* wasm/WasmOMGForOSREntryPlan.h:
* wasm/WasmOMGPlan.cpp:
(JSC::Wasm::OMGPlan::work):
* wasm/WasmSlowPaths.cpp: Added.
(JSC::LLInt::jitCompileAndSetHeuristics):
(JSC::LLInt::WASM_SLOW_PATH_DECL):
(JSC::LLInt::doWasmCall):
(JSC::LLInt::doWasmCallIndirect):
(JSC::LLInt::slow_path_wasm_throw_exception):
(JSC::LLInt::slow_path_wasm_popcount):
(JSC::LLInt::slow_path_wasm_popcountll):
* wasm/WasmSlowPaths.h: Added.
* wasm/WasmTable.cpp:
(JSC::Wasm::FuncRefTable::function const):
(JSC::Wasm::FuncRefTable::instance const):
* wasm/WasmTable.h:
* wasm/WasmTierUpCount.h:
* wasm/WasmValidate.cpp:
(JSC::Wasm::Validate::isControlTypeIf):
* wasm/js/JSToWasm.cpp:
(JSC::Wasm::createJSToWasmWrapper):
* wasm/js/JSToWasm.h:
* wasm/js/WebAssemblyFunction.cpp:
(JSC::WebAssemblyFunction::calleeSaves const):

Tools:

Add a mode that runs WebAssembly tests without the LLInt (i.e. only Air)
and update the no-air mode to also disable the LLInt tier.

* Scripts/run-jsc-stress-tests:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@251886 268f45cc-cd09-0410-ab3c-d52691b4dbfc

104 files changed:
Source/JavaScriptCore/CMakeLists.txt
Source/JavaScriptCore/ChangeLog
Source/JavaScriptCore/DerivedSources-input.xcfilelist
Source/JavaScriptCore/DerivedSources-output.xcfilelist
Source/JavaScriptCore/DerivedSources.make
Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj
Source/JavaScriptCore/Sources.txt
Source/JavaScriptCore/bytecode/BytecodeDumper.cpp
Source/JavaScriptCore/bytecode/BytecodeDumper.h
Source/JavaScriptCore/bytecode/BytecodeGeneratorification.cpp
Source/JavaScriptCore/bytecode/BytecodeList.rb
Source/JavaScriptCore/bytecode/CodeBlock.cpp
Source/JavaScriptCore/bytecode/Fits.h
Source/JavaScriptCore/bytecode/Instruction.h
Source/JavaScriptCore/bytecode/InstructionStream.h
Source/JavaScriptCore/bytecode/Opcode.cpp
Source/JavaScriptCore/bytecode/Opcode.h
Source/JavaScriptCore/bytecode/PreciseJumpTargetsInlines.h
Source/JavaScriptCore/bytecode/UnlinkedCodeBlock.h
Source/JavaScriptCore/bytecode/VirtualRegister.cpp
Source/JavaScriptCore/bytecode/VirtualRegister.h
Source/JavaScriptCore/bytecompiler/BytecodeGenerator.cpp
Source/JavaScriptCore/bytecompiler/BytecodeGenerator.h
Source/JavaScriptCore/bytecompiler/BytecodeGeneratorBase.h [new file with mode: 0644]
Source/JavaScriptCore/bytecompiler/BytecodeGeneratorBaseInlines.h [new file with mode: 0644]
Source/JavaScriptCore/bytecompiler/Label.h
Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp
Source/JavaScriptCore/dfg/DFGCapabilities.cpp
Source/JavaScriptCore/dfg/DFGOperations.cpp
Source/JavaScriptCore/generator/Argument.rb
Source/JavaScriptCore/generator/DSL.rb
Source/JavaScriptCore/generator/GeneratedFile.rb
Source/JavaScriptCore/generator/Opcode.rb
Source/JavaScriptCore/generator/Options.rb
Source/JavaScriptCore/generator/Section.rb
Source/JavaScriptCore/generator/Wasm.rb [new file with mode: 0644]
Source/JavaScriptCore/interpreter/Register.h
Source/JavaScriptCore/interpreter/RegisterInlines.h
Source/JavaScriptCore/jit/JITArithmetic.cpp
Source/JavaScriptCore/jit/JITOpcodes.cpp
Source/JavaScriptCore/llint/LLIntData.cpp
Source/JavaScriptCore/llint/LLIntData.h
Source/JavaScriptCore/llint/LLIntOfflineAsmConfig.h
Source/JavaScriptCore/llint/LLIntOffsetsExtractor.cpp
Source/JavaScriptCore/llint/LLIntSlowPaths.cpp
Source/JavaScriptCore/llint/LLIntThunks.cpp
Source/JavaScriptCore/llint/LLIntThunks.h
Source/JavaScriptCore/llint/LowLevelInterpreter.asm
Source/JavaScriptCore/llint/LowLevelInterpreter32_64.asm
Source/JavaScriptCore/llint/LowLevelInterpreter64.asm
Source/JavaScriptCore/llint/WebAssembly.asm [new file with mode: 0644]
Source/JavaScriptCore/offlineasm/arm64.rb
Source/JavaScriptCore/offlineasm/instructions.rb
Source/JavaScriptCore/offlineasm/parser.rb
Source/JavaScriptCore/offlineasm/registers.rb
Source/JavaScriptCore/offlineasm/transform.rb
Source/JavaScriptCore/offlineasm/x86.rb
Source/JavaScriptCore/parser/Nodes.h
Source/JavaScriptCore/runtime/Error.cpp
Source/JavaScriptCore/runtime/ErrorInstance.cpp
Source/JavaScriptCore/runtime/Options.cpp
Source/JavaScriptCore/runtime/OptionsList.h
Source/JavaScriptCore/runtime/SamplingProfiler.cpp
Source/JavaScriptCore/wasm/WasmAirIRGenerator.cpp
Source/JavaScriptCore/wasm/WasmB3IRGenerator.cpp
Source/JavaScriptCore/wasm/WasmBBQPlan.cpp
Source/JavaScriptCore/wasm/WasmBBQPlan.h
Source/JavaScriptCore/wasm/WasmBBQPlanInlines.h [deleted file]
Source/JavaScriptCore/wasm/WasmCallee.cpp
Source/JavaScriptCore/wasm/WasmCallee.h
Source/JavaScriptCore/wasm/WasmCallingConvention.h
Source/JavaScriptCore/wasm/WasmCodeBlock.cpp
Source/JavaScriptCore/wasm/WasmCodeBlock.h
Source/JavaScriptCore/wasm/WasmCompilationMode.cpp
Source/JavaScriptCore/wasm/WasmCompilationMode.h
Source/JavaScriptCore/wasm/WasmEmbedder.h
Source/JavaScriptCore/wasm/WasmEntryPlan.cpp [new file with mode: 0644]
Source/JavaScriptCore/wasm/WasmEntryPlan.h [new file with mode: 0644]
Source/JavaScriptCore/wasm/WasmFunctionCodeBlock.cpp [new file with mode: 0644]
Source/JavaScriptCore/wasm/WasmFunctionCodeBlock.h [new file with mode: 0644]
Source/JavaScriptCore/wasm/WasmFunctionParser.h
Source/JavaScriptCore/wasm/WasmInstance.h
Source/JavaScriptCore/wasm/WasmLLIntGenerator.cpp [new file with mode: 0644]
Source/JavaScriptCore/wasm/WasmLLIntGenerator.h [new file with mode: 0644]
Source/JavaScriptCore/wasm/WasmLLIntPlan.cpp [new file with mode: 0644]
Source/JavaScriptCore/wasm/WasmLLIntPlan.h [new file with mode: 0644]
Source/JavaScriptCore/wasm/WasmLLIntTierUpCounter.cpp [new file with mode: 0644]
Source/JavaScriptCore/wasm/WasmLLIntTierUpCounter.h [new file with mode: 0644]
Source/JavaScriptCore/wasm/WasmMemoryInformation.cpp
Source/JavaScriptCore/wasm/WasmModule.cpp
Source/JavaScriptCore/wasm/WasmOMGForOSREntryPlan.cpp
Source/JavaScriptCore/wasm/WasmOMGForOSREntryPlan.h
Source/JavaScriptCore/wasm/WasmOMGPlan.cpp
Source/JavaScriptCore/wasm/WasmSlowPaths.cpp [new file with mode: 0644]
Source/JavaScriptCore/wasm/WasmSlowPaths.h [new file with mode: 0644]
Source/JavaScriptCore/wasm/WasmTable.cpp
Source/JavaScriptCore/wasm/WasmTable.h
Source/JavaScriptCore/wasm/WasmTierUpCount.h
Source/JavaScriptCore/wasm/WasmValidate.cpp
Source/JavaScriptCore/wasm/js/JSToWasm.cpp
Source/JavaScriptCore/wasm/js/JSToWasm.h
Source/JavaScriptCore/wasm/js/WebAssemblyFunction.cpp
Tools/ChangeLog
Tools/Scripts/run-jsc-stress-tests

index 2206ff7..94b3dcb 100644 (file)
@@ -213,10 +213,10 @@ set(GENERATOR
 )
 
 add_custom_command(
-    OUTPUT ${JavaScriptCore_DERIVED_SOURCES_DIR}/Bytecodes.h ${JavaScriptCore_DERIVED_SOURCES_DIR}/InitBytecodes.asm ${JavaScriptCore_DERIVED_SOURCES_DIR}/BytecodeStructs.h ${JavaScriptCore_DERIVED_SOURCES_DIR}/BytecodeIndices.h
+    OUTPUT ${JavaScriptCore_DERIVED_SOURCES_DIR}/Bytecodes.h ${JavaScriptCore_DERIVED_SOURCES_DIR}/InitBytecodes.asm ${JavaScriptCore_DERIVED_SOURCES_DIR}/BytecodeStructs.h ${JavaScriptCore_DERIVED_SOURCES_DIR}/BytecodeIndices.h ${JavaScriptCore_DERIVED_SOURCES_DIR}/WasmLLIntGeneratorInlines.h ${JavaScriptCore_DERIVED_SOURCES_DIR}/InitWasm.asm
     MAIN_DEPENDENCY ${JAVASCRIPTCORE_DIR}/generator/main.rb
     DEPENDS ${GENERATOR} bytecode/BytecodeList.rb
-    COMMAND ${RUBY_EXECUTABLE} ${JAVASCRIPTCORE_DIR}/generator/main.rb --bytecodes_h ${JavaScriptCore_DERIVED_SOURCES_DIR}/Bytecodes.h --init_bytecodes_asm ${JavaScriptCore_DERIVED_SOURCES_DIR}/InitBytecodes.asm --bytecode_structs_h ${JavaScriptCore_DERIVED_SOURCES_DIR}/BytecodeStructs.h --bytecode_indices_h ${JavaScriptCore_DERIVED_SOURCES_DIR}/BytecodeIndices.h ${JAVASCRIPTCORE_DIR}/bytecode/BytecodeList.rb
+    COMMAND ${RUBY_EXECUTABLE} ${JAVASCRIPTCORE_DIR}/generator/main.rb --bytecodes_h ${JavaScriptCore_DERIVED_SOURCES_DIR}/Bytecodes.h --init_bytecodes_asm ${JavaScriptCore_DERIVED_SOURCES_DIR}/InitBytecodes.asm --bytecode_structs_h ${JavaScriptCore_DERIVED_SOURCES_DIR}/BytecodeStructs.h --bytecode_indices_h ${JavaScriptCore_DERIVED_SOURCES_DIR}/BytecodeIndices.h ${JAVASCRIPTCORE_DIR}/bytecode/BytecodeList.rb --wasm_json ${JAVASCRIPTCORE_DIR}/wasm/wasm.json --wasm_llint_generator_h ${JavaScriptCore_DERIVED_SOURCES_DIR}/WasmLLIntGeneratorInlines.h --init_wasm_llint ${JavaScriptCore_DERIVED_SOURCES_DIR}/InitWasm.asm
     VERBATIM)
 
 
@@ -257,16 +257,29 @@ else ()
 endif ()
 
 add_custom_command(
+    OUTPUT ${JavaScriptCore_DERIVED_SOURCES_DIR}/AirOpcode.h ${JavaScriptCore_DERIVED_SOURCES_DIR}/AirOpcodeGenerated.h
+    MAIN_DEPENDENCY ${JAVASCRIPTCORE_DIR}/b3/air/AirOpcode.opcodes
+    DEPENDS ${JAVASCRIPTCORE_DIR}/b3/air/opcode_generator.rb
+    COMMAND ${RUBY_EXECUTABLE} ${JAVASCRIPTCORE_DIR}/b3/air/opcode_generator.rb ${JAVASCRIPTCORE_DIR}/b3/air/AirOpcode.opcodes VERBATIM
+    WORKING_DIRECTORY ${JavaScriptCore_DERIVED_SOURCES_DIR}
+)
+
+list(APPEND JavaScriptCore_HEADERS
+    ${JavaScriptCore_DERIVED_SOURCES_DIR}/AirOpcode.h
+    ${JavaScriptCore_DERIVED_SOURCES_DIR}/AirOpcodeGenerated.h
+)
+
+add_custom_command(
     OUTPUT ${JavaScriptCore_DERIVED_SOURCES_DIR}/LLIntDesiredSettings.h
     MAIN_DEPENDENCY ${JAVASCRIPTCORE_DIR}/offlineasm/generate_settings_extractor.rb
-    DEPENDS ${LLINT_ASM} ${OFFLINE_ASM} ${JavaScriptCore_DERIVED_SOURCES_DIR}/InitBytecodes.asm
+    DEPENDS ${LLINT_ASM} ${OFFLINE_ASM} ${JavaScriptCore_DERIVED_SOURCES_DIR}/InitBytecodes.asm ${JavaScriptCore_DERIVED_SOURCES_DIR}/InitWasm.asm
     COMMAND ${RUBY_EXECUTABLE} ${JAVASCRIPTCORE_DIR}/offlineasm/generate_settings_extractor.rb -I${JavaScriptCore_DERIVED_SOURCES_DIR}/ ${JAVASCRIPTCORE_DIR}/llint/LowLevelInterpreter.asm ${JavaScriptCore_DERIVED_SOURCES_DIR}/LLIntDesiredSettings.h ${OFFLINE_ASM_BACKEND}
     VERBATIM)
 
 add_custom_command(
     OUTPUT ${JavaScriptCore_DERIVED_SOURCES_DIR}/LLIntDesiredOffsets.h
     MAIN_DEPENDENCY ${JAVASCRIPTCORE_DIR}/offlineasm/generate_offset_extractor.rb
-    DEPENDS LLIntSettingsExtractor ${LLINT_ASM} ${OFFLINE_ASM} ${JavaScriptCore_DERIVED_SOURCES_DIR}/InitBytecodes.asm
+    DEPENDS LLIntSettingsExtractor ${LLINT_ASM} ${OFFLINE_ASM} ${JavaScriptCore_DERIVED_SOURCES_DIR}/InitBytecodes.asm ${JavaScriptCore_DERIVED_SOURCES_DIR}/InitWasm.asm ${JavaScriptCore_DERIVED_SOURCES_DIR}/AirOpcode.h ${JavaScriptCore_DERIVED_SOURCES_DIR}/WasmOps.h
     COMMAND ${RUBY_EXECUTABLE} ${JAVASCRIPTCORE_DIR}/offlineasm/generate_offset_extractor.rb -I${JavaScriptCore_DERIVED_SOURCES_DIR}/ ${JAVASCRIPTCORE_DIR}/llint/LowLevelInterpreter.asm $<TARGET_FILE:LLIntSettingsExtractor> ${JavaScriptCore_DERIVED_SOURCES_DIR}/LLIntDesiredOffsets.h ${OFFLINE_ASM_BACKEND}
     VERBATIM)
 
@@ -310,7 +323,7 @@ endif ()
 add_custom_command(
     OUTPUT ${JavaScriptCore_DERIVED_SOURCES_DIR}/${LLIntOutput}
     MAIN_DEPENDENCY ${JAVASCRIPTCORE_DIR}/offlineasm/asm.rb
-    DEPENDS LLIntOffsetsExtractor ${LLINT_ASM} ${OFFLINE_ASM} ${JavaScriptCore_DERIVED_SOURCES_DIR}/InitBytecodes.asm
+    DEPENDS LLIntOffsetsExtractor ${LLINT_ASM} ${OFFLINE_ASM} ${JavaScriptCore_DERIVED_SOURCES_DIR}/InitBytecodes.asm ${JavaScriptCore_DERIVED_SOURCES_DIR}/InitWasm.asm
     COMMAND ${RUBY_EXECUTABLE} ${JAVASCRIPTCORE_DIR}/offlineasm/asm.rb -I${JavaScriptCore_DERIVED_SOURCES_DIR}/ ${JAVASCRIPTCORE_DIR}/llint/LowLevelInterpreter.asm $<TARGET_FILE:LLIntOffsetsExtractor> ${JavaScriptCore_DERIVED_SOURCES_DIR}/${LLIntOutput} ${OFFLINE_ASM_ARGS}
     COMMAND ${CMAKE_COMMAND} -E touch_nocreate ${JavaScriptCore_DERIVED_SOURCES_DIR}/${LLIntOutput}
     WORKING_DIRECTORY ${JavaScriptCore_DERIVED_SOURCES_DIR}
@@ -393,6 +406,7 @@ set(JavaScriptCore_PUBLIC_FRAMEWORK_HEADERS
 set(JavaScriptCore_PRIVATE_FRAMEWORK_HEADERS
     ${JavaScriptCore_DERIVED_SOURCES_DIR}/Bytecodes.h
     ${JavaScriptCore_DERIVED_SOURCES_DIR}/JSCBuiltins.h
+    ${JavaScriptCore_DERIVED_SOURCES_DIR}/WasmOps.h
 
     ${JavaScriptCore_DERIVED_SOURCES_DIR}/inspector/InspectorBackendDispatchers.h
     ${JavaScriptCore_DERIVED_SOURCES_DIR}/inspector/InspectorFrontendDispatchers.h
@@ -463,6 +477,10 @@ set(JavaScriptCore_PRIVATE_FRAMEWORK_HEADERS
     assembler/X86Registers.h
     assembler/X86_64Registers.h
 
+    b3/B3Common.h
+    b3/B3Compilation.h
+    b3/B3Type.h
+
     bindings/ScriptFunctionCall.h
     bindings/ScriptObject.h
     bindings/ScriptValue.h
@@ -1012,6 +1030,7 @@ set(JavaScriptCore_PRIVATE_FRAMEWORK_HEADERS
     tools/VMInspector.h
     tools/VMInspectorInlines.h
 
+    wasm/WasmCallee.h
     wasm/WasmCapabilities.h
     wasm/WasmCodeBlock.h
     wasm/WasmCompilationMode.h
@@ -1019,13 +1038,18 @@ set(JavaScriptCore_PRIVATE_FRAMEWORK_HEADERS
     wasm/WasmEmbedder.h
     wasm/WasmExceptionType.h
     wasm/WasmFaultSignalHandler.h
+    wasm/WasmFormat.h
+    wasm/WasmFunctionCodeBlock.h
     wasm/WasmIndexOrName.h
+    wasm/WasmLLIntTierUpCounter.h
     wasm/WasmMemory.h
+    wasm/WasmMemoryInformation.h
     wasm/WasmMemoryMode.h
     wasm/WasmModule.h
     wasm/WasmName.h
     wasm/WasmNameSection.h
     wasm/WasmPageCount.h
+    wasm/WasmSignature.h
     wasm/WasmTierUpCount.h
 
     wasm/js/JSWebAssembly.h
@@ -1255,19 +1279,6 @@ list(APPEND JavaScriptCore_HEADERS
 )
 
 add_custom_command(
-    OUTPUT ${JavaScriptCore_DERIVED_SOURCES_DIR}/AirOpcode.h ${JavaScriptCore_DERIVED_SOURCES_DIR}/AirOpcodeGenerated.h
-    MAIN_DEPENDENCY ${JAVASCRIPTCORE_DIR}/b3/air/AirOpcode.opcodes
-    DEPENDS ${JAVASCRIPTCORE_DIR}/b3/air/opcode_generator.rb
-    COMMAND ${RUBY_EXECUTABLE} ${JAVASCRIPTCORE_DIR}/b3/air/opcode_generator.rb ${JAVASCRIPTCORE_DIR}/b3/air/AirOpcode.opcodes VERBATIM
-    WORKING_DIRECTORY ${JavaScriptCore_DERIVED_SOURCES_DIR}
-)
-
-list(APPEND JavaScriptCore_HEADERS
-    ${JavaScriptCore_DERIVED_SOURCES_DIR}/AirOpcode.h
-    ${JavaScriptCore_DERIVED_SOURCES_DIR}/AirOpcodeGenerated.h
-)
-
-add_custom_command(
     OUTPUT ${JavaScriptCore_DERIVED_SOURCES_DIR}/InjectedScriptSource.h ${JavaScriptCore_DERIVED_SOURCES_DIR}/InjectedScriptSource.min.js
     MAIN_DEPENDENCY ${JAVASCRIPTCORE_DIR}/inspector/InjectedScriptSource.js
     DEPENDS ${JavaScriptCore_SCRIPTS_DIR}/xxd.pl ${JavaScriptCore_SCRIPTS_DIR}/jsmin.py
index e4017e5..2b691e4 100644 (file)
@@ -1,3 +1,350 @@
+2019-10-31  Tadeu Zagallo  <tzagallo@apple.com>
+
+        [WebAssembly] Create a Wasm interpreter
+        https://bugs.webkit.org/show_bug.cgi?id=194257
+        <rdar://problem/44186794>
+
+        Reviewed by Saam Barati.
+
+        Add an interpreter tier to WebAssembly which reuses the LLInt infrastructure. The interpreter
+        currently tiers up straight to OMG and can OSR enter at the prologue and from loops. The initial
+        implementation of the interpreter is very naive, but despite the lack of optimizations it still
+        shows a 2x improvement on the WebAssembly subtests in JetStream2 and 2x improvement on the
+        PSPDFKit benchmark. It reduces "compilation" times by ~3x and it's neutral on throughput.
+
+        The interpreter follows the same calling conventions as the BBQ/OMG, this means that:
+        - We have to allocate locals for all argument registers and write all arguments registers to the
+          stack in the prologue.
+        - Calls have to allocate space for at least as many arguments as the number of argument registers.
+          Before each call, all argument registers must be loaded from the stack, and after we return from
+          the call, all registers must be stored back to the stack, in case they contain return values. We
+          carefully layout the stack so that the arguments that would already have to be passed in the stack
+          end up in the right place. The stack layout for calls is:
+            [ gprs ][ fprs ][ optional stack arguments ][ callee frame ]
+                                                                       ^ sp
+        - The return opcode has to load all registers from the stack, since they might need to contain
+          results of the function.
+        - The calling convention requires that the callee should store itself in the callee slot of the call
+          frame, which is impossible in the interpreter, since the code we execute is the same for all callees.
+          In order to work around that, we generate an entry thunk to the wasm interpreter for each function.
+          All this thunk does is store the callee in the call frame and tail call the interpreter.
+
+        * CMakeLists.txt:
+        * DerivedSources-input.xcfilelist:
+        * DerivedSources-output.xcfilelist:
+        * DerivedSources.make:
+        * JavaScriptCore.xcodeproj/project.pbxproj:
+        * Sources.txt:
+        * bytecode/BytecodeDumper.cpp:
+        (JSC::BytecodeDumper<Block>::constantName const):
+        (JSC::BytecodeDumper<Block>::dumpValue):
+        (JSC::BytecodeDumper<Block>::dumpBytecode):
+        (JSC::CodeBlockBytecodeDumper<Block>::vm const):
+        (JSC::CodeBlockBytecodeDumper<Block>::identifier const):
+        (JSC::CodeBlockBytecodeDumper<Block>::dumpIdentifiers):
+        (JSC::CodeBlockBytecodeDumper<Block>::dumpConstants):
+        (JSC::CodeBlockBytecodeDumper<Block>::dumpExceptionHandlers):
+        (JSC::CodeBlockBytecodeDumper<Block>::dumpSwitchJumpTables):
+        (JSC::CodeBlockBytecodeDumper<Block>::dumpStringSwitchJumpTables):
+        (JSC::CodeBlockBytecodeDumper<Block>::dumpBlock):
+        * bytecode/BytecodeDumper.h:
+        (JSC::BytecodeDumper::dumpValue):
+        (JSC::BytecodeDumper::BytecodeDumper):
+        * bytecode/BytecodeGeneratorification.cpp:
+        (JSC::performGeneratorification):
+        * bytecode/BytecodeList.rb:
+        * bytecode/CodeBlock.cpp:
+        (JSC::CodeBlock::dumpBytecode):
+        * bytecode/Fits.h:
+        * bytecode/Instruction.h:
+        (JSC::BaseInstruction::BaseInstruction):
+        (JSC::BaseInstruction::Impl::opcodeID const):
+        (JSC::BaseInstruction::opcodeID const):
+        (JSC::BaseInstruction::name const):
+        (JSC::BaseInstruction::isWide16 const):
+        (JSC::BaseInstruction::isWide32 const):
+        (JSC::BaseInstruction::hasMetadata const):
+        (JSC::BaseInstruction::sizeShiftAmount const):
+        (JSC::BaseInstruction::size const):
+        (JSC::BaseInstruction::is const):
+        (JSC::BaseInstruction::as const):
+        (JSC::BaseInstruction::cast):
+        (JSC::BaseInstruction::cast const):
+        (JSC::BaseInstruction::wide16 const):
+        (JSC::BaseInstruction::wide32 const):
+        * bytecode/InstructionStream.h:
+        (JSC::InstructionStream::iterator::operator+=):
+        (JSC::InstructionStream::iterator::operator++):
+        (JSC::InstructionStreamWriter::iterator::operator+=):
+        (JSC::InstructionStreamWriter::iterator::operator++):
+        * bytecode/Opcode.cpp:
+        * bytecode/Opcode.h:
+        * bytecode/PreciseJumpTargetsInlines.h:
+        * bytecode/UnlinkedCodeBlock.h:
+        * bytecode/VirtualRegister.cpp:
+        (JSC::VirtualRegister::VirtualRegister):
+        * bytecode/VirtualRegister.h:
+        * bytecompiler/BytecodeGenerator.cpp:
+        (JSC::GenericLabel<JSGeneratorTraits>::setLocation):
+        (JSC::BytecodeGenerator::BytecodeGenerator):
+        * bytecompiler/BytecodeGenerator.h:
+        * bytecompiler/BytecodeGeneratorBase.h: Added.
+        * bytecompiler/BytecodeGeneratorBaseInlines.h: Added.
+        (JSC::shrinkToFit):
+        (JSC::BytecodeGeneratorBase<Traits>::BytecodeGeneratorBase):
+        (JSC::BytecodeGeneratorBase<Traits>::newLabel):
+        (JSC::BytecodeGeneratorBase<Traits>::newEmittedLabel):
+        (JSC::BytecodeGeneratorBase<Traits>::reclaimFreeRegisters):
+        (JSC::BytecodeGeneratorBase<Traits>::emitLabel):
+        (JSC::BytecodeGeneratorBase<Traits>::recordOpcode):
+        (JSC::BytecodeGeneratorBase<Traits>::alignWideOpcode16):
+        (JSC::BytecodeGeneratorBase<Traits>::alignWideOpcode32):
+        (JSC::BytecodeGeneratorBase<Traits>::write):
+        (JSC::BytecodeGeneratorBase<Traits>::newRegister):
+        (JSC::BytecodeGeneratorBase<Traits>::newTemporary):
+        (JSC::BytecodeGeneratorBase<Traits>::addVar):
+        (JSC::BytecodeGeneratorBase<Traits>::allocateCalleeSaveSpace):
+        * bytecompiler/Label.h:
+        (JSC::GenericBoundLabel::GenericBoundLabel):
+        (JSC::GenericBoundLabel::target):
+        (JSC::GenericBoundLabel::saveTarget):
+        (JSC::GenericBoundLabel::commitTarget):
+        * dfg/DFGByteCodeParser.cpp:
+        * dfg/DFGCapabilities.cpp:
+        (JSC::DFG::capabilityLevel):
+        * dfg/DFGOperations.cpp:
+        * generator/Argument.rb:
+        * generator/DSL.rb:
+        * generator/GeneratedFile.rb:
+        * generator/Opcode.rb:
+        * generator/Options.rb:
+        * generator/Section.rb:
+        * generator/Wasm.rb: Added.
+        * interpreter/Register.h:
+        * interpreter/RegisterInlines.h:
+        (JSC::Register::operator=):
+        * jit/JITArithmetic.cpp:
+        * jit/JITOpcodes.cpp:
+        * llint/LLIntData.cpp:
+        (JSC::LLInt::initialize):
+        * llint/LLIntData.h:
+        (JSC::LLInt::wasmExceptionInstructions):
+        * llint/LLIntOfflineAsmConfig.h:
+        * llint/LLIntOffsetsExtractor.cpp:
+        * llint/LLIntSlowPaths.cpp:
+        * llint/LLIntThunks.cpp:
+        (JSC::LLInt::generateThunkWithJumpTo):
+        (JSC::LLInt::wasmFunctionEntryThunk):
+        * llint/LLIntThunks.h:
+        * llint/LowLevelInterpreter.asm:
+        * llint/LowLevelInterpreter32_64.asm:
+        * llint/LowLevelInterpreter64.asm:
+        * llint/WebAssembly.asm: Added.
+        * offlineasm/arm64.rb:
+        * offlineasm/instructions.rb:
+        * offlineasm/parser.rb:
+        * offlineasm/registers.rb:
+        * offlineasm/transform.rb:
+        * offlineasm/x86.rb:
+        * parser/Nodes.h:
+        * runtime/Error.cpp:
+        (JSC::FindFirstCallerFrameWithCodeblockFunctor::operator() const):
+        * runtime/ErrorInstance.cpp:
+        (JSC::ErrorInstance::finishCreation):
+        * runtime/Options.cpp:
+        (JSC::overrideDefaults):
+        * runtime/OptionsList.h:
+        * runtime/SamplingProfiler.cpp:
+        (JSC::FrameWalker::recordJITFrame):
+        (JSC::FrameWalker::resetAtMachineFrame):
+        * wasm/WasmAirIRGenerator.cpp:
+        (JSC::Wasm::AirIRGenerator::isControlTypeIf):
+        (JSC::Wasm::AirIRGenerator::emitLoopTierUpCheck):
+        * wasm/WasmB3IRGenerator.cpp:
+        (JSC::Wasm::B3IRGenerator::isControlTypeIf):
+        * wasm/WasmBBQPlan.cpp:
+        (JSC::Wasm::BBQPlan::prepareImpl):
+        (JSC::Wasm::BBQPlan::work):
+        (JSC::Wasm::BBQPlan::compileFunction):
+        (JSC::Wasm::BBQPlan::didCompleteCompilation):
+        (JSC::Wasm::BBQPlan::initializeCallees):
+        * wasm/WasmBBQPlan.h:
+        * wasm/WasmBBQPlanInlines.h: Removed.
+        * wasm/WasmCallee.cpp:
+        (JSC::Wasm::Callee::Callee):
+        (JSC::Wasm::Callee::dump const):
+        (JSC::Wasm::JITCallee::JITCallee):
+        (JSC::Wasm::LLIntCallee::setEntrypoint):
+        (JSC::Wasm::LLIntCallee::entrypoint const):
+        (JSC::Wasm::LLIntCallee::calleeSaveRegisters):
+        (JSC::Wasm:: const):
+        * wasm/WasmCallee.h:
+        (JSC::Wasm::Callee::setOSREntryCallee):
+        (JSC::Wasm::JITCallee::wasmToWasmCallsites):
+        (JSC::Wasm::JITCallee:: const):
+        * wasm/WasmCallingConvention.h:
+        * wasm/WasmCodeBlock.cpp:
+        (JSC::Wasm::CodeBlock::CodeBlock):
+        * wasm/WasmCodeBlock.h:
+        (JSC::Wasm::CodeBlock::wasmEntrypointCalleeFromFunctionIndexSpace):
+        (JSC::Wasm::CodeBlock::wasmBBQCalleeFromFunctionIndexSpace):
+        (JSC::Wasm::CodeBlock::wasmToWasmExitStub):
+        * wasm/WasmCompilationMode.cpp:
+        (JSC::Wasm::makeString):
+        * wasm/WasmCompilationMode.h:
+        * wasm/WasmEmbedder.h:
+        * wasm/WasmEntryPlan.cpp: Added.
+        (JSC::Wasm::EntryPlan::EntryPlan):
+        (JSC::Wasm::EntryPlan::stateString):
+        (JSC::Wasm::EntryPlan::moveToState):
+        (JSC::Wasm::EntryPlan::didReceiveFunctionData):
+        (JSC::Wasm::EntryPlan::parseAndValidateModule):
+        (JSC::Wasm::EntryPlan::prepare):
+        (JSC::Wasm::EntryPlan::ThreadCountHolder::ThreadCountHolder):
+        (JSC::Wasm::EntryPlan::ThreadCountHolder::~ThreadCountHolder):
+        (JSC::Wasm::EntryPlan::complete):
+        (JSC::Wasm::EntryPlan::compileFunctions):
+        (JSC::Wasm::EntryPlan::work):
+        * wasm/WasmEntryPlan.h: Copied from Source/JavaScriptCore/wasm/WasmBBQPlan.h.
+        (JSC::Wasm::EntryPlan::parseAndValidateModule):
+        (JSC::Wasm::EntryPlan::exports const):
+        (JSC::Wasm::EntryPlan::internalFunctionCount const):
+        (JSC::Wasm::EntryPlan::takeModuleInformation):
+        (JSC::Wasm::EntryPlan::takeWasmToWasmExitStubs):
+        (JSC::Wasm::EntryPlan::takeWasmToWasmCallsites):
+        (JSC::Wasm::EntryPlan::hasBeenPrepared const):
+        (JSC::Wasm::EntryPlan::tryReserveCapacity):
+        * wasm/WasmFunctionCodeBlock.cpp: Added.
+        (JSC::Wasm::FunctionCodeBlock::setInstructions):
+        (JSC::Wasm::FunctionCodeBlock::dumpBytecode):
+        (JSC::Wasm::FunctionCodeBlock::addOutOfLineJumpTarget):
+        (JSC::Wasm::FunctionCodeBlock::outOfLineJumpOffset):
+        (JSC::Wasm::FunctionCodeBlock::outOfLineJumpTarget):
+        (JSC::Wasm::FunctionCodeBlock::addSignature):
+        (JSC::Wasm::FunctionCodeBlock::signature const):
+        (JSC::Wasm::FunctionCodeBlock::addJumpTable):
+        (JSC::Wasm::FunctionCodeBlock::jumpTable const const):
+        (JSC::Wasm::FunctionCodeBlock::numberOfJumpTables const):
+        * wasm/WasmFunctionCodeBlock.h: Added.
+        (JSC::Wasm::FunctionCodeBlock::FunctionCodeBlock):
+        (JSC::Wasm::FunctionCodeBlock::getConstant const):
+        (JSC::Wasm::FunctionCodeBlock::functionIndex const):
+        (JSC::Wasm::FunctionCodeBlock::addJumpTarget):
+        (JSC::Wasm::FunctionCodeBlock::numberOfJumpTargets):
+        (JSC::Wasm::FunctionCodeBlock::lastJumpTarget):
+        (JSC::Wasm::FunctionCodeBlock::outOfLineJumpOffset):
+        (JSC::Wasm::FunctionCodeBlock::bytecodeOffset):
+        (JSC::Wasm::FunctionCodeBlock::tierUpCounter):
+        * wasm/WasmFunctionParser.h:
+        (JSC::Wasm::FunctionParser<Context>::parseExpression):
+        (JSC::Wasm::FunctionParser<Context>::parseUnreachableExpression):
+        * wasm/WasmInstance.h:
+        * wasm/WasmLLIntGenerator.cpp: Added.
+        (JSC::Wasm::LLIntGenerator::ControlType::ControlType):
+        (JSC::Wasm::LLIntGenerator::ControlType::loop):
+        (JSC::Wasm::LLIntGenerator::ControlType::topLevel):
+        (JSC::Wasm::LLIntGenerator::ControlType::block):
+        (JSC::Wasm::LLIntGenerator::ControlType::if_):
+        (JSC::Wasm::LLIntGenerator::ControlType::targetLabelForBranch const):
+        (JSC::Wasm::LLIntGenerator::fail const):
+        (JSC::Wasm::LLIntGenerator::unifyValuesWithBlock):
+        (JSC::Wasm::LLIntGenerator::emptyExpression):
+        (JSC::Wasm::LLIntGenerator::createStack):
+        (JSC::Wasm::LLIntGenerator::isControlTypeIf):
+        (JSC::Wasm::LLIntGenerator::addEndToUnreachable):
+        (JSC::Wasm::LLIntGenerator::setParser):
+        (JSC::Wasm::LLIntGenerator::dump):
+        (JSC::Wasm::LLIntGenerator::virtualRegisterForLocal):
+        (JSC::Wasm::LLIntGenerator::tmpsForSignature):
+        (JSC::Wasm::LLIntGenerator::jsNullConstant):
+        (JSC::Wasm::LLIntGenerator::isConstant):
+        (JSC::Wasm::parseAndCompileBytecode):
+        (JSC::Wasm::LLIntGenerator::LLIntGenerator):
+        (JSC::Wasm::LLIntGenerator::finalize):
+        (JSC::Wasm::LLIntGenerator::callInformationFor):
+        (JSC::Wasm::LLIntGenerator::addArguments):
+        (JSC::Wasm::LLIntGenerator::addLocal):
+        (JSC::Wasm::LLIntGenerator::addConstant):
+        (JSC::Wasm::LLIntGenerator::getLocal):
+        (JSC::Wasm::LLIntGenerator::setLocal):
+        (JSC::Wasm::LLIntGenerator::getGlobal):
+        (JSC::Wasm::LLIntGenerator::setGlobal):
+        (JSC::Wasm::LLIntGenerator::addLoop):
+        (JSC::Wasm::LLIntGenerator::addTopLevel):
+        (JSC::Wasm::LLIntGenerator::addBlock):
+        (JSC::Wasm::LLIntGenerator::addIf):
+        (JSC::Wasm::LLIntGenerator::addElse):
+        (JSC::Wasm::LLIntGenerator::addElseToUnreachable):
+        (JSC::Wasm::LLIntGenerator::addReturn):
+        (JSC::Wasm::LLIntGenerator::addBranch):
+        (JSC::Wasm::LLIntGenerator::addSwitch):
+        (JSC::Wasm::LLIntGenerator::endBlock):
+        (JSC::Wasm::LLIntGenerator::addCall):
+        (JSC::Wasm::LLIntGenerator::addCallIndirect):
+        (JSC::Wasm::LLIntGenerator::addRefIsNull):
+        (JSC::Wasm::LLIntGenerator::addRefFunc):
+        (JSC::Wasm::LLIntGenerator::addTableGet):
+        (JSC::Wasm::LLIntGenerator::addTableSet):
+        (JSC::Wasm::LLIntGenerator::addTableSize):
+        (JSC::Wasm::LLIntGenerator::addTableGrow):
+        (JSC::Wasm::LLIntGenerator::addTableFill):
+        (JSC::Wasm::LLIntGenerator::addUnreachable):
+        (JSC::Wasm::LLIntGenerator::addCurrentMemory):
+        (JSC::Wasm::LLIntGenerator::addGrowMemory):
+        (JSC::Wasm::LLIntGenerator::addSelect):
+        (JSC::Wasm::LLIntGenerator::load):
+        (JSC::Wasm::LLIntGenerator::store):
+        (JSC::GenericLabel<Wasm::GeneratorTraits>::setLocation):
+        * wasm/WasmLLIntGenerator.h: Copied from Source/JavaScriptCore/wasm/WasmCompilationMode.h.
+        * wasm/WasmLLIntPlan.cpp: Added.
+        (JSC::Wasm::LLIntPlan::prepareImpl):
+        (JSC::Wasm::LLIntPlan::compileFunction):
+        (JSC::Wasm::LLIntPlan::didCompleteCompilation):
+        (JSC::Wasm::LLIntPlan::initializeCallees):
+        * wasm/WasmLLIntPlan.h: Copied from Source/JavaScriptCore/wasm/WasmOMGForOSREntryPlan.h.
+        * wasm/WasmLLIntTierUpCounter.cpp: Copied from Source/JavaScriptCore/wasm/WasmCompilationMode.cpp.
+        (JSC::Wasm::LLIntTierUpCounter::addOSREntryDataForLoop):
+        (JSC::Wasm::LLIntTierUpCounter::osrEntryDataForLoop const const):
+        * wasm/WasmLLIntTierUpCounter.h: Copied from Source/JavaScriptCore/wasm/WasmOMGForOSREntryPlan.h.
+        (JSC::Wasm::LLIntTierUpCounter::LLIntTierUpCounter):
+        (JSC::Wasm::LLIntTierUpCounter::optimizeAfterWarmUp):
+        (JSC::Wasm::LLIntTierUpCounter::checkIfOptimizationThresholdReached):
+        (JSC::Wasm::LLIntTierUpCounter::optimizeSoon):
+        * wasm/WasmMemoryInformation.cpp:
+        (JSC::Wasm::PinnedRegisterInfo::get):
+        * wasm/WasmModule.cpp:
+        (JSC::Wasm::makeValidationResult):
+        (JSC::Wasm::makeValidationCallback):
+        (JSC::Wasm::Module::validateSync):
+        (JSC::Wasm::Module::validateAsync):
+        * wasm/WasmOMGForOSREntryPlan.cpp:
+        (JSC::Wasm::OMGForOSREntryPlan::OMGForOSREntryPlan):
+        (JSC::Wasm::OMGForOSREntryPlan::work):
+        * wasm/WasmOMGForOSREntryPlan.h:
+        * wasm/WasmOMGPlan.cpp:
+        (JSC::Wasm::OMGPlan::work):
+        * wasm/WasmSlowPaths.cpp: Added.
+        (JSC::LLInt::jitCompileAndSetHeuristics):
+        (JSC::LLInt::WASM_SLOW_PATH_DECL):
+        (JSC::LLInt::doWasmCall):
+        (JSC::LLInt::doWasmCallIndirect):
+        (JSC::LLInt::slow_path_wasm_throw_exception):
+        (JSC::LLInt::slow_path_wasm_popcount):
+        (JSC::LLInt::slow_path_wasm_popcountll):
+        * wasm/WasmSlowPaths.h: Added.
+        * wasm/WasmTable.cpp:
+        (JSC::Wasm::FuncRefTable::function const):
+        (JSC::Wasm::FuncRefTable::instance const):
+        * wasm/WasmTable.h:
+        * wasm/WasmTierUpCount.h:
+        * wasm/WasmValidate.cpp:
+        (JSC::Wasm::Validate::isControlTypeIf):
+        * wasm/js/JSToWasm.cpp:
+        (JSC::Wasm::createJSToWasmWrapper):
+        * wasm/js/JSToWasm.h:
+        * wasm/js/WebAssemblyFunction.cpp:
+        (JSC::WebAssemblyFunction::calleeSaves const):
+
 2019-10-31  Yusuke Suzuki  <ysuzuki@apple.com>
 
         [JSC] Make String#localeCompare faster by inlining JSGlobalObject::defaultCollator
index a76a5ed..7196e46 100644 (file)
@@ -74,6 +74,7 @@ $(PROJECT_DIR)/generator/Options.rb
 $(PROJECT_DIR)/generator/Section.rb
 $(PROJECT_DIR)/generator/Template.rb
 $(PROJECT_DIR)/generator/Type.rb
+$(PROJECT_DIR)/generator/Wasm.rb
 $(PROJECT_DIR)/generator/main.rb
 $(PROJECT_DIR)/inspector/InjectedScriptSource.js
 $(PROJECT_DIR)/inspector/protocol/ApplicationCache.json
index 6918b2a..14aa420 100644 (file)
@@ -17,8 +17,10 @@ $(BUILT_PRODUCTS_DIR)/DerivedSources/JavaScriptCore/DateConstructor.lut.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/JavaScriptCore/DatePrototype.lut.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/JavaScriptCore/EnabledInspectorDomains
 $(BUILT_PRODUCTS_DIR)/DerivedSources/JavaScriptCore/ErrorPrototype.lut.h
+$(BUILT_PRODUCTS_DIR)/DerivedSources/JavaScriptCore/GeneratedWasmOps.asm
 $(BUILT_PRODUCTS_DIR)/DerivedSources/JavaScriptCore/GeneratorPrototype.lut.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/JavaScriptCore/InitBytecodes.asm
+$(BUILT_PRODUCTS_DIR)/DerivedSources/JavaScriptCore/InitWasm.asm
 $(BUILT_PRODUCTS_DIR)/DerivedSources/JavaScriptCore/InjectedScriptSource.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/JavaScriptCore/InspectorInstrumentationObject.lut.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/JavaScriptCore/IntlCanonicalizeLanguage.h
@@ -57,6 +59,7 @@ $(BUILT_PRODUCTS_DIR)/DerivedSources/JavaScriptCore/SymbolConstructor.lut.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/JavaScriptCore/SymbolPrototype.lut.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/JavaScriptCore/UnicodePatternTables.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/JavaScriptCore/WasmB3IRGeneratorInlines.h
+$(BUILT_PRODUCTS_DIR)/DerivedSources/JavaScriptCore/WasmLLIntGeneratorInlines.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/JavaScriptCore/WasmOps.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/JavaScriptCore/WasmValidateInlines.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/JavaScriptCore/WebAssemblyCompileErrorConstructor.lut.h
index b59fe06..be19b63 100644 (file)
@@ -215,13 +215,22 @@ BYTECODE_FILES = \
     BytecodeIndices.h \
     BytecodeStructs.h \
     InitBytecodes.asm \
+    WasmLLIntGeneratorInlines.h \
+    InitWasm.asm \
 #
 BYTECODE_FILES_PATTERNS = $(subst .,%,$(BYTECODE_FILES))
 
 all : $(BYTECODE_FILES)
 
-$(BYTECODE_FILES_PATTERNS): $(wildcard $(JavaScriptCore)/generator/*.rb) $(JavaScriptCore)/bytecode/BytecodeList.rb
-       $(RUBY) $(JavaScriptCore)/generator/main.rb $(JavaScriptCore)/bytecode/BytecodeList.rb --bytecode_structs_h BytecodeStructs.h --init_bytecodes_asm InitBytecodes.asm --bytecodes_h Bytecodes.h --bytecode_indices_h BytecodeIndices.h
+$(BYTECODE_FILES_PATTERNS): $(wildcard $(JavaScriptCore)/generator/*.rb) $(JavaScriptCore)/bytecode/BytecodeList.rb $(JavaScriptCore)/wasm/wasm.json
+       $(RUBY) $(JavaScriptCore)/generator/main.rb $(JavaScriptCore)/bytecode/BytecodeList.rb \
+    --bytecode_structs_h BytecodeStructs.h \
+    --init_bytecodes_asm InitBytecodes.asm \
+    --bytecodes_h Bytecodes.h \
+    --bytecode_indices_h BytecodeIndices.h \
+    --wasm_json $(JavaScriptCore)/wasm/wasm.json \
+    --wasm_llint_generator_h WasmLLIntGeneratorInlines.h \
+    --init_wasm_llint InitWasm.asm \
 
 # Inspector interfaces
 
index c6ae4e6..0b315c4 100644 (file)
                0F338DF61BE93D550013C88F /* B3ConstrainedValue.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F338DF41BE93D550013C88F /* B3ConstrainedValue.h */; };
                0F338DFA1BE96AA80013C88F /* B3CCallValue.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F338DF81BE96AA80013C88F /* B3CCallValue.h */; };
                0F338DFE1BED51270013C88F /* AirSimplifyCFG.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F338DFC1BED51270013C88F /* AirSimplifyCFG.h */; };
-               0F338E0C1BF0276C0013C88F /* B3Compilation.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F338E001BF0276C0013C88F /* B3Compilation.h */; };
+               0F338E0C1BF0276C0013C88F /* B3Compilation.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F338E001BF0276C0013C88F /* B3Compilation.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0F338E0E1BF0276C0013C88F /* B3DataSection.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F338E021BF0276C0013C88F /* B3DataSection.h */; };
                0F338E101BF0276C0013C88F /* B3MoveConstants.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F338E041BF0276C0013C88F /* B3MoveConstants.h */; };
                0F338E111BF0276C0013C88F /* B3OpaqueByproduct.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F338E051BF0276C0013C88F /* B3OpaqueByproduct.h */; };
                0FEC85041BDACDAC0080FF74 /* B3BlockWorklist.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FEC84BA1BDACDAC0080FF74 /* B3BlockWorklist.h */; };
                0FEC85061BDACDAC0080FF74 /* B3CheckSpecial.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FEC84BC1BDACDAC0080FF74 /* B3CheckSpecial.h */; };
                0FEC85081BDACDAC0080FF74 /* B3CheckValue.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FEC84BE1BDACDAC0080FF74 /* B3CheckValue.h */; };
-               0FEC850A1BDACDAC0080FF74 /* B3Common.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FEC84C01BDACDAC0080FF74 /* B3Common.h */; };
+               0FEC850A1BDACDAC0080FF74 /* B3Common.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FEC84C01BDACDAC0080FF74 /* B3Common.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0FEC850C1BDACDAC0080FF74 /* B3Commutativity.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FEC84C21BDACDAC0080FF74 /* B3Commutativity.h */; };
                0FEC850E1BDACDAC0080FF74 /* B3Const32Value.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FEC84C41BDACDAC0080FF74 /* B3Const32Value.h */; };
                0FEC85101BDACDAC0080FF74 /* B3Const64Value.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FEC84C61BDACDAC0080FF74 /* B3Const64Value.h */; };
                0FEC85361BDACDAC0080FF74 /* B3SuccessorCollection.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FEC84EC1BDACDAC0080FF74 /* B3SuccessorCollection.h */; };
                0FEC85381BDACDAC0080FF74 /* B3SwitchCase.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FEC84EE1BDACDAC0080FF74 /* B3SwitchCase.h */; };
                0FEC853A1BDACDAC0080FF74 /* B3SwitchValue.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FEC84F01BDACDAC0080FF74 /* B3SwitchValue.h */; };
-               0FEC853C1BDACDAC0080FF74 /* B3Type.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FEC84F21BDACDAC0080FF74 /* B3Type.h */; };
+               0FEC853C1BDACDAC0080FF74 /* B3Type.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FEC84F21BDACDAC0080FF74 /* B3Type.h */; settings = {ATTRIBUTES = (Private, ); }; };
                0FEC853E1BDACDAC0080FF74 /* B3UpsilonValue.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FEC84F41BDACDAC0080FF74 /* B3UpsilonValue.h */; };
                0FEC85401BDACDAC0080FF74 /* B3UseCounts.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FEC84F61BDACDAC0080FF74 /* B3UseCounts.h */; };
                0FEC85421BDACDAC0080FF74 /* B3Validate.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FEC84F81BDACDAC0080FF74 /* B3Validate.h */; };
                1442566215EDE98D0066A49B /* JSWithScope.h in Headers */ = {isa = PBXBuildFile; fileRef = 1442566015EDE98D0066A49B /* JSWithScope.h */; settings = {ATTRIBUTES = (Private, ); }; };
                144836E7132DA7BE005BE785 /* ConservativeRoots.h in Headers */ = {isa = PBXBuildFile; fileRef = 149DAAF212EB559D0083B12B /* ConservativeRoots.h */; settings = {ATTRIBUTES = (Private, ); }; };
                144CA3502224180100817789 /* CachedBytecode.h in Headers */ = {isa = PBXBuildFile; fileRef = 144CA34F221F037900817789 /* CachedBytecode.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               1450FA1C2357BDBE0093CD4D /* WasmFunctionCodeBlock.h in Headers */ = {isa = PBXBuildFile; fileRef = 14C5AD6B22F33FC000F1FB83 /* WasmFunctionCodeBlock.h */; settings = {ATTRIBUTES = (Private, ); }; };
+               1450FA1F2357BEC90093CD4D /* WasmLLIntTierUpCounter.h in Headers */ = {isa = PBXBuildFile; fileRef = 1450FA1E2357BEC40093CD4D /* WasmLLIntTierUpCounter.h */; settings = {ATTRIBUTES = (Private, ); }; };
                145722861437E140005FDE26 /* StrongInlines.h in Headers */ = {isa = PBXBuildFile; fileRef = 145722851437E140005FDE26 /* StrongInlines.h */; settings = {ATTRIBUTES = (Private, ); }; };
                146C384B2177ACDF0079F6D9 /* UnlinkedMetadataTable.h in Headers */ = {isa = PBXBuildFile; fileRef = 142D52BE21762958002DB086 /* UnlinkedMetadataTable.h */; settings = {ATTRIBUTES = (Private, ); }; };
                1471483020D323D30090E630 /* JSWrapperMapTests.mm in Sources */ = {isa = PBXBuildFile; fileRef = 1471482F20D323650090E630 /* JSWrapperMapTests.mm */; };
                53F40E8D1D5901F20099A1B6 /* WasmParser.h in Headers */ = {isa = PBXBuildFile; fileRef = 53F40E8C1D5901F20099A1B6 /* WasmParser.h */; };
                53F40E931D5A4AB30099A1B6 /* WasmB3IRGenerator.h in Headers */ = {isa = PBXBuildFile; fileRef = 53F40E921D5A4AB30099A1B6 /* WasmB3IRGenerator.h */; };
                53F6BF6D1C3F060A00F41E5D /* InternalFunctionAllocationProfile.h in Headers */ = {isa = PBXBuildFile; fileRef = 53F6BF6C1C3F060A00F41E5D /* InternalFunctionAllocationProfile.h */; settings = {ATTRIBUTES = (Private, ); }; };
-               53F8D2001E8387D400D21116 /* WasmBBQPlanInlines.h in Headers */ = {isa = PBXBuildFile; fileRef = 53F8D1FF1E8387D400D21116 /* WasmBBQPlanInlines.h */; };
                53FA2AE11CF37F3F0022711D /* LLIntPrototypeLoadAdaptiveStructureWatchpoint.h in Headers */ = {isa = PBXBuildFile; fileRef = 53FA2AE01CF37F3F0022711D /* LLIntPrototypeLoadAdaptiveStructureWatchpoint.h */; settings = {ATTRIBUTES = (Private, ); }; };
                53FD04D41D7AB291003287D3 /* WasmCallingConvention.h in Headers */ = {isa = PBXBuildFile; fileRef = 53FD04D21D7AB187003287D3 /* WasmCallingConvention.h */; settings = {ATTRIBUTES = (Private, ); }; };
                53FF7F991DBFCD9000A26CCC /* WasmValidate.h in Headers */ = {isa = PBXBuildFile; fileRef = 53FF7F981DBFCD9000A26CCC /* WasmValidate.h */; };
                1442565F15EDE98D0066A49B /* JSWithScope.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = JSWithScope.cpp; sourceTree = "<group>"; };
                1442566015EDE98D0066A49B /* JSWithScope.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSWithScope.h; sourceTree = "<group>"; };
                144CA34F221F037900817789 /* CachedBytecode.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = CachedBytecode.h; sourceTree = "<group>"; };
+               1450FA1D2357BEC40093CD4D /* WasmLLIntTierUpCounter.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = WasmLLIntTierUpCounter.cpp; sourceTree = "<group>"; };
+               1450FA1E2357BEC40093CD4D /* WasmLLIntTierUpCounter.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = WasmLLIntTierUpCounter.h; sourceTree = "<group>"; };
                145348572291737F00B1C2EB /* testapi.entitlements */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.plist.entitlements; path = testapi.entitlements; sourceTree = "<group>"; };
                145722851437E140005FDE26 /* StrongInlines.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = StrongInlines.h; sourceTree = "<group>"; };
                145C507F0D9DF63B0088F6B9 /* CallData.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = CallData.h; sourceTree = "<group>"; };
                14A46809216FA534000D2B1A /* Fits.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = Fits.h; sourceTree = "<group>"; };
                14A4680A216FA535000D2B1A /* Instruction.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = Instruction.h; sourceTree = "<group>"; };
                14A4680B216FA535000D2B1A /* OpcodeSize.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = OpcodeSize.h; sourceTree = "<group>"; };
+               14AB0C91231747B6000250BC /* WasmEntryPlan.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = WasmEntryPlan.cpp; sourceTree = "<group>"; };
+               14AB0C92231747B7000250BC /* WasmSlowPaths.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = WasmSlowPaths.cpp; sourceTree = "<group>"; };
+               14AB0C93231747B7000250BC /* WasmSlowPaths.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = WasmSlowPaths.h; sourceTree = "<group>"; };
+               14AB0C94231747B7000250BC /* WasmEntryPlan.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = WasmEntryPlan.h; sourceTree = "<group>"; };
                14AB66751DECF40900A56C26 /* UnlinkedSourceCode.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = UnlinkedSourceCode.h; sourceTree = "<group>"; };
                14ABB36E099C076400E2A24F /* JSCJSValue.h */ = {isa = PBXFileReference; fileEncoding = 30; lastKnownFileType = sourcecode.c.h; path = JSCJSValue.h; sourceTree = "<group>"; };
                14ABB454099C2A0F00E2A24F /* JSType.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSType.h; sourceTree = "<group>"; };
                14BD689C215191B30050DAFF /* LLIntSettingsExtractor.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = LLIntSettingsExtractor.cpp; path = llint/LLIntSettingsExtractor.cpp; sourceTree = "<group>"; };
                14BE7D3217135CF400D1807A /* WeakInlines.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WeakInlines.h; sourceTree = "<group>"; };
                14BFCE6810CDB1FC00364CCE /* WeakGCMap.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WeakGCMap.h; sourceTree = "<group>"; };
+               14C1F9CF22EA51FD0040B145 /* Wasm.rb */ = {isa = PBXFileReference; lastKnownFileType = text.script.ruby; path = Wasm.rb; sourceTree = "<group>"; };
+               14C5AD6522F1866C00F1FB83 /* BytecodeGeneratorBase.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = BytecodeGeneratorBase.h; sourceTree = "<group>"; };
+               14C5AD6622F1866C00F1FB83 /* BytecodeGeneratorBaseInlines.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = BytecodeGeneratorBaseInlines.h; sourceTree = "<group>"; };
+               14C5AD6722F33FBF00F1FB83 /* WasmFunctionCodeBlock.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = WasmFunctionCodeBlock.cpp; sourceTree = "<group>"; };
+               14C5AD6822F33FBF00F1FB83 /* WasmLLIntGenerator.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = WasmLLIntGenerator.cpp; sourceTree = "<group>"; };
+               14C5AD6922F33FBF00F1FB83 /* WasmLLIntGenerator.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = WasmLLIntGenerator.h; sourceTree = "<group>"; };
+               14C5AD6A22F33FBF00F1FB83 /* WasmLLIntPlan.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = WasmLLIntPlan.cpp; sourceTree = "<group>"; };
+               14C5AD6B22F33FC000F1FB83 /* WasmFunctionCodeBlock.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = WasmFunctionCodeBlock.h; sourceTree = "<group>"; };
+               14C5AD6C22F33FC000F1FB83 /* WasmLLIntPlan.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = WasmLLIntPlan.h; sourceTree = "<group>"; };
                14CA958A16AB50DE00938A06 /* StaticPropertyAnalyzer.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = StaticPropertyAnalyzer.h; sourceTree = "<group>"; };
                14CA958C16AB50FA00938A06 /* ObjectAllocationProfile.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ObjectAllocationProfile.h; sourceTree = "<group>"; };
                14CC3BA12138A238002D58B6 /* InstructionStream.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = InstructionStream.cpp; sourceTree = "<group>"; };
                53F40E8E1D5902820099A1B6 /* WasmB3IRGenerator.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = WasmB3IRGenerator.cpp; sourceTree = "<group>"; };
                53F40E921D5A4AB30099A1B6 /* WasmB3IRGenerator.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WasmB3IRGenerator.h; sourceTree = "<group>"; };
                53F6BF6C1C3F060A00F41E5D /* InternalFunctionAllocationProfile.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = InternalFunctionAllocationProfile.h; sourceTree = "<group>"; };
-               53F8D1FF1E8387D400D21116 /* WasmBBQPlanInlines.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WasmBBQPlanInlines.h; sourceTree = "<group>"; };
                53FA2AE01CF37F3F0022711D /* LLIntPrototypeLoadAdaptiveStructureWatchpoint.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = LLIntPrototypeLoadAdaptiveStructureWatchpoint.h; sourceTree = "<group>"; };
                53FA2AE21CF380390022711D /* LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = LLIntPrototypeLoadAdaptiveStructureWatchpoint.cpp; sourceTree = "<group>"; };
                53FD04D11D7AB187003287D3 /* WasmCallingConvention.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = WasmCallingConvention.cpp; sourceTree = "<group>"; };
                                53F40E921D5A4AB30099A1B6 /* WasmB3IRGenerator.h */,
                                53CA73071EA533D80076049D /* WasmBBQPlan.cpp */,
                                53CA73081EA533D80076049D /* WasmBBQPlan.h */,
-                               53F8D1FF1E8387D400D21116 /* WasmBBQPlanInlines.h */,
                                AD4B1DF71DF244D70071AE32 /* WasmBinding.cpp */,
                                AD4B1DF81DF244D70071AE32 /* WasmBinding.h */,
                                525C0DD71E935847002184CD /* WasmCallee.cpp */,
                                A27958D7FA1142B0AC9E364D /* WasmContextInlines.h */,
                                E36CC9462086314F0051FFD6 /* WasmCreationMode.h */,
                                AD5C36DC1F688B5F000BCAAF /* WasmEmbedder.h */,
+                               14AB0C91231747B6000250BC /* WasmEntryPlan.cpp */,
+                               14AB0C94231747B7000250BC /* WasmEntryPlan.h */,
                                79DAE2791E03C82200B526AA /* WasmExceptionType.h */,
                                5381B9361E60E9660090F794 /* WasmFaultSignalHandler.cpp */,
                                5381B9381E60E97D0090F794 /* WasmFaultSignalHandler.h */,
                                AD2FCC321DC4045300B3E736 /* WasmFormat.cpp */,
                                7BC547D21B69599B00959B58 /* WasmFormat.h */,
+                               14C5AD6722F33FBF00F1FB83 /* WasmFunctionCodeBlock.cpp */,
+                               14C5AD6B22F33FC000F1FB83 /* WasmFunctionCodeBlock.h */,
                                53F40E8A1D5901BB0099A1B6 /* WasmFunctionParser.h */,
                                AD8FF3961EB5BD850087FF82 /* WasmIndexOrName.cpp */,
                                AD8FF3951EB5BD850087FF82 /* WasmIndexOrName.h */,
                                AD5C36DE1F699EB6000BCAAF /* WasmInstance.cpp */,
                                AD5C36DF1F699EB6000BCAAF /* WasmInstance.h */,
                                AD00659D1ECAC7FE000CA926 /* WasmLimits.h */,
+                               14C5AD6822F33FBF00F1FB83 /* WasmLLIntGenerator.cpp */,
+                               14C5AD6922F33FBF00F1FB83 /* WasmLLIntGenerator.h */,
+                               14C5AD6A22F33FBF00F1FB83 /* WasmLLIntPlan.cpp */,
+                               14C5AD6C22F33FC000F1FB83 /* WasmLLIntPlan.h */,
+                               1450FA1D2357BEC40093CD4D /* WasmLLIntTierUpCounter.cpp */,
+                               1450FA1E2357BEC40093CD4D /* WasmLLIntTierUpCounter.h */,
                                53E9E0A91EAE83DE00FEE251 /* WasmMachineThreads.cpp */,
                                53E9E0AA1EAE83DE00FEE251 /* WasmMachineThreads.h */,
                                535557151D9DFA32006D583B /* WasmMemory.cpp */,
                                AD7438BE1E04579200FD0C2A /* WasmSignature.cpp */,
                                AD7438BF1E04579200FD0C2A /* WasmSignature.h */,
                                30A5F403F11C4F599CD596D5 /* WasmSignatureInlines.h */,
+                               14AB0C92231747B7000250BC /* WasmSlowPaths.cpp */,
+                               14AB0C93231747B7000250BC /* WasmSlowPaths.h */,
                                E3A0531921342B670022EC14 /* WasmStreamingParser.cpp */,
                                E3A0531621342B660022EC14 /* WasmStreamingParser.h */,
                                AD5C36E31F69EC8B000BCAAF /* WasmTable.cpp */,
                        children = (
                                969A07200ED1CE3300F1F681 /* BytecodeGenerator.cpp */,
                                969A07210ED1CE3300F1F681 /* BytecodeGenerator.h */,
+                               14C5AD6522F1866C00F1FB83 /* BytecodeGeneratorBase.h */,
+                               14C5AD6622F1866C00F1FB83 /* BytecodeGeneratorBaseInlines.h */,
                                969A07270ED1CE6900F1F681 /* Label.h */,
                                960097A50EBABB58007A7297 /* LabelScope.h */,
                                655EB29A10CE2581001A990E /* NodesCodegen.cpp */,
                                FE35C2F421B1E6C6000F4CA8 /* Section.rb */,
                                FE35C2F021B1E6C5000F4CA8 /* Template.rb */,
                                FE35C2F921B1E6C7000F4CA8 /* Type.rb */,
+                               14C1F9CF22EA51FD0040B145 /* Wasm.rb */,
                        );
                        path = generator;
                        sourceTree = "<group>";
                                52847ADC21FFB8690061A9DB /* WasmAirIRGenerator.h in Headers */,
                                53F40E931D5A4AB30099A1B6 /* WasmB3IRGenerator.h in Headers */,
                                53CA730A1EA533D80076049D /* WasmBBQPlan.h in Headers */,
-                               53F8D2001E8387D400D21116 /* WasmBBQPlanInlines.h in Headers */,
                                AD4B1DFA1DF244E20071AE32 /* WasmBinding.h in Headers */,
                                525C0DDA1E935847002184CD /* WasmCallee.h in Headers */,
                                E3FB853A22F3667B008F90ED /* WasmCalleeRegistry.h in Headers */,
                                79DAE27A1E03C82200B526AA /* WasmExceptionType.h in Headers */,
                                5381B9391E60E97D0090F794 /* WasmFaultSignalHandler.h in Headers */,
                                7BC547D31B6959A100959B58 /* WasmFormat.h in Headers */,
+                               1450FA1C2357BDBE0093CD4D /* WasmFunctionCodeBlock.h in Headers */,
                                53F40E8B1D5901BB0099A1B6 /* WasmFunctionParser.h in Headers */,
                                AD8FF3981EB5BDB20087FF82 /* WasmIndexOrName.h in Headers */,
                                AD5C36E21F699EC0000BCAAF /* WasmInstance.h in Headers */,
                                AD00659E1ECAC812000CA926 /* WasmLimits.h in Headers */,
+                               1450FA1F2357BEC90093CD4D /* WasmLLIntTierUpCounter.h in Headers */,
                                53E9E0AC1EAE83DF00FEE251 /* WasmMachineThreads.h in Headers */,
                                535557141D9D9EA5006D583B /* WasmMemory.h in Headers */,
                                79B759751DFA4C600052174C /* WasmMemoryInformation.h in Headers */,
index d1ce45d..c908afb 100644 (file)
@@ -1003,11 +1003,16 @@ wasm/WasmCodeBlock.cpp
 wasm/WasmCompilationMode.cpp
 wasm/WasmContext.cpp
 wasm/WasmEmbedder.h
+wasm/WasmEntryPlan.cpp
 wasm/WasmFaultSignalHandler.cpp
 wasm/WasmFormat.cpp
+wasm/WasmFunctionCodeBlock.cpp
 wasm/WasmIndexOrName.cpp
 wasm/WasmInstance.cpp
 wasm/WasmInstance.h
+wasm/WasmLLIntGenerator.cpp
+wasm/WasmLLIntPlan.cpp
+wasm/WasmLLIntTierUpCounter.cpp
 wasm/WasmMachineThreads.cpp
 wasm/WasmMemory.cpp
 wasm/WasmMemoryInformation.cpp
@@ -1023,6 +1028,7 @@ wasm/WasmPageCount.cpp
 wasm/WasmPlan.cpp
 wasm/WasmSectionParser.cpp
 wasm/WasmSignature.cpp
+wasm/WasmSlowPaths.cpp
 wasm/WasmStreamingParser.cpp
 wasm/WasmTable.cpp
 wasm/WasmTable.h
index ceca1cb..af80769 100644 (file)
@@ -28,6 +28,7 @@
 #include "BytecodeDumper.h"
 
 #include "ArithProfile.h"
+#include "BytecodeGenerator.h"
 #include "BytecodeStructs.h"
 #include "CallLinkStatus.h"
 #include "CodeBlock.h"
 #include "ToThisStatus.h"
 #include "UnlinkedCodeBlock.h"
 #include "UnlinkedMetadataTableInlines.h"
+#include "WasmFunctionCodeBlock.h"
 
 namespace JSC {
 
-template<class Block>
-VM& BytecodeDumper<Block>::vm() const
-{
-    return block()->vm();
-}
-
-template<class Block>
-const Identifier& BytecodeDumper<Block>::identifier(int index) const
-{
-    return block()->identifier(index);
-}
-
 static ALWAYS_INLINE bool isConstantRegisterIndex(int index)
 {
     return index >= FirstConstantRegisterIndex;
@@ -72,7 +62,7 @@ CString BytecodeDumper<Block>::registerName(int r) const
 template<class Block>
 CString BytecodeDumper<Block>::constantName(int index) const
 {
-    JSValue value = block()->getConstant(index);
+    auto value = block()->getConstant(index);
     return toCString(value, "(", VirtualRegister(index), ")");
 }
 
@@ -84,10 +74,16 @@ void BytecodeDumper<Block>::printLocationAndOp(InstructionStream::Offset locatio
 }
 
 template<class Block>
+void BytecodeDumper<Block>::dumpValue(VirtualRegister reg)
+{
+    m_out.printf("%s", registerName(reg.offset()).data());
+}
+
+template<class Block>
 void BytecodeDumper<Block>::dumpBytecode(const InstructionStream::Ref& it, const ICStatusMap&)
 {
     ::JSC::dumpBytecode(this, it.offset(), it.ptr());
-    m_out.print("\n");
+    this->m_out.print("\n");
 }
 
 template<class Block>
@@ -98,27 +94,39 @@ void BytecodeDumper<Block>::dumpBytecode(Block* block, PrintStream& out, const I
 }
 
 template<class Block>
-void BytecodeDumper<Block>::dumpIdentifiers()
+VM& CodeBlockBytecodeDumper<Block>::vm() const
+{
+    return this->block()->vm();
+}
+
+template<class Block>
+const Identifier& CodeBlockBytecodeDumper<Block>::identifier(int index) const
+{
+    return this->block()->identifier(index);
+}
+
+template<class Block>
+void CodeBlockBytecodeDumper<Block>::dumpIdentifiers()
 {
-    if (size_t count = block()->numberOfIdentifiers()) {
-        m_out.printf("\nIdentifiers:\n");
+    if (size_t count = this->block()->numberOfIdentifiers()) {
+        this->m_out.printf("\nIdentifiers:\n");
         size_t i = 0;
         do {
-            m_out.print("  id", static_cast<unsigned>(i), " = ", identifier(i), "\n");
+            this->m_out.print("  id", static_cast<unsigned>(i), " = ", identifier(i), "\n");
             ++i;
         } while (i != count);
     }
 }
 
 template<class Block>
-void BytecodeDumper<Block>::dumpConstants()
+void CodeBlockBytecodeDumper<Block>::dumpConstants()
 {
-    if (!block()->constantRegisters().isEmpty()) {
-        m_out.printf("\nConstants:\n");
+    if (!this->block()->constantRegisters().isEmpty()) {
+        this->m_out.printf("\nConstants:\n");
         size_t i = 0;
-        for (const auto& constant : block()->constantRegisters()) {
+        for (const auto& constant : this->block()->constantRegisters()) {
             const char* sourceCodeRepresentationDescription = nullptr;
-            switch (block()->constantsSourceCodeRepresentation()[i]) {
+            switch (this->block()->constantsSourceCodeRepresentation()[i]) {
             case SourceCodeRepresentation::Double:
                 sourceCodeRepresentationDescription = ": in source as double";
                 break;
@@ -129,68 +137,68 @@ void BytecodeDumper<Block>::dumpConstants()
                 sourceCodeRepresentationDescription = "";
                 break;
             }
-            m_out.printf("   k%u = %s%s\n", static_cast<unsigned>(i), toCString(constant.get()).data(), sourceCodeRepresentationDescription);
+            this->m_out.printf("   k%u = %s%s\n", static_cast<unsigned>(i), toCString(constant.get()).data(), sourceCodeRepresentationDescription);
             ++i;
         }
     }
 }
 
 template<class Block>
-void BytecodeDumper<Block>::dumpExceptionHandlers()
+void CodeBlockBytecodeDumper<Block>::dumpExceptionHandlers()
 {
-    if (unsigned count = block()->numberOfExceptionHandlers()) {
-        m_out.printf("\nException Handlers:\n");
+    if (unsigned count = this->block()->numberOfExceptionHandlers()) {
+        this->m_out.printf("\nException Handlers:\n");
         unsigned i = 0;
         do {
-            const auto& handler = block()->exceptionHandler(i);
-            m_out.printf("\t %d: { start: [%4d] end: [%4d] target: [%4d] } %s\n", i + 1, handler.start, handler.end, handler.target, handler.typeName());
+            const auto& handler = this->block()->exceptionHandler(i);
+            this->m_out.printf("\t %d: { start: [%4d] end: [%4d] target: [%4d] } %s\n", i + 1, handler.start, handler.end, handler.target, handler.typeName());
             ++i;
         } while (i < count);
     }
 }
 
 template<class Block>
-void BytecodeDumper<Block>::dumpSwitchJumpTables()
+void CodeBlockBytecodeDumper<Block>::dumpSwitchJumpTables()
 {
-    if (unsigned count = block()->numberOfSwitchJumpTables()) {
-        m_out.printf("Switch Jump Tables:\n");
+    if (unsigned count = this->block()->numberOfSwitchJumpTables()) {
+        this->m_out.printf("Switch Jump Tables:\n");
         unsigned i = 0;
         do {
-            m_out.printf("  %1d = {\n", i);
-            const auto& switchJumpTable = block()->switchJumpTable(i);
+            this->m_out.printf("  %1d = {\n", i);
+            const auto& switchJumpTable = this->block()->switchJumpTable(i);
             int entry = 0;
             auto end = switchJumpTable.branchOffsets.end();
             for (auto iter = switchJumpTable.branchOffsets.begin(); iter != end; ++iter, ++entry) {
                 if (!*iter)
                     continue;
-                m_out.printf("\t\t%4d => %04d\n", entry + switchJumpTable.min, *iter);
+                this->m_out.printf("\t\t%4d => %04d\n", entry + switchJumpTable.min, *iter);
             }
-            m_out.printf("      }\n");
+            this->m_out.printf("      }\n");
             ++i;
         } while (i < count);
     }
 }
 
 template<class Block>
-void BytecodeDumper<Block>::dumpStringSwitchJumpTables()
+void CodeBlockBytecodeDumper<Block>::dumpStringSwitchJumpTables()
 {
-    if (unsigned count = block()->numberOfStringSwitchJumpTables()) {
-        m_out.printf("\nString Switch Jump Tables:\n");
+    if (unsigned count = this->block()->numberOfStringSwitchJumpTables()) {
+        this->m_out.printf("\nString Switch Jump Tables:\n");
         unsigned i = 0;
         do {
-            m_out.printf("  %1d = {\n", i);
-            const auto& stringSwitchJumpTable = block()->stringSwitchJumpTable(i);
+            this->m_out.printf("  %1d = {\n", i);
+            const auto& stringSwitchJumpTable = this->block()->stringSwitchJumpTable(i);
             auto end = stringSwitchJumpTable.offsetTable.end();
             for (auto iter = stringSwitchJumpTable.offsetTable.begin(); iter != end; ++iter)
-                m_out.printf("\t\t\"%s\" => %04d\n", iter->key->utf8().data(), iter->value.branchOffset);
-            m_out.printf("      }\n");
+                this->m_out.printf("\t\t\"%s\" => %04d\n", iter->key->utf8().data(), iter->value.branchOffset);
+            this->m_out.printf("      }\n");
             ++i;
         } while (i < count);
     }
 }
 
 template<class Block>
-void BytecodeDumper<Block>::dumpBlock(Block* block, const InstructionStream& instructions, PrintStream& out, const ICStatusMap& statusMap)
+void CodeBlockBytecodeDumper<Block>::dumpBlock(Block* block, const InstructionStream& instructions, PrintStream& out, const ICStatusMap& statusMap)
 {
     size_t instructionCount = 0;
     size_t wide16InstructionCount = 0;
@@ -220,7 +228,7 @@ void BytecodeDumper<Block>::dumpBlock(Block* block, const InstructionStream& ins
     out.print("; scope at ", block->scopeRegister());
     out.printf("\n");
 
-    BytecodeDumper<Block> dumper(block, out);
+    CodeBlockBytecodeDumper<Block> dumper(block, out);
     for (const auto& it : instructions)
         dumper.dumpBytecode(it, statusMap);
 
@@ -233,7 +241,11 @@ void BytecodeDumper<Block>::dumpBlock(Block* block, const InstructionStream& ins
     out.printf("\n");
 }
 
-template class BytecodeDumper<UnlinkedCodeBlock>;
 template class BytecodeDumper<CodeBlock>;
+#if ENABLE(WEBASSEMBLY)
+template class BytecodeDumper<Wasm::FunctionCodeBlock>;
+#endif
+template class CodeBlockBytecodeDumper<UnlinkedCodeBlock>;
+template class CodeBlockBytecodeDumper<CodeBlock>;
 
 }
index e662ae2..948a5a5 100644 (file)
@@ -40,7 +40,6 @@ template<class Block>
 class BytecodeDumper {
 public:
     static void dumpBytecode(Block*, PrintStream& out, const InstructionStream::Ref& it, const ICStatusMap& = ICStatusMap());
-    static void dumpBlock(Block*, const InstructionStream&, PrintStream& out, const ICStatusMap& = ICStatusMap());
 
     void printLocationAndOp(InstructionStream::Offset location, const char* op);
 
@@ -52,29 +51,50 @@ public:
         dumpValue(operand);
     }
 
-    void dumpValue(VirtualRegister reg) { m_out.printf("%s", registerName(reg.offset()).data()); }
-    void dumpValue(BoundLabel label)
+    void dumpValue(VirtualRegister);
+
+    template<typename Traits>
+    void dumpValue(GenericBoundLabel<Traits> label)
     {
         InstructionStream::Offset targetOffset = label.target() + m_currentLocation;
         m_out.print(label.target(), "(->", targetOffset, ")");
     }
+
+
     template<typename T>
     void dumpValue(T v) { m_out.print(v); }
 
-private:
     BytecodeDumper(Block* block, PrintStream& out)
-        : m_block(block)
-        , m_out(out)
+        : m_out(out)
+        , m_block(block)
     {
     }
 
+protected:
     Block* block() const { return m_block; }
 
-    ALWAYS_INLINE VM& vm() const;
+    void dumpBytecode(const InstructionStream::Ref& it, const ICStatusMap&);
 
+    PrintStream& m_out;
+
+private:
     CString registerName(int r) const;
     CString constantName(int index) const;
 
+    Block* m_block;
+    InstructionStream::Offset m_currentLocation { 0 };
+};
+
+template<class Block>
+class CodeBlockBytecodeDumper : public BytecodeDumper<Block> {
+public:
+    static void dumpBlock(Block*, const InstructionStream&, PrintStream& out, const ICStatusMap& = ICStatusMap());
+
+private:
+    using BytecodeDumper<Block>::BytecodeDumper;
+
+    ALWAYS_INLINE VM& vm() const;
+
     const Identifier& identifier(int index) const;
 
     void dumpIdentifiers();
@@ -82,12 +102,6 @@ private:
     void dumpExceptionHandlers();
     void dumpSwitchJumpTables();
     void dumpStringSwitchJumpTables();
-
-    void dumpBytecode(const InstructionStream::Ref& it, const ICStatusMap&);
-
-    Block* m_block;
-    PrintStream& m_out;
-    InstructionStream::Offset m_currentLocation { 0 };
 };
 
 }
index 43ce561..33d2bee 100644 (file)
@@ -28,6 +28,7 @@
 #include "BytecodeGeneratorification.h"
 
 #include "BytecodeDumper.h"
+#include "BytecodeGeneratorBaseInlines.h"
 #include "BytecodeLivenessAnalysisInlines.h"
 #include "BytecodeRewriter.h"
 #include "BytecodeStructs.h"
@@ -300,7 +301,7 @@ void BytecodeGeneratorification::run()
 void performGeneratorification(BytecodeGenerator& bytecodeGenerator, UnlinkedCodeBlock* codeBlock, InstructionStreamWriter& instructions, SymbolTable* generatorFrameSymbolTable, int generatorFrameSymbolTableIndex)
 {
     if (Options::dumpBytecodesBeforeGeneratorification())
-        BytecodeDumper<UnlinkedCodeBlock>::dumpBlock(codeBlock, instructions, WTF::dataFile());
+        CodeBlockBytecodeDumper<UnlinkedCodeBlock>::dumpBlock(codeBlock, instructions, WTF::dataFile());
 
     BytecodeGeneratorification pass(bytecodeGenerator, codeBlock, instructions, generatorFrameSymbolTable, generatorFrameSymbolTableIndex);
     pass.run();
index c086b8e..d652b65 100644 (file)
@@ -53,6 +53,7 @@ types [
     :SymbolTableOrScopeDepth,
     :ToThisStatus,
     :TypeLocation,
+    :WasmBoundLabel,
     :WatchpointSet,
 
     :ValueProfile,
@@ -73,7 +74,7 @@ templates [
 ]
 
 
-begin_section :Bytecodes,
+begin_section :Bytecode,
     emit_in_h_file: true,
     emit_in_structs_file: true,
     emit_in_asm_file: true,
@@ -1170,7 +1171,7 @@ op :super_sampler_begin
 
 op :super_sampler_end
 
-end_section :Bytecodes
+end_section :Bytecode
 
 begin_section :CLoopHelpers,
     emit_in_h_file: true,
@@ -1246,5 +1247,217 @@ op :op_get_by_id_return_location
 op :op_get_by_val_return_location
 op :op_put_by_id_return_location
 op :op_put_by_val_return_location
+op :wasm_function_prologue
+op :wasm_function_prologue_no_tls
 
 end_section :NativeHelpers
+
+begin_section :Wasm,
+    emit_in_h_file: true,
+    emit_in_structs_file: true,
+    macro_name_component: :WASM,
+    op_prefix: "wasm_"
+
+autogenerate_wasm_opcodes
+
+# Helpers
+
+op :throw_from_slow_path_trampoline
+
+# FIXME: Wasm and JS LLInt should share common opcodes
+# https://bugs.webkit.org/show_bug.cgi?id=203656
+
+op :wide16
+op :wide32
+
+op :enter
+op :nop
+op :loop_hint
+
+op :mov,
+    args: {
+        dst: VirtualRegister,
+        src: VirtualRegister,
+    }
+
+op_group :ConditionalJump,
+    [
+        :jtrue,
+        :jfalse,
+    ],
+    args: {
+        condition: VirtualRegister,
+        targetLabel: WasmBoundLabel,
+    }
+
+op :jmp,
+    args: {
+        targetLabel: WasmBoundLabel,
+    }
+
+op :ret,
+    args: {
+        stackOffset: unsigned,
+    }
+
+op :switch,
+    args: {
+        scrutinee: VirtualRegister,
+        tableIndex: unsigned,
+        defaultTarget: WasmBoundLabel,
+    }
+
+# Wasm specific bytecodes
+
+op :unreachable
+op :ret_void
+
+op :ref_is_null,
+    args: {
+        dst: VirtualRegister,
+        ref: VirtualRegister,
+    }
+
+op :ref_func,
+    args: {
+        dst: VirtualRegister,
+        functionIndex: unsigned,
+    }
+
+op :get_global,
+    args: {
+        dst: VirtualRegister,
+        globalIndex: unsigned,
+    }
+
+op :set_global,
+    args: {
+        globalIndex: unsigned,
+        value: VirtualRegister,
+    }
+
+op :set_global_ref,
+    args: {
+        globalIndex: unsigned,
+        value: VirtualRegister,
+    }
+
+op :table_get,
+    args: {
+        dst: VirtualRegister,
+        index: VirtualRegister,
+        tableIndex: unsigned,
+    }
+
+op :table_set,
+    args: {
+        index: VirtualRegister,
+        value: VirtualRegister,
+        tableIndex: unsigned,
+    }
+
+op :table_size,
+    args: {
+        dst: VirtualRegister,
+        tableIndex: unsigned,
+    }
+
+op :table_grow,
+    args: {
+        dst: VirtualRegister,
+        fill: VirtualRegister,
+        size: VirtualRegister,
+        tableIndex: unsigned,
+    }
+
+op :table_fill,
+    args: {
+        offset: VirtualRegister,
+        fill: VirtualRegister,
+        size: VirtualRegister,
+        tableIndex: unsigned,
+    }
+
+op :call,
+    args: {
+        functionIndex: unsigned,
+        stackOffset: unsigned,
+        numberOfStackArgs: unsigned,
+    }
+
+op :call_no_tls,
+    args: {
+        functionIndex: unsigned,
+        stackOffset: unsigned,
+        numberOfStackArgs: unsigned,
+    }
+
+op :call_indirect,
+    args: {
+        functionIndex: VirtualRegister,
+        signatureIndex: unsigned,
+        stackOffset: unsigned,
+        numberOfStackArgs: unsigned,
+        tableIndex: unsigned,
+    }
+
+op :call_indirect_no_tls,
+    args: {
+        functionIndex: VirtualRegister,
+        signatureIndex: unsigned,
+        stackOffset: unsigned,
+        numberOfStackArgs: unsigned,
+        tableIndex: unsigned,
+    }
+
+op :current_memory,
+    args: {
+        dst: VirtualRegister,
+    }
+
+op :grow_memory,
+    args: {
+        dst: VirtualRegister,
+        delta: VirtualRegister
+    }
+
+op :select,
+    args: {
+        dst: VirtualRegister,
+        condition: VirtualRegister,
+        nonZero: VirtualRegister,
+        zero: VirtualRegister,
+    }
+
+op_group :Load,
+    [
+        :load8_u,
+        :load16_u,
+        :load32_u,
+        :load64_u,
+        :i32_load8_s,
+        :i64_load8_s,
+        :i32_load16_s,
+        :i64_load16_s,
+        :i64_load32_s,
+    ],
+    args: {
+        dst: VirtualRegister,
+        pointer: VirtualRegister,
+        offset: unsigned,
+    }
+
+op_group :Store,
+    [
+        :store8,
+        :store16,
+        :store32,
+        :store64,
+    ],
+    args: {
+        pointer: VirtualRegister,
+        value: VirtualRegister,
+        offset: unsigned,
+    }
+
+end_section :Wasm
index 7e21168..5ec246c 100644 (file)
@@ -246,7 +246,7 @@ void CodeBlock::dumpBytecode(PrintStream& out)
 {
     ICStatusMap statusMap;
     getICStatusMap(statusMap);
-    BytecodeDumper<CodeBlock>::dumpBlock(this, instructions(), out, statusMap);
+    CodeBlockBytecodeDumper<CodeBlock>::dumpBlock(this, instructions(), out, statusMap);
 }
 
 void CodeBlock::dumpBytecode(PrintStream& out, const InstructionStream::Ref& it, const ICStatusMap& statusMap)
index 9ab7713..e244fd6 100644 (file)
@@ -290,8 +290,8 @@ struct Fits<OperandTypes, size, std::enable_if_t<sizeof(OperandTypes) != size, s
     }
 };
 
-template<OpcodeSize size>
-struct Fits<BoundLabel, size> : public Fits<int, size> {
+template<OpcodeSize size, typename GeneratorTraits>
+struct Fits<GenericBoundLabel<GeneratorTraits>, size> : public Fits<int, size> {
     // This is a bit hacky: we need to delay computing jump targets, since we
     // might have to emit `nop`s to align the instructions stream. Additionally,
     // we have to compute the target before we start writing to the instruction
@@ -300,19 +300,19 @@ struct Fits<BoundLabel, size> : public Fits<int, size> {
     // later we use the saved target when we call convert.
 
     using Base = Fits<int, size>;
-    static bool check(BoundLabel& label)
+    static bool check(GenericBoundLabel<GeneratorTraits>& label)
     {
         return Base::check(label.saveTarget());
     }
 
-    static typename Base::TargetType convert(BoundLabel& label)
+    static typename Base::TargetType convert(GenericBoundLabel<GeneratorTraits>& label)
     {
         return Base::convert(label.commitTarget());
     }
 
-    static BoundLabel convert(typename Base::TargetType target)
+    static GenericBoundLabel<GeneratorTraits> convert(typename Base::TargetType target)
     {
-        return BoundLabel(Base::convert(target));
+        return GenericBoundLabel<GeneratorTraits>(Base::convert(target));
     }
 };
 
index 651ce8f..159d26a 100644 (file)
 
 namespace JSC {
 
-struct Instruction {
+struct JSOpcodeTraits {
+    using OpcodeID = ::JSC::OpcodeID;
+    static constexpr OpcodeID numberOfBytecodesWithMetadata = static_cast<OpcodeID>(NUMBER_OF_BYTECODE_WITH_METADATA);
+    static constexpr OpcodeID wide16 = op_wide16;
+    static constexpr OpcodeID wide32 = op_wide32;
+    static constexpr const unsigned* opcodeLengths = ::JSC::opcodeLengths;
+    static constexpr const char* const* opcodeNames = ::JSC::opcodeNames;
+};
+
+struct WasmOpcodeTraits {
+    using OpcodeID = WasmOpcodeID;
+    static constexpr OpcodeID numberOfBytecodesWithMetadata = static_cast<OpcodeID>(NUMBER_OF_WASM_WITH_METADATA);
+    static constexpr OpcodeID wide16 = wasm_wide16;
+    static constexpr OpcodeID wide32 = wasm_wide32;
+    static constexpr const unsigned* opcodeLengths = wasmOpcodeLengths;
+    static constexpr const char* const* opcodeNames = wasmOpcodeNames;
+};
+
+
+template<typename Opcode>
+struct BaseInstruction {
 
     struct Metadata { };
 
 protected:
-    Instruction()
+    BaseInstruction()
     { }
 
 private:
     template<OpcodeSize Width>
     class Impl {
     public:
-        OpcodeID opcodeID() const { return static_cast<OpcodeID>(m_opcode); }
+        template<typename Traits = JSOpcodeTraits>
+        typename Traits::OpcodeID opcodeID() const { return static_cast<typename Traits::OpcodeID>(m_opcode); }
 
     private:
         typename TypeBySize<Width>::unsignedType m_opcode;
     };
 
 public:
-    OpcodeID opcodeID() const
+    template<typename Traits = JSOpcodeTraits>
+    typename Traits::OpcodeID opcodeID() const
     {
-        if (isWide32())
-            return wide32()->opcodeID();
-        if (isWide16())
-            return wide16()->opcodeID();
-        return narrow()->opcodeID();
+        if (isWide32<Traits>())
+            return wide32<Traits>()->template opcodeID<Traits>();
+        if (isWide16<Traits>())
+            return wide16<Traits>()->template opcodeID<Traits>();
+        return narrow()->template opcodeID<Traits>();
     }
 
+    template<typename Traits = JSOpcodeTraits>
     const char* name() const
     {
-        return opcodeNames[opcodeID()];
+        return Traits::opcodeNames[opcodeID()];
     }
 
+    template<typename Traits = JSOpcodeTraits>
     bool isWide16() const
     {
-        return narrow()->opcodeID() == op_wide16;
+        return narrow()->template opcodeID<Traits>() == Traits::wide16;
     }
 
+    template<typename Traits = JSOpcodeTraits>
     bool isWide32() const
     {
-        return narrow()->opcodeID() == op_wide32;
+        return narrow()->template opcodeID<Traits>() == Traits::wide32;
     }
 
+    template<typename Traits = JSOpcodeTraits>
     bool hasMetadata() const
     {
-        return opcodeID() < NUMBER_OF_BYTECODE_WITH_METADATA;
+        return opcodeID<Traits>() < Traits::numberOfBytecodesWithMetadata;
     }
 
+    template<typename Traits = JSOpcodeTraits>
     int sizeShiftAmount() const
     {
-        if (isWide32())
+        if (isWide32<Traits>())
             return 2;
-        if (isWide16())
+        if (isWide16<Traits>())
             return 1;
         return 0;
     }
 
+    template<typename Traits = JSOpcodeTraits>
     size_t size() const
     {
-        auto sizeShiftAmount = this->sizeShiftAmount();
+        auto sizeShiftAmount = this->sizeShiftAmount<Traits>();
         auto padding = sizeShiftAmount ? 1 : 0;
         auto size = 1 << sizeShiftAmount;
-        return opcodeLengths[opcodeID()] * size + padding;
+        return Traits::opcodeLengths[opcodeID<Traits>()] * size + padding;
     }
 
-    template<class T>
+    template<class T, typename Traits = JSOpcodeTraits>
     bool is() const
     {
-        return opcodeID() == T::opcodeID;
+        return opcodeID<Traits>() == T::opcodeID;
     }
 
-    template<class T>
+    template<class T, typename Traits = JSOpcodeTraits>
     T as() const
     {
-        ASSERT(is<T>());
+        ASSERT((is<T, Traits>()));
         return T::decode(reinterpret_cast<const uint8_t*>(this));
     }
 
-    template<class T>
+    template<class T, typename Traits = JSOpcodeTraits>
     T* cast()
     {
-        ASSERT(is<T>());
+        ASSERT((is<T, Traits>()));
         return bitwise_cast<T*>(this);
     }
 
-    template<class T>
+    template<class T, typename Traits = JSOpcodeTraits>
     const T* cast() const
     {
-        ASSERT(is<T>());
+        ASSERT((is<T, Traits>()));
         return reinterpret_cast<const T*>(this);
     }
 
@@ -127,19 +155,27 @@ public:
         return reinterpret_cast<const Impl<OpcodeSize::Narrow>*>(this);
     }
 
+    template<typename Traits = JSOpcodeTraits>
     const Impl<OpcodeSize::Wide16>* wide16() const
     {
 
-        ASSERT(isWide16());
+        ASSERT(isWide16<Traits>());
         return reinterpret_cast<const Impl<OpcodeSize::Wide16>*>(bitwise_cast<uintptr_t>(this) + 1);
     }
 
+    template<typename Traits = JSOpcodeTraits>
     const Impl<OpcodeSize::Wide32>* wide32() const
     {
 
-        ASSERT(isWide32());
+        ASSERT(isWide32<Traits>());
         return reinterpret_cast<const Impl<OpcodeSize::Wide32>*>(bitwise_cast<uintptr_t>(this) + 1);
     }
 };
 
+struct Instruction : public BaseInstruction<OpcodeID> {
+};
+
+struct WasmInstance : public BaseInstruction<WasmOpcodeID> {
+};
+
 } // namespace JSC
index a5dedcb..5963dea 100644 (file)
@@ -32,8 +32,6 @@
 
 namespace JSC {
 
-struct Instruction;
-
 class InstructionStream {
     WTF_MAKE_FAST_ALLOCATED;
 
@@ -133,11 +131,16 @@ private:
             return *this;
         }
 
-        iterator operator++()
+        iterator& operator+=(size_t size)
         {
-            m_index += ptr()->size();
+            m_index += size;
             return *this;
         }
+
+        iterator& operator++()
+        {
+            return *this += ptr()->size();
+        }
     };
 
 public:
@@ -278,11 +281,16 @@ private:
             return *this;
         }
 
-        iterator operator++()
+        iterator& operator+=(size_t size)
         {
-            m_index += ptr()->size();
+            m_index += size;
             return *this;
         }
+
+        iterator& operator++()
+        {
+            return *this += ptr()->size();
+        }
     };
 
 public:
index 0c05f83..d3fad98 100644 (file)
@@ -52,6 +52,18 @@ const char* const opcodeNames[] = {
 #undef OPCODE_NAME_ENTRY
 };
 
+const unsigned wasmOpcodeLengths[] = {
+#define OPCODE_LENGTH(opcode, length) length,
+    FOR_EACH_WASM_ID(OPCODE_LENGTH)
+#undef OPCODE_LENGTH
+};
+
+const char* const wasmOpcodeNames[] = {
+#define OPCODE_NAME_ENTRY(opcode, size) #opcode,
+    FOR_EACH_WASM_ID(OPCODE_NAME_ENTRY)
+#undef OPCODE_NAME_ENTRY
+};
+
 #if ENABLE(OPCODE_STATS)
 
 inline const char* padOpcodeName(OpcodeID op, unsigned width)
index f314709..d3debed 100644 (file)
@@ -60,25 +60,32 @@ const int numOpcodeIDs = NUMBER_OF_BYTECODE_IDS + NUMBER_OF_CLOOP_BYTECODE_HELPE
 const int numOpcodeIDs = NUMBER_OF_BYTECODE_IDS + NUMBER_OF_BYTECODE_HELPER_IDS;
 #endif
 
+constexpr int numWasmOpcodeIDs = NUMBER_OF_WASM_IDS + NUMBER_OF_BYTECODE_HELPER_IDS;
+
 #define OPCODE_ID_ENUM(opcode, length) opcode,
     enum OpcodeID : unsigned { FOR_EACH_OPCODE_ID(OPCODE_ID_ENUM) };
+    enum WasmOpcodeID : unsigned { FOR_EACH_WASM_ID(OPCODE_ID_ENUM) };
 #undef OPCODE_ID_ENUM
 
 #if ENABLE(C_LOOP) && !HAVE(COMPUTED_GOTO)
 
 #define OPCODE_ID_ENUM(opcode, length) opcode##_wide16 = numOpcodeIDs + opcode,
     enum OpcodeIDWide16 : unsigned { FOR_EACH_OPCODE_ID(OPCODE_ID_ENUM) };
+    enum WasmOpcodeIDWide16 : unsigned { FOR_EACH_WASM_ID(OPCODE_ID_ENUM) };
 #undef OPCODE_ID_ENUM
 
 #define OPCODE_ID_ENUM(opcode, length) opcode##_wide32 = numOpcodeIDs * 2 + opcode,
     enum OpcodeIDWide32 : unsigned { FOR_EACH_OPCODE_ID(OPCODE_ID_ENUM) };
+    enum WasmOpcodeIDWide32 : unsigned { FOR_EACH_WASM_ID(OPCODE_ID_ENUM) };
 #undef OPCODE_ID_ENUM
 #endif
 
 extern const unsigned opcodeLengths[];
+extern const unsigned wasmOpcodeLengths[];
 
 #define OPCODE_ID_LENGTHS(id, length) const int id##_length = length;
     FOR_EACH_OPCODE_ID(OPCODE_ID_LENGTHS);
+    FOR_EACH_WASM_ID(OPCODE_ID_LENGTHS);
 #undef OPCODE_ID_LENGTHS
 
 #define FOR_EACH_OPCODE_WITH_VALUE_PROFILE(macro) \
@@ -151,6 +158,7 @@ typedef OpcodeID Opcode;
 #endif
 
 extern const char* const opcodeNames[];
+extern const char* const wasmOpcodeNames[];
 
 #if ENABLE(OPCODE_STATS)
 
index 90052b5..4d0716e 100644 (file)
@@ -25,6 +25,7 @@
 
 #pragma once
 
+#include "BytecodeGenerator.h"
 #include "BytecodeStructs.h"
 #include "InterpreterInlines.h"
 #include "Opcode.h"
index 1ba907f..4739da7 100644 (file)
@@ -50,7 +50,6 @@
 
 namespace JSC {
 
-class BytecodeGenerator;
 class BytecodeLivenessAnalysis;
 class BytecodeRewriter;
 class CodeBlock;
@@ -384,6 +383,8 @@ protected:
 private:
     friend class BytecodeRewriter;
     friend class BytecodeGenerator;
+    template<typename Traits>
+    friend class BytecodeGeneratorBase;
 
     template<typename CodeBlockType>
     friend class CachedCodeBlock;
index 7e18ca8..45c6ba1 100644 (file)
@@ -83,4 +83,9 @@ VirtualRegister::VirtualRegister(RegisterID* reg)
 {
 }
 
+VirtualRegister::VirtualRegister(RefPtr<RegisterID> reg)
+    : VirtualRegister(reg.get())
+{
+}
+
 } // namespace JSC
index d95ed31..3619529 100644 (file)
@@ -50,6 +50,7 @@ public:
     friend VirtualRegister virtualRegisterForArgument(int, int);
 
     VirtualRegister(RegisterID*);
+    VirtualRegister(RefPtr<RegisterID>);
 
     VirtualRegister()
         : m_virtualRegister(s_invalidVirtualRegister)
index 154bd3c..2db818f 100644 (file)
@@ -34,9 +34,9 @@
 #include "ArithProfile.h"
 #include "BuiltinExecutables.h"
 #include "BuiltinNames.h"
+#include "BytecodeGeneratorBaseInlines.h"
 #include "BytecodeGeneratorification.h"
 #include "BytecodeLivenessAnalysis.h"
-#include "BytecodeStructs.h"
 #include "BytecodeUseDef.h"
 #include "CatchScope.h"
 #include "DefinePropertyAttributes.h"
@@ -85,15 +85,8 @@ struct VarArgsOp<CallOp, std::enable_if_t<!std::is_same<CallOp, OpTailCall>::val
     using type = OpCallVarargs;
 };
 
-
-template<typename T>
-static inline void shrinkToFit(T& segmentedVector)
-{
-    while (segmentedVector.size() && !segmentedVector.last().refCount())
-        segmentedVector.removeLast();
-}
-
-void Label::setLocation(BytecodeGenerator& generator, unsigned location)
+template<>
+void GenericLabel<JSGeneratorTraits>::setLocation(BytecodeGenerator& generator, unsigned location)
 {
     m_location = location;
     
@@ -139,41 +132,6 @@ void Label::setLocation(BytecodeGenerator& generator, unsigned location)
     }
 }
 
-int BoundLabel::target()
-{
-    switch (m_type) {
-    case Offset:
-        return m_target;
-    case GeneratorBackward:
-        return m_target - m_generator->m_writer.position();
-    case GeneratorForward:
-        return 0;
-    default:
-        RELEASE_ASSERT_NOT_REACHED();
-    }
-}
-
-int BoundLabel::saveTarget()
-{
-    if (m_type == GeneratorForward) {
-        m_savedTarget = m_generator->m_writer.position();
-        return 0;
-    }
-
-    m_savedTarget = target();
-    return m_savedTarget;
-}
-
-int BoundLabel::commitTarget()
-{
-    if (m_type == GeneratorForward) {
-        m_label->m_unresolvedJumps.append(m_savedTarget);
-        return 0;
-    }
-
-    return m_savedTarget;
-}
-
 void Variable::dump(PrintStream& out) const
 {
     out.print(
@@ -340,9 +298,9 @@ ParserError BytecodeGenerator::generate()
 }
 
 BytecodeGenerator::BytecodeGenerator(VM& vm, ProgramNode* programNode, UnlinkedProgramCodeBlock* codeBlock, OptionSet<CodeGenerationMode> codeGenerationMode, const VariableEnvironment* parentScopeTDZVariables)
-    : m_codeGenerationMode(codeGenerationMode)
+    : BytecodeGeneratorBase(Strong<UnlinkedCodeBlock>(vm, codeBlock), CodeBlock::llintBaselineCalleeSaveSpaceAsVirtualRegisters())
+    , m_codeGenerationMode(codeGenerationMode)
     , m_scopeNode(programNode)
-    , m_codeBlock(vm, codeBlock)
     , m_thisRegister(CallFrame::thisArgumentOffset())
     , m_codeType(GlobalCode)
     , m_vm(vm)
@@ -353,8 +311,6 @@ BytecodeGenerator::BytecodeGenerator(VM& vm, ProgramNode* programNode, UnlinkedP
     for (auto& constantRegister : m_linkTimeConstantRegisters)
         constantRegister = nullptr;
 
-    allocateCalleeSaveSpace();
-
     m_codeBlock->setNumParameters(1); // Allocate space for "this"
 
     emitEnter();
@@ -386,9 +342,9 @@ BytecodeGenerator::BytecodeGenerator(VM& vm, ProgramNode* programNode, UnlinkedP
 }
 
 BytecodeGenerator::BytecodeGenerator(VM& vm, FunctionNode* functionNode, UnlinkedFunctionCodeBlock* codeBlock, OptionSet<CodeGenerationMode> codeGenerationMode, const VariableEnvironment* parentScopeTDZVariables)
-    : m_codeGenerationMode(codeGenerationMode)
+    : BytecodeGeneratorBase(Strong<UnlinkedCodeBlock>(vm, codeBlock), CodeBlock::llintBaselineCalleeSaveSpaceAsVirtualRegisters())
+    , m_codeGenerationMode(codeGenerationMode)
     , m_scopeNode(functionNode)
-    , m_codeBlock(vm, codeBlock)
     , m_codeType(FunctionCode)
     , m_vm(vm)
     , m_isBuiltinFunction(codeBlock->isBuiltinFunction())
@@ -407,8 +363,6 @@ BytecodeGenerator::BytecodeGenerator(VM& vm, FunctionNode* functionNode, Unlinke
     for (auto& constantRegister : m_linkTimeConstantRegisters)
         constantRegister = nullptr;
 
-    allocateCalleeSaveSpace();
-    
     SymbolTable* functionSymbolTable = SymbolTable::create(m_vm);
     functionSymbolTable->setUsesNonStrictEval(m_usesNonStrictEval);
     int symbolTableConstantIndex = 0;
@@ -884,9 +838,9 @@ BytecodeGenerator::BytecodeGenerator(VM& vm, FunctionNode* functionNode, Unlinke
 }
 
 BytecodeGenerator::BytecodeGenerator(VM& vm, EvalNode* evalNode, UnlinkedEvalCodeBlock* codeBlock, OptionSet<CodeGenerationMode> codeGenerationMode, const VariableEnvironment* parentScopeTDZVariables)
-    : m_codeGenerationMode(codeGenerationMode)
+    : BytecodeGeneratorBase(Strong<UnlinkedCodeBlock>(vm, codeBlock), CodeBlock::llintBaselineCalleeSaveSpaceAsVirtualRegisters())
+    , m_codeGenerationMode(codeGenerationMode)
     , m_scopeNode(evalNode)
-    , m_codeBlock(vm, codeBlock)
     , m_thisRegister(CallFrame::thisArgumentOffset())
     , m_codeType(EvalCode)
     , m_vm(vm)
@@ -897,8 +851,6 @@ BytecodeGenerator::BytecodeGenerator(VM& vm, EvalNode* evalNode, UnlinkedEvalCod
     for (auto& constantRegister : m_linkTimeConstantRegisters)
         constantRegister = nullptr;
 
-    allocateCalleeSaveSpace();
-
     m_codeBlock->setNumParameters(1);
 
     pushTDZVariables(*parentScopeTDZVariables, TDZCheckOptimization::DoNotOptimize, TDZRequirement::UnderTDZ);
@@ -947,9 +899,9 @@ BytecodeGenerator::BytecodeGenerator(VM& vm, EvalNode* evalNode, UnlinkedEvalCod
 }
 
 BytecodeGenerator::BytecodeGenerator(VM& vm, ModuleProgramNode* moduleProgramNode, UnlinkedModuleProgramCodeBlock* codeBlock, OptionSet<CodeGenerationMode> codeGenerationMode, const VariableEnvironment* parentScopeTDZVariables)
-    : m_codeGenerationMode(codeGenerationMode)
+    : BytecodeGeneratorBase(Strong<UnlinkedCodeBlock>(vm, codeBlock), CodeBlock::llintBaselineCalleeSaveSpaceAsVirtualRegisters())
+    , m_codeGenerationMode(codeGenerationMode)
     , m_scopeNode(moduleProgramNode)
-    , m_codeBlock(vm, codeBlock)
     , m_thisRegister(CallFrame::thisArgumentOffset())
     , m_codeType(ModuleCode)
     , m_vm(vm)
@@ -961,8 +913,6 @@ BytecodeGenerator::BytecodeGenerator(VM& vm, ModuleProgramNode* moduleProgramNod
     for (auto& constantRegister : m_linkTimeConstantRegisters)
         constantRegister = nullptr;
 
-    allocateCalleeSaveSpace();
-
     SymbolTable* moduleEnvironmentSymbolTable = SymbolTable::create(m_vm);
     moduleEnvironmentSymbolTable->setUsesNonStrictEval(m_usesNonStrictEval);
     moduleEnvironmentSymbolTable->setScopeType(SymbolTable::ScopeType::LexicalScope);
@@ -1308,20 +1258,6 @@ UniquedStringImpl* BytecodeGenerator::visibleNameForParameter(DestructuringPatte
     return nullptr;
 }
 
-RegisterID* BytecodeGenerator::newRegister()
-{
-    m_calleeLocals.append(virtualRegisterForLocal(m_calleeLocals.size()));
-    int numCalleeLocals = std::max<int>(m_codeBlock->m_numCalleeLocals, m_calleeLocals.size());
-    numCalleeLocals = WTF::roundUpToMultipleOf(stackAlignmentRegisters(), numCalleeLocals);
-    m_codeBlock->m_numCalleeLocals = numCalleeLocals;
-    return &m_calleeLocals.last();
-}
-
-void BytecodeGenerator::reclaimFreeRegisters()
-{
-    shrinkToFit(m_calleeLocals);
-}
-
 RegisterID* BytecodeGenerator::newBlockScopeVariable()
 {
     reclaimFreeRegisters();
@@ -1329,15 +1265,6 @@ RegisterID* BytecodeGenerator::newBlockScopeVariable()
     return newRegister();
 }
 
-RegisterID* BytecodeGenerator::newTemporary()
-{
-    reclaimFreeRegisters();
-
-    RegisterID* result = newRegister();
-    result->setTemporary();
-    return result;
-}
-
 Ref<LabelScope> BytecodeGenerator::newLabelScope(LabelScope::Type type, const Identifier* name)
 {
     shrinkToFit(m_labelScopes);
@@ -1347,65 +1274,6 @@ Ref<LabelScope> BytecodeGenerator::newLabelScope(LabelScope::Type type, const Id
     return m_labelScopes.last();
 }
 
-Ref<Label> BytecodeGenerator::newLabel()
-{
-    shrinkToFit(m_labels);
-
-    // Allocate new label ID.
-    m_labels.append();
-    return m_labels.last();
-}
-
-Ref<Label> BytecodeGenerator::newEmittedLabel()
-{
-    Ref<Label> label = newLabel();
-    emitLabel(label.get());
-    return label;
-}
-
-void BytecodeGenerator::recordOpcode(OpcodeID opcodeID)
-{
-    ASSERT(m_lastOpcodeID == op_end || (m_lastOpcodeID == m_lastInstruction->opcodeID() && m_writer.position() == m_lastInstruction.offset() + m_lastInstruction->size()));
-    m_lastInstruction = m_writer.ref();
-    m_lastOpcodeID = opcodeID;
-}
-
-void BytecodeGenerator::alignWideOpcode16()
-{
-#if CPU(NEEDS_ALIGNED_ACCESS)
-    while ((m_writer.position() + 1) % OpcodeSize::Wide16)
-        OpNop::emit<OpcodeSize::Narrow>(this);
-#endif
-}
-
-void BytecodeGenerator::alignWideOpcode32()
-{
-#if CPU(NEEDS_ALIGNED_ACCESS)
-    while ((m_writer.position() + 1) % OpcodeSize::Wide32)
-        OpNop::emit<OpcodeSize::Narrow>(this);
-#endif
-}
-
-void BytecodeGenerator::emitLabel(Label& l0)
-{
-    unsigned newLabelIndex = instructions().size();
-    l0.setLocation(*this, newLabelIndex);
-
-    if (m_codeBlock->numberOfJumpTargets()) {
-        unsigned lastLabelIndex = m_codeBlock->lastJumpTarget();
-        ASSERT(lastLabelIndex <= newLabelIndex);
-        if (newLabelIndex == lastLabelIndex) {
-            // Peephole optimizations have already been disabled by emitting the last label
-            return;
-        }
-    }
-
-    m_codeBlock->addJumpTarget(newLabelIndex);
-
-    // This disables peephole optimizations when an instruction is a jump target
-    m_lastOpcodeID = op_end;
-}
-
 void BytecodeGenerator::emitEnter()
 {
     OpEnter::emit(this);
@@ -3687,17 +3555,6 @@ LabelScope* BytecodeGenerator::continueTarget(const Identifier& name)
     return nullptr;
 }
 
-void BytecodeGenerator::allocateCalleeSaveSpace()
-{
-    size_t virtualRegisterCountForCalleeSaves = CodeBlock::llintBaselineCalleeSaveSpaceAsVirtualRegisters();
-
-    for (size_t i = 0; i < virtualRegisterCountForCalleeSaves; i++) {
-        RegisterID* localRegister = addVar();
-        localRegister->ref();
-        m_localRegistersForCalleeSaveRegisters.append(localRegister);
-    }
-}
-
 void BytecodeGenerator::allocateAndEmitScope()
 {
     m_scopeRegister = addVar();
index 4197abc..2d721eb 100644 (file)
@@ -30,6 +30,8 @@
 
 #pragma once
 
+#include "BytecodeGeneratorBase.h"
+#include "BytecodeStructs.h"
 #include "CodeBlock.h"
 #include "Instruction.h"
 #include "Interpreter.h"
@@ -360,13 +362,20 @@ namespace JSC {
         TryData* tryData;
     };
 
-    class BytecodeGenerator {
+
+    struct JSGeneratorTraits {
+        using OpcodeTraits = JSOpcodeTraits;
+        using OpcodeID = ::JSC::OpcodeID;
+        using OpNop = ::JSC::OpNop;
+        using CodeBlock = Strong<UnlinkedCodeBlock>;
+        static constexpr OpcodeID opcodeForDisablingOptimizations = op_end;
+    };
+
+    class BytecodeGenerator : public BytecodeGeneratorBase<JSGeneratorTraits> {
         WTF_MAKE_FAST_ALLOCATED;
         WTF_MAKE_NONCOPYABLE(BytecodeGenerator);
 
-        friend class BoundLabel;
         friend class FinallyContext;
-        friend class Label;
         friend class IndexedForInContext;
         friend class StructureForInContext;
     public:
@@ -433,12 +442,6 @@ namespace JSC {
 
         RegisterID* promiseRegister() { return m_promiseRegister; }
 
-        // Returns the next available temporary register. Registers returned by
-        // newTemporary require a modified form of reference counting: any
-        // register with a refcount of 0 is considered "available", meaning that
-        // the next instruction may overwrite it.
-        RegisterID* newTemporary();
-
         // The same as newTemporary(), but this function returns "suggestion" if
         // "suggestion" is a temporary. This function is helpful in situations
         // where you've put "suggestion" in a RefPtr, but you'd like to allow
@@ -488,8 +491,6 @@ namespace JSC {
         }
 
         Ref<LabelScope> newLabelScope(LabelScope::Type, const Identifier* = 0);
-        Ref<Label> newLabel();
-        Ref<Label> newEmittedLabel();
 
         void emitNode(RegisterID* dst, StatementNode* n)
         {
@@ -510,8 +511,6 @@ namespace JSC {
             n->emitBytecode(*this, dst);
         }
 
-        void recordOpcode(OpcodeID);
-
         ALWAYS_INLINE unsigned addMetadataFor(OpcodeID opcodeID)
         {
             return m_codeBlock->metadata().addEntry(opcodeID);
@@ -840,7 +839,6 @@ namespace JSC {
 
         RegisterID* initializeVariable(const Variable&, RegisterID* value);
 
-        void emitLabel(Label&);
         void emitLoopHint();
         void emitJump(Label& target);
         void emitJumpIfTrue(RegisterID* cond, Label& target);
@@ -1055,7 +1053,6 @@ namespace JSC {
 
     private:
         ParserError generate();
-        void reclaimFreeRegisters();
         Variable variableForLocalEntry(const Identifier&, const SymbolTableEntry&, int symbolTableConstantIndex, bool isLexicallyScoped);
 
         RegisterID* kill(RegisterID* dst)
@@ -1067,7 +1064,6 @@ namespace JSC {
         void retrieveLastUnaryOp(int& dstIndex, int& srcIndex);
         ALWAYS_INLINE void rewind();
 
-        void allocateCalleeSaveSpace();
         void allocateAndEmitScope();
 
         template<typename JumpOp>
@@ -1090,17 +1086,6 @@ namespace JSC {
         RegisterID* emitCall(RegisterID* dst, RegisterID* func, ExpectedFunction, CallArguments&, const JSTextPosition& divot, const JSTextPosition& divotStart, const JSTextPosition& divotEnd, DebuggableCall);
 
         RegisterID* emitCallIterator(RegisterID* iterator, RegisterID* argument, ThrowableExpressionData*);
-        RegisterID* newRegister();
-
-        // Adds an anonymous local var slot. To give this slot a name, add it to symbolTable().
-        RegisterID* addVar()
-        {
-            ++m_codeBlock->m_numVars;
-            RegisterID* result = newRegister();
-            ASSERT(VirtualRegister(result->index()).toLocal() == m_codeBlock->m_numVars - 1);
-            result->ref(); // We should never free this slot.
-            return result;
-        }
 
         // Initializes the stack form the parameter; does nothing for the symbol table.
         RegisterID* initializeNextParameter();
@@ -1177,15 +1162,6 @@ namespace JSC {
 
         RegisterID* emitThrowExpressionTooDeepException();
 
-        void write(uint8_t byte) { m_writer.write(byte); }
-        void write(uint16_t h) { m_writer.write(h); }
-        void write(uint32_t i) { m_writer.write(i); }
-        void write(int8_t byte) { m_writer.write(static_cast<uint8_t>(byte)); }
-        void write(int16_t h) { m_writer.write(static_cast<uint16_t>(h)); }
-        void write(int32_t i) { m_writer.write(static_cast<uint32_t>(i)); }
-        void alignWideOpcode16();
-        void alignWideOpcode32();
-
         class PreservedTDZStack {
         private:
             Vector<TDZMap> m_preservedTDZStack;
@@ -1210,8 +1186,6 @@ namespace JSC {
         }
 
     private:
-        InstructionStreamWriter m_writer;
-
         OptionSet<CodeGenerationMode> m_codeGenerationMode;
 
         struct LexicalScopeStackEntry {
@@ -1227,7 +1201,6 @@ namespace JSC {
         void pushTDZVariables(const VariableEnvironment&, TDZCheckOptimization, TDZRequirement);
 
         ScopeNode* const m_scopeNode;
-        Strong<UnlinkedCodeBlock> m_codeBlock;
 
         // Some of these objects keep pointers to one another. They are arranged
         // to ensure a sane destruction order that avoids references to freed memory.
@@ -1249,11 +1222,7 @@ namespace JSC {
 
         FinallyContext* m_currentFinallyContext { nullptr };
 
-        SegmentedVector<RegisterID*, 16> m_localRegistersForCalleeSaveRegisters;
-        SegmentedVector<RegisterID, 32> m_constantPoolRegisters;
-        SegmentedVector<RegisterID, 32> m_calleeLocals;
         SegmentedVector<RegisterID, 32> m_parameters;
-        SegmentedVector<Label, 32> m_labels;
         SegmentedVector<LabelScope, 32> m_labelScopes;
         unsigned m_finallyDepth { 0 };
         unsigned m_localScopeDepth { 0 };
@@ -1303,9 +1272,6 @@ namespace JSC {
 
         VM& m_vm;
 
-        OpcodeID m_lastOpcodeID = op_end;
-        InstructionStream::MutableRef m_lastInstruction { m_writer.ref() };
-
         bool m_usesExceptions { false };
         bool m_expressionTooDeep { false };
         bool m_isBuiltinFunction { false };
diff --git a/Source/JavaScriptCore/bytecompiler/BytecodeGeneratorBase.h b/Source/JavaScriptCore/bytecompiler/BytecodeGeneratorBase.h
new file mode 100644 (file)
index 0000000..1bc7a7b
--- /dev/null
@@ -0,0 +1,91 @@
+/*
+ * Copyright (C) 2019 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#include "InstructionStream.h"
+#include <wtf/SegmentedVector.h>
+
+namespace JSC {
+
+class RegisterID;
+
+template<typename BytecodeGenerator>
+class GenericBoundLabel;
+
+template<typename BytecodeGenerator>
+class GenericLabel;
+
+template<typename Traits>
+class BytecodeGeneratorBase {
+    template<typename BytecodeGenerator>
+    friend class GenericBoundLabel;
+
+    template<typename BytecodeGenerator>
+    friend class GenericLabel;
+
+public:
+    BytecodeGeneratorBase(typename Traits::CodeBlock, uint32_t virtualRegisterCountForCalleeSaves);
+
+    void allocateCalleeSaveSpace(uint32_t virtualRegisterCountForCalleeSaves);
+
+    Ref<GenericLabel<Traits>> newLabel();
+    Ref<GenericLabel<Traits>> newEmittedLabel();
+    RegisterID* newRegister();
+    RegisterID* addVar();
+
+    // Returns the next available temporary register. Registers returned by
+    // newTemporary require a modified form of reference counting: any
+    // register with a refcount of 0 is considered "available", meaning that
+    // the next instruction may overwrite it.
+    RegisterID* newTemporary();
+
+    void emitLabel(GenericLabel<Traits>&);
+    void recordOpcode(typename Traits::OpcodeID);
+    void alignWideOpcode16();
+    void alignWideOpcode32();
+
+    void write(uint8_t);
+    void write(uint16_t);
+    void write(uint32_t);
+    void write(int8_t);
+    void write(int16_t);
+    void write(int32_t);
+
+protected:
+    void reclaimFreeRegisters();
+
+    InstructionStreamWriter m_writer;
+    typename Traits::CodeBlock m_codeBlock;
+
+    typename Traits::OpcodeID m_lastOpcodeID = Traits::opcodeForDisablingOptimizations;
+    InstructionStream::MutableRef m_lastInstruction { m_writer.ref() };
+
+    SegmentedVector<GenericLabel<Traits>, 32> m_labels;
+    SegmentedVector<RegisterID, 32> m_calleeLocals;
+    SegmentedVector<RegisterID, 32> m_constantPoolRegisters;
+};
+
+} // namespace JSC
diff --git a/Source/JavaScriptCore/bytecompiler/BytecodeGeneratorBaseInlines.h b/Source/JavaScriptCore/bytecompiler/BytecodeGeneratorBaseInlines.h
new file mode 100644 (file)
index 0000000..e74bc23
--- /dev/null
@@ -0,0 +1,192 @@
+/*
+ * Copyright (C) 2019 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "BytecodeGeneratorBase.h"
+
+#include "RegisterID.h"
+#include "StackAlignment.h"
+
+namespace JSC {
+
+template<typename T>
+static inline void shrinkToFit(T& segmentedVector)
+{
+    while (segmentedVector.size() && !segmentedVector.last().refCount())
+        segmentedVector.removeLast();
+}
+
+template<typename Traits>
+BytecodeGeneratorBase<Traits>::BytecodeGeneratorBase(typename Traits::CodeBlock codeBlock, uint32_t virtualRegisterCountForCalleeSaves)
+    : m_codeBlock(WTFMove(codeBlock))
+{
+    allocateCalleeSaveSpace(virtualRegisterCountForCalleeSaves);
+}
+
+template<typename Traits>
+Ref<GenericLabel<Traits>> BytecodeGeneratorBase<Traits>::newLabel()
+{
+    shrinkToFit(m_labels);
+
+    // Allocate new label ID.
+    m_labels.append();
+    return m_labels.last();
+}
+
+template<typename Traits>
+Ref<GenericLabel<Traits>> BytecodeGeneratorBase<Traits>::newEmittedLabel()
+{
+    auto label = newLabel();
+    emitLabel(label.get());
+    return label;
+}
+
+template<typename Traits>
+void BytecodeGeneratorBase<Traits>::reclaimFreeRegisters()
+{
+    shrinkToFit(m_calleeLocals);
+}
+
+template<typename Traits>
+void BytecodeGeneratorBase<Traits>::emitLabel(GenericLabel<Traits>& label)
+{
+    unsigned newLabelIndex = m_writer.position();
+    label.setLocation(*this, newLabelIndex);
+
+    if (m_codeBlock->numberOfJumpTargets()) {
+        unsigned lastLabelIndex = m_codeBlock->lastJumpTarget();
+        ASSERT(lastLabelIndex <= newLabelIndex);
+        if (newLabelIndex == lastLabelIndex) {
+            // Peephole optimizations have already been disabled by emitting the last label
+            return;
+        }
+    }
+
+    m_codeBlock->addJumpTarget(newLabelIndex);
+
+    m_lastOpcodeID = Traits::opcodeForDisablingOptimizations;
+}
+
+template<typename Traits>
+void BytecodeGeneratorBase<Traits>::recordOpcode(typename Traits::OpcodeID opcodeID)
+{
+    ASSERT(m_lastOpcodeID == Traits::opcodeForDisablingOptimizations || (m_lastOpcodeID == m_lastInstruction->opcodeID<typename Traits::OpcodeTraits>() && m_writer.position() == m_lastInstruction.offset() + m_lastInstruction->size<typename Traits::OpcodeTraits>()));
+    m_lastInstruction = m_writer.ref();
+    m_lastOpcodeID = opcodeID;
+}
+
+template<typename Traits>
+void BytecodeGeneratorBase<Traits>::alignWideOpcode16()
+{
+#if CPU(NEEDS_ALIGNED_ACCESS)
+    while ((m_writer.position() + 1) % OpcodeSize::Wide16)
+        Traits::OpNop::emit<OpcodeSize::Narrow>(this);
+#endif
+}
+
+template<typename Traits>
+void BytecodeGeneratorBase<Traits>::alignWideOpcode32()
+{
+#if CPU(NEEDS_ALIGNED_ACCESS)
+    while ((m_writer.position() + 1) % OpcodeSize::Wide32)
+        Traits::OpNop::emit<OpcodeSize::Narrow>(this);
+#endif
+}
+
+template<typename Traits>
+void BytecodeGeneratorBase<Traits>::write(uint8_t b)
+{
+    m_writer.write(b);
+}
+
+
+template<typename Traits>
+void BytecodeGeneratorBase<Traits>::write(uint16_t h)
+{
+    m_writer.write(h);
+}
+
+template<typename Traits>
+void BytecodeGeneratorBase<Traits>::write(uint32_t i)
+{
+    m_writer.write(i);
+}
+
+template<typename Traits>
+void BytecodeGeneratorBase<Traits>::write(int8_t b)
+{
+    m_writer.write(static_cast<uint8_t>(b));
+}
+
+template<typename Traits>
+void BytecodeGeneratorBase<Traits>::write(int16_t h)
+{
+    m_writer.write(static_cast<uint16_t>(h));
+}
+
+template<typename Traits>
+void BytecodeGeneratorBase<Traits>::write(int32_t i)
+{
+    m_writer.write(static_cast<uint32_t>(i));
+}
+
+template<typename Traits>
+RegisterID* BytecodeGeneratorBase<Traits>::newRegister()
+{
+    m_calleeLocals.append(virtualRegisterForLocal(m_calleeLocals.size()));
+    int numCalleeLocals = std::max<int>(m_codeBlock->m_numCalleeLocals, m_calleeLocals.size());
+    numCalleeLocals = WTF::roundUpToMultipleOf(stackAlignmentRegisters(), numCalleeLocals);
+    m_codeBlock->m_numCalleeLocals = numCalleeLocals;
+    return &m_calleeLocals.last();
+}
+
+template<typename Traits>
+RegisterID* BytecodeGeneratorBase<Traits>::newTemporary()
+{
+    reclaimFreeRegisters();
+
+    RegisterID* result = newRegister();
+    result->setTemporary();
+    return result;
+}
+
+// Adds an anonymous local var slot. To give this slot a name, add it to symbolTable().
+template<typename Traits>
+RegisterID* BytecodeGeneratorBase<Traits>::addVar()
+{
+    ++m_codeBlock->m_numVars;
+    RegisterID* result = newRegister();
+    ASSERT(VirtualRegister(result->index()).toLocal() == m_codeBlock->m_numVars - 1);
+    result->ref(); // We should never free this slot.
+    return result;
+}
+
+template<typename Traits>
+void BytecodeGeneratorBase<Traits>::allocateCalleeSaveSpace(uint32_t virtualRegisterCountForCalleeSaves)
+{
+    for (size_t i = 0; i < virtualRegisterCountForCalleeSaves; i++)
+        addVar();
+}
+
+} // namespace JSC
index 736768e..9031362 100644 (file)
 
 #pragma once
 
-#include "Instruction.h"
 #include <wtf/Assertions.h>
 #include <wtf/Vector.h>
 #include <limits.h>
 
 namespace JSC {
-    class BytecodeGenerator;
-    class Label;
+    template<typename Traits>
+    class BytecodeGeneratorBase;
+    struct JSGeneratorTraits;
+
+    template<typename Traits>
+    class GenericLabel;
+
+    template<typename Traits>
+    class GenericBoundLabel {
+        using BytecodeGenerator = BytecodeGeneratorBase<Traits>;
+        using Label = GenericLabel<Traits>;
 
-    class BoundLabel {
     public:
-        BoundLabel()
+        GenericBoundLabel()
             : m_type(Offset)
             , m_generator(nullptr)
             , m_target(0)
         { }
 
-        explicit BoundLabel(int target)
+        explicit GenericBoundLabel(int target)
             : m_type(Offset)
             , m_generator(nullptr)
             , m_target(target)
         { }
 
-        BoundLabel(BytecodeGenerator* generator, Label* label)
+        GenericBoundLabel(BytecodeGenerator* generator, Label* label)
             : m_type(GeneratorForward)
             , m_generator(generator)
             , m_label(label)
         { }
 
-        BoundLabel(BytecodeGenerator* generator, int offset)
+        GenericBoundLabel(BytecodeGenerator* generator, int offset)
             : m_type(GeneratorBackward)
             , m_generator(generator)
             , m_target(offset)
         { }
 
-        int target();
-        int saveTarget();
-        int commitTarget();
+        int target()
+        {
+            switch (m_type) {
+            case Offset:
+                return m_target;
+            case GeneratorBackward:
+                return m_target - m_generator->m_writer.position();
+            case GeneratorForward:
+                return 0;
+            default:
+                RELEASE_ASSERT_NOT_REACHED();
+            }
+        }
+
+        int saveTarget()
+        {
+            if (m_type == GeneratorForward) {
+                m_savedTarget = m_generator->m_writer.position();
+                return 0;
+            }
+
+            m_savedTarget = target();
+            return m_savedTarget;
+        }
+
+        int commitTarget()
+        {
+            if (m_type == GeneratorForward) {
+                m_label->m_unresolvedJumps.append(m_savedTarget);
+                return 0;
+            }
+
+            return m_savedTarget;
+        }
 
         operator int() { return target(); }
 
@@ -85,12 +123,16 @@ namespace JSC {
         };
     };
 
-    class Label {
-    WTF_MAKE_NONCOPYABLE(Label);
+    template<typename Traits>
+    class GenericLabel {
+        WTF_MAKE_NONCOPYABLE(GenericLabel);
+        using BytecodeGenerator = BytecodeGeneratorBase<Traits>;
+        using BoundLabel = GenericBoundLabel<Traits>;
+
     public:
-        Label() = default;
+        GenericLabel() = default;
 
-        void setLocation(BytecodeGenerator&, unsigned);
+        void setLocation(BytecodeGenerator&, unsigned location);
 
         BoundLabel bind(BytecodeGenerator* generator)
         {
@@ -129,7 +171,7 @@ namespace JSC {
         bool isBound() const { return m_bound; }
 
     private:
-        friend class BoundLabel;
+        friend BoundLabel;
 
         typedef Vector<int, 8> JumpVector;
 
@@ -141,4 +183,16 @@ namespace JSC {
         mutable JumpVector m_unresolvedJumps;
     };
 
+    using Label = GenericLabel<JSGeneratorTraits>;
+    using BoundLabel = GenericBoundLabel<JSGeneratorTraits>;
+
+    namespace Wasm {
+    struct GeneratorTraits;
+    using Label = GenericLabel<Wasm::GeneratorTraits>;
+    }
+
+    // This cannot be declared in the Wasm namespace as it conflicts with the
+    // ruby Wasm namespace when referencing it from BytecodeList.rb
+    using WasmBoundLabel = GenericBoundLabel<Wasm::GeneratorTraits>;
+
 } // namespace JSC
index 6d68f0f..e0eef49 100644 (file)
@@ -32,7 +32,7 @@
 #include "ArrayConstructor.h"
 #include "BasicBlockLocation.h"
 #include "BuiltinNames.h"
-#include "BytecodeStructs.h"
+#include "BytecodeGenerator.h"
 #include "CallLinkStatus.h"
 #include "CodeBlock.h"
 #include "CodeBlockWithJITType.h"
index b6e588a..1889d99 100644 (file)
@@ -314,6 +314,8 @@ CapabilityLevel capabilityLevel(OpcodeID opcodeID, CodeBlock* codeBlock, const I
     case op_get_by_val_return_location:
     case op_put_by_id_return_location:
     case op_put_by_val_return_location:
+    case wasm_function_prologue:
+    case wasm_function_prologue_no_tls:
         return CannotCompile;
     }
     return CannotCompile;
index 4db4627..2076cd6 100644 (file)
@@ -51,6 +51,7 @@
 #include "JIT.h"
 #include "JITExceptions.h"
 #include "JSArrayInlines.h"
+#include "JSAsyncGenerator.h"
 #include "JSBigInt.h"
 #include "JSCInlines.h"
 #include "JSFixedArray.h"
index 38a2815..6d037b6 100644 (file)
@@ -66,14 +66,14 @@ class Argument
         "#{field_name}(#{Fits::convert(size, "stream[#{index}]", @type)})"
     end
 
-    def setter
+    def setter(traits)
         <<-EOF
     template<typename Functor>
     void set#{capitalized_name}(#{@type.to_s} value, Functor func)
     {
-        if (isWide32())
+        if (isWide32<#{traits}>())
             set#{capitalized_name}<OpcodeSize::Wide32>(value, func);
-        else if (isWide16())
+        else if (isWide16<#{traits}>())
             set#{capitalized_name}<OpcodeSize::Wide16>(value, func);
         else
             set#{capitalized_name}<OpcodeSize::Narrow>(value, func);
index 92c7f94..fcaa792 100644 (file)
 # THE POSSIBILITY OF SUCH DAMAGE.
 
 require_relative 'Assertion'
+require_relative 'GeneratedFile'
 require_relative 'Section'
 require_relative 'Template'
 require_relative 'Type'
-require_relative 'GeneratedFile'
+require_relative 'Wasm'
 
 module DSL
     @sections = []
+    @wasm_section = nil
     @current_section = nil
     @context = binding()
     @namespaces = []
@@ -42,6 +44,10 @@ module DSL
         assert("current section's name is `#{@current_section.name}`, but end_section was called with `#{name}`") { @current_section.name == name }
         @current_section.sort!
         @sections << @current_section
+        if @current_section.is_wasm?
+          assert("Cannot have 2 wasm sections") { @wasm_section.nil? }
+          @wasm_section = @current_section
+        end
         @current_section = nil
     end
 
@@ -85,27 +91,37 @@ module DSL
         @namespaces.pop
     end
 
+    def self.autogenerate_wasm_opcodes()
+        assert("`autogenerate_wasm_opcodes` can only be called in between `begin_section` and `end_section`") { not @current_section.nil? }
+        assert("`autogenerate_wasm_opcodes` can only be called from the `Wasm` section") { @current_section.name == :Wasm }
+        Wasm::autogenerate_opcodes(@context, @wasm_json)
+    end
+
     def self.run(options)
-        bytecodeListPath = options[:bytecodeList]
-        bytecodeList = File.open(bytecodeListPath)
-        @context.eval(bytecodeList.read, bytecodeListPath)
+        bytecode_list_path = options[:bytecode_list]
+        bytecode_list = File.open(bytecode_list_path).read
+
+        @wasm_json = File.open(options[:wasm_json_filename]).read
+
+        @context.eval(bytecode_list, bytecode_list_path)
         assert("must end last section") { @current_section.nil? }
 
-        write_bytecodes(bytecodeList, options[:bytecodesFilename])
-        write_bytecode_structs(bytecodeList, options[:bytecodeStructsFilename])
-        write_init_asm(bytecodeList, options[:initAsmFilename])
-        write_indices(bytecodeList, options[:bytecodeIndicesFilename])
+        write_bytecodes(bytecode_list, options[:bytecodes_filename])
+        write_bytecode_structs(bytecode_list, options[:bytecode_structs_filename])
+        write_bytecodes_init(options[:init_asm_filename], bytecode_list)
+        write_indices(bytecode_list, options[:bytecode_indices_filename])
+        write_llint_generator(options[:wasm_llint_generator_filename], bytecode_list, @wasm_json)
+        write_wasm_init(options[:wasm_init_filename], bytecode_list, @wasm_json)
     end
 
     def self.write_bytecodes(bytecode_list, bytecodes_filename)
         GeneratedFile::create(bytecodes_filename, bytecode_list) do |template|
             template.prefix = "#pragma once\n"
             num_opcodes = @sections.map(&:opcodes).flatten.size
-            template.body = <<-EOF
-#{@sections.map { |s| s.header_helpers(num_opcodes) }.join("\n")}
-#define FOR_EACH_BYTECODE_STRUCT(macro) \\
-#{opcodes_for(:emit_in_structs_file).map { |op| "    macro(#{op.capitalized_name}) \\" }.join("\n")}
-            EOF
+            template.body = [
+                @sections.map { |s| s.header_helpers(num_opcodes) },
+                @sections.select { |s| s.config[:emit_in_structs_file] }.map(&:for_each_struct)
+            ].flatten.join("\n")
         end
     end
 
@@ -118,7 +134,6 @@ module DSL
 
 #include "ArithProfile.h"
 #include "BytecodeDumper.h"
-#include "BytecodeGenerator.h"
 #include "Fits.h"
 #include "GetByIdMetadata.h"
 #include "Instruction.h"
@@ -132,22 +147,35 @@ EOF
 
             template.body = <<-EOF
 #{opcodes.map(&:struct).join("\n")}
-#{Opcode.dump_bytecode(opcodes)}
+#{Opcode.dump_bytecode(:Bytecode, :JSOpcodeTraits, opcodes_filter { |s| s.config[:emit_in_structs_file] && !s.is_wasm? })}
+#{Opcode.dump_bytecode(:Wasm, :WasmOpcodeTraits, opcodes_filter { |s| s.is_wasm? })}
 EOF
             template.suffix = "} // namespace JSC"
         end
     end
 
-    def self.write_init_asm(bytecode_list, init_asm_filename)
-        opcodes = opcodes_for(:emit_in_asm_file)
-
-        GeneratedFile::create(init_asm_filename, bytecode_list) do |template|
+    def self.write_init_asm(opcodes, filename, *dependencies)
+        GeneratedFile::create(filename, *dependencies) do |template|
             template.multiline_comment = nil
             template.line_comment = "#"
             template.body = (opcodes.map.with_index(&:set_entry_address) + opcodes.map.with_index(&:set_entry_address_wide16) + opcodes.map.with_index(&:set_entry_address_wide32)) .join("\n")
         end
     end
 
+    def self.write_bytecodes_init(bytecodes_init_filename, *dependencies)
+        write_init_asm(opcodes_for(:emit_in_asm_file), bytecodes_init_filename, *dependencies)
+    end
+
+    def self.write_wasm_init(wasm_init_filename, *dependencies)
+        write_init_asm(@wasm_section.opcodes, wasm_init_filename, *dependencies)
+    end
+
+    def self.write_llint_generator(generator_filename, *dependencies)
+        GeneratedFile::create(generator_filename, *dependencies) do |template|
+            template.body = Wasm::generate_llint_generator(@wasm_section)
+        end
+    end
+
     def self.write_indices(bytecode_list, indices_filename)
         opcodes = opcodes_for(:emit_in_structs_file)
 
@@ -162,4 +190,9 @@ EOF
         sections = @sections.select { |s| s.config[file] }
         sections.map(&:opcodes).flatten
     end
+
+    def self.opcodes_filter
+        sections = @sections.select { |s| yield s }
+        sections.map(&:opcodes).flatten
+    end
 end
index 160ad1b..78b8b0d 100644 (file)
@@ -56,13 +56,13 @@ module GeneratedFile
         end
     end
 
-    def self.create(filename, dependency)
+    def self.create(filename, *dependencies)
         template = Template.new
         yield template
 
         file = File.open(filename, "w")
-        self.sha1(file, template, dependency)
-        self.license(file, template, dependency)
+        self.sha1(file, template, dependencies)
+        self.license(file, template, dependencies)
 
         unless template.prefix.nil?
             write(file, template.prefix.to_s, "\n")
@@ -75,18 +75,18 @@ module GeneratedFile
         end
     end
 
-    def self.sha1(file, template, dependency)
-        write(file, template.line_comment, " SHA1Hash: ", Digest::SHA1.hexdigest(dependency.read), "\n")
+    def self.sha1(file, template, dependencies)
+      write(file, template.line_comment, " SHA1Hash: ", Digest::SHA1.hexdigest(dependencies.join), "\n")
     end
 
-    def self.license(file, template, dependency)
+    def self.license(file, template, dependencies)
         unless template.multiline_comment.nil?
             write(file, template.multiline_comment[0], "\n")
         end
 
         comment = if template.multiline_comment.nil? then template.line_comment else template.multiline_comment[1] end
         write(file, $LICENSE.strip.split("\n").map { |line| "#{comment} #{line}" }.join("\n"), "\n\n")
-        write(file, comment, " Autogenerated from ", dependency.path, ", do not modify.\n")
+        write(file, comment, " Autogenerated, do not modify.\n")
 
         unless template.multiline_comment.nil?
             write(file, template.multiline_comment[2], "\n")
index 3d25a96..434c4fa 100644 (file)
@@ -29,6 +29,7 @@ class Opcode
     attr_reader :id
     attr_reader :args
     attr_reader :metadata
+    attr_reader :extras
 
     module Size
         Narrow = "OpcodeSize::Narrow"
@@ -44,9 +45,10 @@ class Opcode
         tid
     end
 
-    def initialize(section, name, args, metadata, metadata_initializers)
+    def initialize(section, name, extras, args, metadata, metadata_initializers)
         @section = section
         @name = name
+        @extras = extras || {}
         @metadata = Metadata.new metadata, metadata_initializers
         @args = args.map.with_index { |(arg_name, type), index| Argument.new arg_name, type, index + 1 } unless args.nil?
     end
@@ -87,8 +89,24 @@ class Opcode
         @args.map(&:name).unshift("").join(", ")
     end
 
+    def opcodeIDType
+      @section.is_wasm? ? :WasmOpcodeID : :OpcodeID
+    end
+
+    def wide16
+      @section.is_wasm? ? :wasm_wide16 : :op_wide16
+    end
+
+    def wide32
+      @section.is_wasm? ? :wasm_wide32 : :op_wide32
+    end
+
+    def traits
+      @section.is_wasm? ? "WasmOpcodeTraits" : "JSOpcodeTraits"
+    end
+
     def map_fields_with_size(prefix, size, &block)
-        args = [Argument.new("opcodeID", :OpcodeID, 0)]
+        args = [Argument.new("opcodeID", opcodeIDType, 0)]
         args += @args.dup if @args
         unless @metadata.empty?
             args << @metadata.emitter_local
@@ -111,88 +129,89 @@ EOF
     end
 
     def opcodeID
-        "static constexpr OpcodeID opcodeID = #{name};"
+        "static constexpr #{opcodeIDType} opcodeID = #{name};"
     end
 
     def emitter
-        op_wide16 = Argument.new("op_wide16", :OpcodeID, 0)
-        op_wide32 = Argument.new("op_wide32", :OpcodeID, 0)
+        op_wide16 = Argument.new(wide16, opcodeIDType, 0)
+        op_wide32 = Argument.new(wide32, opcodeIDType, 0)
         metadata_param = @metadata.empty? ? "" : ", #{@metadata.emitter_local.create_param}"
         metadata_arg = @metadata.empty? ? "" : ", #{@metadata.emitter_local.name}"
         <<-EOF.chomp
+    template<typename BytecodeGenerator>
     static void emit(BytecodeGenerator* gen#{typed_args})
     {
-        emitWithSmallestSizeRequirement<OpcodeSize::Narrow>(gen#{untyped_args});
+        emitWithSmallestSizeRequirement<OpcodeSize::Narrow, BytecodeGenerator>(gen#{untyped_args});
     }
 #{%{
-    template<OpcodeSize size, FitsAssertion shouldAssert = Assert>
+    template<OpcodeSize __size, typename BytecodeGenerator, FitsAssertion shouldAssert = Assert>
     static bool emit(BytecodeGenerator* gen#{typed_args})
     {#{@metadata.create_emitter_local}
-        return emit<size, shouldAssert>(gen#{untyped_args}#{metadata_arg});
+        return emit<__size, BytecodeGenerator, shouldAssert>(gen#{untyped_args}#{metadata_arg});
     }
 
-    template<OpcodeSize size>
+    template<OpcodeSize __size, typename BytecodeGenerator>
     static bool checkWithoutMetadataID(BytecodeGenerator* gen#{typed_args})
     {
         decltype(gen->addMetadataFor(opcodeID)) __metadataID { };
-        return checkImpl<size>(gen#{untyped_args}#{metadata_arg});
+        return checkImpl<__size, BytecodeGenerator>(gen#{untyped_args}#{metadata_arg});
     }
 } unless @metadata.empty?}
-    template<OpcodeSize size, FitsAssertion shouldAssert = Assert, bool recordOpcode = true>
+    template<OpcodeSize __size, typename BytecodeGenerator, FitsAssertion shouldAssert = Assert, bool recordOpcode = true>
     static bool emit(BytecodeGenerator* gen#{typed_args}#{metadata_param})
     {
-        bool didEmit = emitImpl<size, recordOpcode>(gen#{untyped_args}#{metadata_arg});
+        bool didEmit = emitImpl<__size, recordOpcode, BytecodeGenerator>(gen#{untyped_args}#{metadata_arg});
         if (shouldAssert == Assert)
             ASSERT(didEmit);
         return didEmit;
     }
 
-    template<OpcodeSize size>
+    template<OpcodeSize __size, typename BytecodeGenerator>
     static void emitWithSmallestSizeRequirement(BytecodeGenerator* gen#{typed_args})
     {
         #{@metadata.create_emitter_local}
-        if (static_cast<unsigned>(size) <= static_cast<unsigned>(OpcodeSize::Narrow)) {
-            if (emit<OpcodeSize::Narrow, NoAssert, true>(gen#{untyped_args}#{metadata_arg}))
+        if (static_cast<unsigned>(__size) <= static_cast<unsigned>(OpcodeSize::Narrow)) {
+            if (emit<OpcodeSize::Narrow, BytecodeGenerator, NoAssert, true>(gen#{untyped_args}#{metadata_arg}))
                 return;
         }
-        if (static_cast<unsigned>(size) <= static_cast<unsigned>(OpcodeSize::Wide16)) {
-            if (emit<OpcodeSize::Wide16, NoAssert, true>(gen#{untyped_args}#{metadata_arg}))
+        if (static_cast<unsigned>(__size) <= static_cast<unsigned>(OpcodeSize::Wide16)) {
+            if (emit<OpcodeSize::Wide16, BytecodeGenerator, NoAssert, true>(gen#{untyped_args}#{metadata_arg}))
                 return;
         }
-        emit<OpcodeSize::Wide32, Assert, true>(gen#{untyped_args}#{metadata_arg});
+        emit<OpcodeSize::Wide32, BytecodeGenerator, Assert, true>(gen#{untyped_args}#{metadata_arg});
     }
 
 private:
-    template<OpcodeSize size>
+    template<OpcodeSize __size, typename BytecodeGenerator>
     static bool checkImpl(BytecodeGenerator* gen#{typed_reference_args}#{metadata_param})
     {
         UNUSED_PARAM(gen);
 #if OS(WINDOWS) && ENABLE(C_LOOP)
         // FIXME: Disable wide16 optimization for Windows CLoop
         // https://bugs.webkit.org/show_bug.cgi?id=198283
-        if (size == OpcodeSize::Wide16)
+        if (__size == OpcodeSize::Wide16)
             return false;
 #endif
-        return #{map_fields_with_size("", "size", &:fits_check).join "\n            && "}
-            && (size == OpcodeSize::Wide16 ? #{op_wide16.fits_check(Size::Narrow)} : true)
-            && (size == OpcodeSize::Wide32 ? #{op_wide32.fits_check(Size::Narrow)} : true);
+        return #{map_fields_with_size("", "__size", &:fits_check).join "\n            && "}
+            && (__size == OpcodeSize::Wide16 ? #{op_wide16.fits_check(Size::Narrow)} : true)
+            && (__size == OpcodeSize::Wide32 ? #{op_wide32.fits_check(Size::Narrow)} : true);
     }
 
-    template<OpcodeSize size, bool recordOpcode>
+    template<OpcodeSize __size, bool recordOpcode, typename BytecodeGenerator>
     static bool emitImpl(BytecodeGenerator* gen#{typed_args}#{metadata_param})
     {
-        if (size == OpcodeSize::Wide16)
+        if (__size == OpcodeSize::Wide16)
             gen->alignWideOpcode16();
-        else if (size == OpcodeSize::Wide32)
+        else if (__size == OpcodeSize::Wide32)
             gen->alignWideOpcode32();
-        if (checkImpl<size>(gen#{untyped_args}#{metadata_arg})) {
+        if (checkImpl<__size>(gen#{untyped_args}#{metadata_arg})) {
             if (recordOpcode)
                 gen->recordOpcode(opcodeID);
-            if (size == OpcodeSize::Wide16)
+            if (__size == OpcodeSize::Wide16)
                 #{op_wide16.fits_write Size::Narrow}
-            else if (size == OpcodeSize::Wide32)
+            else if (__size == OpcodeSize::Wide32)
                 #{op_wide32.fits_write Size::Narrow}
-#{map_fields_with_size("            ", "size", &:fits_write).join "\n"}
+#{map_fields_with_size("            ", "__size", &:fits_write).join "\n"}
             return true;
         }
         return false;
@@ -243,9 +262,9 @@ EOF
 
     static #{capitalized_name} decode(const uint8_t* stream)
     {
-        if (*stream == op_wide32) 
+        if (*stream == #{wide32})
             return { bitwise_cast<const uint32_t*>(stream + 1) };
-        if (*stream == op_wide16) 
+        if (*stream == #{wide16})
             return { bitwise_cast<const uint16_t*>(stream + 1) };
         return { stream };
     }
@@ -253,7 +272,7 @@ EOF
     end
 
     def setters
-        print_args(&:setter)
+        print_args { |a| a.setter(traits) }
     end
 
     def metadata_struct_and_accessor
@@ -297,20 +316,24 @@ EOF
         "#{@section.config[:op_prefix]}#{@name}"
     end
 
+    def unprefixed_name
+      @name
+    end
+
     def length
         1 + (@args.nil? ? 0 : @args.length) + (@metadata.empty? ? 0 : 1)
     end
 
-    def self.dump_bytecode(opcodes)
+    def self.dump_bytecode(name, opcode_traits, opcodes)
         <<-EOF.chomp
-template<typename Block>
-static void dumpBytecode(BytecodeDumper<Block>* dumper, InstructionStream::Offset __location, const Instruction* __instruction)
+template<typename BytecodeDumper>
+static void dump#{name}(BytecodeDumper* dumper, InstructionStream::Offset __location, const Instruction* __instruction)
 {
-    switch (__instruction->opcodeID()) {
+    switch (__instruction->opcodeID<#{opcode_traits}>()) {
 #{opcodes.map { |op|
         <<-EOF.chomp
     case #{op.name}:
-        __instruction->as<#{op.capitalized_name}>().dump(dumper, __location, __instruction->sizeShiftAmount());
+        __instruction->as<#{op.capitalized_name}, #{opcode_traits}>().dump(dumper, __location, __instruction->sizeShiftAmount<#{opcode_traits}>());
         break;
 EOF
     }.join "\n"}
index b79086e..a829220 100644 (file)
 require 'optparse'
 
 $config = {
-    bytecodesFilename: {
+    bytecodes_filename: {
         short: "-b",
         long: "--bytecodes_h FILE",
         desc: "generate bytecodes macro .h FILE",
     },
-    bytecodeStructsFilename: {
+    bytecode_structs_filename: {
         short: "-s",
         long: "--bytecode_structs_h FILE",
         desc: "generate bytecode structs .h FILE",
     },
-    initAsmFilename: {
+    init_asm_filename: {
         short: "-a",
         long: "--init_bytecodes_asm FILE",
         desc: "generate ASM bytecodes init FILE",
     },
-    bytecodeIndicesFilename: {
+    bytecode_indices_filename: {
         short: "-i",
         long: "--bytecode_indices_h FILE",
         desc: "generate indices of bytecode structs .h FILE",
     },
+    wasm_json_filename: {
+        long: "--wasm_json FILE",
+        desc: "Path to JavaScriptCore's wasm.json file",
+    },
+    wasm_llint_generator_filename: {
+        long: "--wasm_llint_generator_h FILE",
+        desc: "Path to generate WasmLLIntGenerator entries from wasm.json file",
+    },
+    wasm_init_filename: {
+        long: "--init_wasm_llint FILE",
+        desc: "generate Wasm bytecodes init FILE",
+    },
 };
 
 module Options
@@ -51,7 +63,7 @@ module Options
         OptionParser.new do |opts|
             opts.banner = "usage: #{opts.program_name} [options] <bytecode-list-file>"
             $config.map do |key, option|
-                opts.on(option[:short], option[:long], option[:desc]) do |v|
+                opts.on(*option.values) do |v|
                     options[key] = v
                 end
             end
@@ -81,7 +93,7 @@ module Options
             exit 1
         end
 
-        options[:bytecodeList] = argv[0]
+        options[:bytecode_list] = argv[0]
         options
     end
 end
index 8cd21db..204ac75 100644 (file)
@@ -43,7 +43,7 @@ class Section
   end
 
   def create_opcode(name, config)
-      Opcode.new(self, name, config[:args], config[:metadata], config[:metadata_initializers])
+      Opcode.new(self, name, config[:extras], config[:args], config[:metadata], config[:metadata_initializers])
   end
 
   def add_opcode_group(name, opcodes, config)
@@ -57,6 +57,10 @@ class Section
       @opcodes.each(&:create_id!)
   end
 
+  def is_wasm?
+    @name == :Wasm
+  end
+
   def header_helpers(num_opcodes)
       out = StringIO.new
       if config[:emit_in_h_file]
@@ -108,4 +112,14 @@ class Section
       end
       out.string
   end
+
+    def for_each_struct
+        <<-EOF
+#define FOR_EACH_#{config[:macro_name_component]}_STRUCT(macro) \\
+#{opcodes.map do |op|
+    "    macro(#{op.capitalized_name}) \\"
+end.join("\n")}
+EOF
+    end
+
 end
diff --git a/Source/JavaScriptCore/generator/Wasm.rb b/Source/JavaScriptCore/generator/Wasm.rb
new file mode 100644 (file)
index 0000000..24f27e0
--- /dev/null
@@ -0,0 +1,83 @@
+require 'json'
+
+module Wasm
+    def self.normalize(name)
+        name.gsub(/(\.|\/)/, "_")
+    end
+
+    def self.autogenerate_opcodes(context, wasm_json)
+        JSON.parse(wasm_json)["opcode"].each do |name, value|
+            category = value["category"]
+
+            next unless ["arithmetic", "comparison", "conversion"].include? category
+
+            returnCount = value["return"].size
+            parameterCount = value["parameter"].size
+
+            assert("should return 0 or 1 values") { [0, 1].include? returnCount }
+            assert("should only have 1 or 2 parameters") { [1, 2].include? parameterCount }
+
+            name = normalize(name)
+            arguments = {}
+
+            virtualRegister = context.eval("VirtualRegister")
+
+            if returnCount > 0
+                arguments[:dst] = virtualRegister
+            end
+
+            case parameterCount
+            when 1
+                arguments[:operand] = virtualRegister
+            when 2
+                arguments[:lhs] = virtualRegister
+                arguments[:rhs] = virtualRegister
+            end
+
+            context.eval("Proc.new { |arguments, extras | op('#{name}', { extras: extras, args: arguments }) }").call(arguments, value)
+        end
+    end
+
+    def self.generate_llint_generator(section)
+        opcodes = section.opcodes.select { |op| ["arithmetic", "comparison", "conversion"].include? op.extras["category"] }
+        methods = opcodes.map do |op|
+            case op.args.size
+            when 2
+                generate_unary_op(op)
+            when 3
+                generate_binary_op(op)
+            else
+                assert("Invalid argument count #{op.args.size} for op #{op.name}") { false }
+            end
+        end
+        methods.join("\n")
+    end
+
+    def self.generate_binary_op(op)
+        <<-EOF
+template<>
+auto LLIntGenerator::addOp<#{op_type(op)}>(ExpressionType lhs, ExpressionType rhs, ExpressionType& result) -> PartialResult
+{
+    result = newTemporary();
+    #{op.capitalized_name}::emit(this, result, lhs, rhs);
+    return { };
+}
+        EOF
+    end
+
+    def self.generate_unary_op(op)
+        <<-EOF
+template<>
+auto LLIntGenerator::addOp<#{op_type(op)}>(ExpressionType operand, ExpressionType& result) -> PartialResult
+{
+    result = newTemporary();
+    #{op.capitalized_name}::emit(this, result, operand);
+    return { };
+}
+        EOF
+    end
+
+    def self.op_type(op)
+        "OpType::#{op.unprefixed_name.gsub(/^.|[^a-z0-9]./) { |c| c[-1].upcase }}"
+    end
+end
index 43adc91..468289d 100644 (file)
@@ -46,7 +46,6 @@ namespace JSC {
         Register();
 
         Register(const JSValue&);
-        Register& operator=(const JSValue&);
         JSValue jsValue() const;
         JSValue asanUnsafeJSValue() const;
         EncodedJSValue encodedJSValue() const;
@@ -55,6 +54,7 @@ namespace JSC {
         ALWAYS_INLINE Register& operator=(CodeBlock*);
         ALWAYS_INLINE Register& operator=(JSScope*);
         ALWAYS_INLINE Register& operator=(JSObject*);
+        ALWAYS_INLINE Register& operator=(EncodedJSValue);
 
         int32_t i() const;
         ALWAYS_INLINE CallFrame* callFrame() const;
@@ -112,12 +112,6 @@ namespace JSC {
         u.value = JSValue::encode(v);
     }
 
-    ALWAYS_INLINE Register& Register::operator=(const JSValue& v)
-    {
-        u.value = JSValue::encode(v);
-        return *this;
-    }
-
     // FIXME (rdar://problem/19379214): ASan only needs to be suppressed for Register::jsValue() when called from prepareOSREntry(), but there is currently no way to express this short of adding a separate copy of the function.
     SUPPRESS_ASAN ALWAYS_INLINE JSValue Register::asanUnsafeJSValue() const
     {
index f4e4d0c..c56ef5e 100644 (file)
@@ -75,6 +75,12 @@ ALWAYS_INLINE Register& Register::operator=(JSScope* scope)
     return *this;
 }
 
+ALWAYS_INLINE Register& Register::operator=(EncodedJSValue encodedJSValue)
+{
+    u.value = encodedJSValue;
+    return *this;
+}
+
 ALWAYS_INLINE JSScope* Register::scope() const
 {
     return jsCast<JSScope*>(unboxedCell());
index 5fcf889..ac372c9 100644 (file)
@@ -29,6 +29,7 @@
 #include "JIT.h"
 
 #include "ArithProfile.h"
+#include "BytecodeGenerator.h"
 #include "CodeBlock.h"
 #include "JITAddGenerator.h"
 #include "JITBitAndGenerator.h"
index 0fc4069..0d04994 100644 (file)
@@ -29,7 +29,7 @@
 #include "JIT.h"
 
 #include "BasicBlockLocation.h"
-#include "BytecodeStructs.h"
+#include "BytecodeGenerator.h"
 #include "Exception.h"
 #include "Heap.h"
 #include "InterpreterInlines.h"
index bda308c..51ed653 100644 (file)
@@ -48,13 +48,19 @@ namespace LLInt {
 
 
 uint8_t Data::s_exceptionInstructions[maxOpcodeLength + 1] = { };
-Opcode g_opcodeMap[numOpcodeIDs] = { };
-Opcode g_opcodeMapWide16[numOpcodeIDs] = { };
-Opcode g_opcodeMapWide32[numOpcodeIDs] = { };
+uint8_t Data::s_wasmExceptionInstructions[maxOpcodeLength + 1] = { };
+Opcode g_opcodeMap[numOpcodeIDs + numWasmOpcodeIDs] = { };
+Opcode g_opcodeMapWide16[numOpcodeIDs + numWasmOpcodeIDs] = { };
+Opcode g_opcodeMapWide32[numOpcodeIDs + numWasmOpcodeIDs] = { };
 
 #if !ENABLE(C_LOOP)
 extern "C" void llint_entry(void*, void*, void*);
-#endif
+
+#if ENABLE(WEBASSEMBLY)
+extern "C" void wasm_entry(void*, void*, void*);
+#endif // ENABLE(WEBASSEMBLY)
+
+#endif // !ENABLE(C_LOOP)
 
 void initialize()
 {
@@ -64,15 +70,21 @@ void initialize()
 #else // !ENABLE(C_LOOP)
     llint_entry(&g_opcodeMap, &g_opcodeMapWide16, &g_opcodeMapWide32);
 
-    for (int i = 0; i < numOpcodeIDs; ++i) {
+#if ENABLE(WEBASSEMBLY)
+    wasm_entry(&g_opcodeMap[numOpcodeIDs], &g_opcodeMapWide16[numOpcodeIDs], &g_opcodeMapWide32[numOpcodeIDs]);
+#endif // ENABLE(WEBASSEMBLY)
+
+    for (int i = 0; i < numOpcodeIDs + numWasmOpcodeIDs; ++i) {
         g_opcodeMap[i] = tagCodePtr(g_opcodeMap[i], BytecodePtrTag);
         g_opcodeMapWide16[i] = tagCodePtr(g_opcodeMapWide16[i], BytecodePtrTag);
         g_opcodeMapWide32[i] = tagCodePtr(g_opcodeMapWide32[i], BytecodePtrTag);
     }
 
     ASSERT(llint_throw_from_slow_path_trampoline < UINT8_MAX);
-    for (int i = 0; i < maxOpcodeLength + 1; ++i)
+    for (int i = 0; i < maxOpcodeLength + 1; ++i) {
         Data::s_exceptionInstructions[i] = llint_throw_from_slow_path_trampoline;
+        Data::s_wasmExceptionInstructions[i] = wasm_throw_from_slow_path_trampoline;
+    }
 #endif // ENABLE(C_LOOP)
 }
 
index 8cc1152..ae8dee2 100644 (file)
@@ -43,9 +43,9 @@ typedef void (*LLIntCode)();
 
 namespace LLInt {
 
-extern "C" JS_EXPORT_PRIVATE Opcode g_opcodeMap[numOpcodeIDs];
-extern "C" JS_EXPORT_PRIVATE Opcode g_opcodeMapWide16[numOpcodeIDs];
-extern "C" JS_EXPORT_PRIVATE Opcode g_opcodeMapWide32[numOpcodeIDs];
+extern "C" JS_EXPORT_PRIVATE Opcode g_opcodeMap[numOpcodeIDs + numWasmOpcodeIDs];
+extern "C" JS_EXPORT_PRIVATE Opcode g_opcodeMapWide16[numOpcodeIDs + numWasmOpcodeIDs];
+extern "C" JS_EXPORT_PRIVATE Opcode g_opcodeMapWide32[numOpcodeIDs + numWasmOpcodeIDs];
 
 class Data {
 
@@ -54,10 +54,12 @@ public:
 
 private:
     static uint8_t s_exceptionInstructions[maxOpcodeLength + 1];
+    static uint8_t s_wasmExceptionInstructions[maxOpcodeLength + 1];
 
     friend void initialize();
 
     friend Instruction* exceptionInstructions();
+    friend Instruction* wasmExceptionInstructions();
     friend Opcode* opcodeMap();
     friend Opcode* opcodeMapWide16();
     friend Opcode* opcodeMapWide32();
@@ -77,6 +79,11 @@ inline Instruction* exceptionInstructions()
     return reinterpret_cast<Instruction*>(Data::s_exceptionInstructions);
 }
     
+inline Instruction* wasmExceptionInstructions()
+{
+    return bitwise_cast<Instruction*>(Data::s_wasmExceptionInstructions);
+}
+
 inline Opcode* opcodeMap()
 {
     return g_opcodeMap;
index 8104e97..aa3ec34 100644 (file)
 #endif
 
 #define OFFLINE_ASM_GIGACAGE_ENABLED GIGACAGE_ENABLED
+
+#if ENABLE(WEBASSEMBLY)
+#define OFFLINE_ASM_WEBASSEMBLY 1
+#else
+#define OFFLINE_ASM_WEBASSEMBLY 0
+#endif
+
+#if HAVE(FAST_TLS)
+#define OFFLINE_ASM_HAVE_FAST_TLS 1
+#else
+#define OFFLINE_ASM_HAVE_FAST_TLS 0
+#endif
index a25f984..78add02 100644 (file)
 #include "TypeProfilerLog.h"
 #include "VM.h"
 #include "ValueProfile.h"
+#include "WasmCallingConvention.h"
+#include "WasmFunctionCodeBlock.h"
+#include "WasmInstance.h"
 #include "Watchdog.h"
+#include "WebAssemblyFunction.h"
 #include <stdio.h>
+#include <wtf/FastTLS.h>
 #include <wtf/text/StringImpl.h>
 
 namespace JSC {
index 4421121..7bd7e1f 100644 (file)
@@ -27,6 +27,7 @@
 #include "LLIntSlowPaths.h"
 
 #include "ArrayConstructor.h"
+#include "BytecodeGenerator.h"
 #include "CallFrame.h"
 #include "CommonSlowPaths.h"
 #include "Error.h"
index c73af62..8e11b82 100644 (file)
@@ -39,6 +39,8 @@
 #include "ProtoCallFrame.h"
 #include "StackAlignment.h"
 #include "VM.h"
+#include "WasmCallingConvention.h"
+#include "WasmContextInlines.h"
 #include <wtf/NeverDestroyed.h>
 
 namespace JSC {
@@ -60,8 +62,13 @@ static MacroAssemblerCodeRef<JITThunkPtrTag> generateThunkWithJumpTo(OpcodeID op
     LLIntCode target = LLInt::getCodeFunctionPtr<JSEntryPtrTag>(opcodeID);
     assertIsTaggedWith(target, JSEntryPtrTag);
 
-    jit.move(JSInterfaceJIT::TrustedImmPtr(target), JSInterfaceJIT::regT0);
-    jit.farJump(JSInterfaceJIT::regT0, JSEntryPtrTag);
+#if ENABLE(WEBASSEMBLY)
+    CCallHelpers::RegisterID scratch = Wasm::wasmCallingConvention().prologueScratchGPRs[0];
+#else
+    CCallHelpers::RegisterID scratch = JSInterfaceJIT::regT0;
+#endif
+    jit.move(JSInterfaceJIT::TrustedImmPtr(target), scratch);
+    jit.farJump(scratch, JSEntryPtrTag);
 
     LinkBuffer patchBuffer(jit, GLOBAL_THUNK_ID);
     return FINALIZE_CODE(patchBuffer, JITThunkPtrTag, "LLInt %s prologue thunk", thunkKind);
@@ -137,6 +144,21 @@ MacroAssemblerCodeRef<JITThunkPtrTag> moduleProgramEntryThunk()
     return codeRef;
 }
 
+#if ENABLE(WEBASSEMBLY)
+MacroAssemblerCodeRef<JITThunkPtrTag> wasmFunctionEntryThunk()
+{
+    static LazyNeverDestroyed<MacroAssemblerCodeRef<JITThunkPtrTag>> codeRef;
+    static std::once_flag onceKey;
+    std::call_once(onceKey, [&] {
+        if (Wasm::Context::useFastTLS())
+            codeRef.construct(generateThunkWithJumpTo(wasm_function_prologue, "function for call"));
+        else
+            codeRef.construct(generateThunkWithJumpTo(wasm_function_prologue_no_tls, "function for call"));
+    });
+    return codeRef;
+}
+#endif // ENABLE(WEBASSEMBLY)
+
 } // namespace LLInt
 
 #endif
index e2293df..fcaf80d 100644 (file)
@@ -54,4 +54,8 @@ MacroAssemblerCodeRef<JITThunkPtrTag> evalEntryThunk();
 MacroAssemblerCodeRef<JITThunkPtrTag> programEntryThunk();
 MacroAssemblerCodeRef<JITThunkPtrTag> moduleProgramEntryThunk();
 
+#if ENABLE(WEBASSEMBLY)
+MacroAssemblerCodeRef<JITThunkPtrTag> wasmFunctionEntryThunk();
+#endif // ENABLE(WEBASSEMBLY)
+
 } } // namespace JSC::LLInt
index 0135877..b350a06 100644 (file)
@@ -1,4 +1,4 @@
-# Copyrsght (C) 2011-2019 Apple Inc. All rights reserved.
+# Copyright (C) 2011-2019 Apple Inc. All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without
 # modification, are permitted provided that the following conditions
@@ -184,6 +184,7 @@ const CallFrameHeaderSize = ThisArgumentOffset
 
 const MetadataOffsetTable16Offset = 0
 const MetadataOffsetTable32Offset = constexpr UnlinkedMetadataTable::s_offset16TableSize
+const NumberOfJSOpcodeIDs = constexpr numOpcodeIDs
 
 # Some value representation constants.
 if JSVALUE64
@@ -312,7 +313,7 @@ macro dispatchIndirect(offsetReg)
     dispatch(offsetReg)
 end
 
-macro dispatchOp(size, opcodeName)
+macro genericDispatchOp(dispatch, size, opcodeName)
     macro dispatchNarrow()
         dispatch(constexpr %opcodeName%_length)
     end
@@ -328,6 +329,11 @@ macro dispatchOp(size, opcodeName)
     size(dispatchNarrow, dispatchWide16, dispatchWide32, macro (dispatch) dispatch() end)
 end
 
+macro dispatchOp(size, opcodeName)
+    genericDispatchOp(dispatch, size, opcodeName)
+end
+
+
 macro getu(size, opcodeStruct, fieldName, dst)
     size(getuOperandNarrow, getuOperandWide16, getuOperandWide32, macro (getu)
         getu(opcodeStruct, fieldName, dst)
@@ -363,7 +369,7 @@ macro metadata(size, opcode, dst, scratch)
     addp metadataTable, dst # return &metadataTable[offset]
 end
 
-macro jumpImpl(targetOffsetReg)
+macro jumpImpl(dispatchIndirect, targetOffsetReg)
     btiz targetOffsetReg, .outOfLineJumpTarget
     dispatchIndirect(targetOffsetReg)
 .outOfLineJumpTarget:
@@ -375,6 +381,10 @@ macro commonOp(label, prologue, fn)
 _%label%:
     prologue()
     fn(narrow)
+    if ASSERT_ENABLED
+        break
+        break
+    end
 
 # FIXME: We cannot enable wide16 bytecode in Windows CLoop. With MSVC, as CLoop::execute gets larger code
 # size, CLoop::execute gets higher stack height requirement. This makes CLoop::execute takes 160KB stack
@@ -384,11 +394,19 @@ if not C_LOOP_WIN
 _%label%_wide16:
     prologue()
     fn(wide16)
+    if ASSERT_ENABLED
+        break
+        break
+    end
 end
 
 _%label%_wide32:
     prologue()
     fn(wide32)
+    if ASSERT_ENABLED
+        break
+        break
+    end
 end
 
 macro op(l, fn)
@@ -432,7 +450,7 @@ macro llintOpWithJump(opcodeName, opcodeStruct, impl)
     llintOpWithMetadata(opcodeName, opcodeStruct, macro(size, get, dispatch, metadata, return)
         macro jump(fieldName)
             get(fieldName, t0)
-            jumpImpl(t0)
+            jumpImpl(dispatchIndirect, t0)
         end
 
         impl(size, get, jump, dispatch)
@@ -564,6 +582,14 @@ macro assert(assertion)
     end
 end
 
+macro assert_with(assertion, crash)
+    if ASSERT_ENABLED
+        assertion(.ok)
+        crash()
+    .ok:
+    end
+end
+
 # The probe macro can be used to insert some debugging code without perturbing scalar
 # registers. Presently, the probe macro only preserves scalar registers. Hence, the
 # C probe callback function should not trash floating point registers.
@@ -781,50 +807,56 @@ macro restoreCalleeSavesUsedByLLInt()
     end
 end
 
-macro copyCalleeSavesToVMEntryFrameCalleeSavesBuffer(vm, temp)
+macro copyCalleeSavesToEntryFrameCalleeSavesBuffer(entryFrame)
     if ARM64 or ARM64E or X86_64 or X86_64_WIN or ARMv7 or MIPS
-        loadp VM::topEntryFrame[vm], temp
-        vmEntryRecord(temp, temp)
-        leap VMEntryRecord::calleeSaveRegistersBuffer[temp], temp
+        vmEntryRecord(entryFrame, entryFrame)
+        leap VMEntryRecord::calleeSaveRegistersBuffer[entryFrame], entryFrame
         if ARM64 or ARM64E
-            storeq csr0, [temp]
-            storeq csr1, 8[temp]
-            storeq csr2, 16[temp]
-            storeq csr3, 24[temp]
-            storeq csr4, 32[temp]
-            storeq csr5, 40[temp]
-            storeq csr6, 48[temp]
-            storeq csr7, 56[temp]
-            storeq csr8, 64[temp]
-            storeq csr9, 72[temp]
-            stored csfr0, 80[temp]
-            stored csfr1, 88[temp]
-            stored csfr2, 96[temp]
-            stored csfr3, 104[temp]
-            stored csfr4, 112[temp]
-            stored csfr5, 120[temp]
-            stored csfr6, 128[temp]
-            stored csfr7, 136[temp]
+            storeq csr0, [entryFrame]
+            storeq csr1, 8[entryFrame]
+            storeq csr2, 16[entryFrame]
+            storeq csr3, 24[entryFrame]
+            storeq csr4, 32[entryFrame]
+            storeq csr5, 40[entryFrame]
+            storeq csr6, 48[entryFrame]
+            storeq csr7, 56[entryFrame]
+            storeq csr8, 64[entryFrame]
+            storeq csr9, 72[entryFrame]
+            stored csfr0, 80[entryFrame]
+            stored csfr1, 88[entryFrame]
+            stored csfr2, 96[entryFrame]
+            stored csfr3, 104[entryFrame]
+            stored csfr4, 112[entryFrame]
+            stored csfr5, 120[entryFrame]
+            stored csfr6, 128[entryFrame]
+            stored csfr7, 136[entryFrame]
         elsif X86_64
-            storeq csr0, [temp]
-            storeq csr1, 8[temp]
-            storeq csr2, 16[temp]
-            storeq csr3, 24[temp]
-            storeq csr4, 32[temp]
+            storeq csr0, [entryFrame]
+            storeq csr1, 8[entryFrame]
+            storeq csr2, 16[entryFrame]
+            storeq csr3, 24[entryFrame]
+            storeq csr4, 32[entryFrame]
         elsif X86_64_WIN
-            storeq csr0, [temp]
-            storeq csr1, 8[temp]
-            storeq csr2, 16[temp]
-            storeq csr3, 24[temp]
-            storeq csr4, 32[temp]
-            storeq csr5, 40[temp]
-            storeq csr6, 48[temp]
+            storeq csr0, [entryFrame]
+            storeq csr1, 8[entryFrame]
+            storeq csr2, 16[entryFrame]
+            storeq csr3, 24[entryFrame]
+            storeq csr4, 32[entryFrame]
+            storeq csr5, 40[entryFrame]
+            storeq csr6, 48[entryFrame]
         elsif ARMv7 or MIPS
-            storep csr0, [temp]
+            storep csr0, [entryFrame]
         end
     end
 end
 
+macro copyCalleeSavesToVMEntryFrameCalleeSavesBuffer(vm, temp)
+    if ARM64 or ARM64E or X86_64 or X86_64_WIN or ARMv7 or MIPS
+        loadp VM::topEntryFrame[vm], temp
+        copyCalleeSavesToEntryFrameCalleeSavesBuffer(temp)
+    end
+end
+
 macro restoreCalleeSavesFromVMEntryFrameCalleeSavesBuffer(vm, temp)
     if ARM64 or ARM64E or X86_64 or X86_64_WIN or ARMv7 or MIPS
         loadp VM::topEntryFrame[vm], temp
@@ -1155,6 +1187,7 @@ macro prologue(codeBlockGetter, codeBlockSetter, osrSlowPath, traceSlowPath)
         addp maxFrameExtentForSlowPathCall, sp
     end
     codeBlockGetter(t1)
+    codeBlockSetter(t1)
     if not (C_LOOP or C_LOOP_WIN)
         baddis 5, CodeBlock::m_llintExecuteCounter + BaselineExecutionCounter::m_counter[t1], .continue
         if JSVALUE64
@@ -1184,12 +1217,10 @@ macro prologue(codeBlockGetter, codeBlockSetter, osrSlowPath, traceSlowPath)
         end
         jmp r0, JSEntryPtrTag
     .recover:
-        codeBlockGetter(t1)
+        notFunctionCodeBlockGetter(t1)
     .continue:
     end
 
-    codeBlockSetter(t1)
-
     preserveCalleeSavesUsedByLLInt()
 
     # Set up the PC.
@@ -1229,7 +1260,7 @@ macro prologue(codeBlockGetter, codeBlockSetter, osrSlowPath, traceSlowPath)
 .stackHeightOKGetCodeBlock:
     # Stack check slow path returned that the stack was ok.
     # Since they were clobbered, need to get CodeBlock and new sp
-    codeBlockGetter(t1)
+    notFunctionCodeBlockGetter(t1)
     getFrameRegisterSizeForCodeBlock(t1, t0)
     subp cfr, t0, t0
 
@@ -1371,90 +1402,103 @@ if C_LOOP or C_LOOP_WIN
     _llint_entry:
         crash()
 else
-    macro initPCRelative(pcBase)
+    macro initPCRelative(kind, pcBase)
         if X86_64 or X86_64_WIN or X86 or X86_WIN
-            call _relativePCBase
-        _relativePCBase:
+            call _%kind%_relativePCBase
+        _%kind%_relativePCBase:
             pop pcBase
         elsif ARM64 or ARM64E
         elsif ARMv7
-        _relativePCBase:
+        _%kind%_relativePCBase:
             move pc, pcBase
             subp 3, pcBase   # Need to back up the PC and set the Thumb2 bit
         elsif MIPS
-            la _relativePCBase, pcBase
+            la _%kind%_relativePCBase, pcBase
             setcallreg pcBase # needed to set $t9 to the right value for the .cpload created by the label.
-        _relativePCBase:
+        _%kind%_relativePCBase:
         end
-end
+    end
 
-# The PC base is in t3, as this is what _llint_entry leaves behind through
-# initPCRelative(t3)
-macro setEntryAddress(index, label)
-    setEntryAddressCommon(index, label, a0)
-end
+    # The PC base is in t3, as this is what _llint_entry leaves behind through
+    # initPCRelative(t3)
+    macro setEntryAddressCommon(kind, index, label, map)
+        if X86_64
+            leap (label - _%kind%_relativePCBase)[t3], t4
+            move index, t5
+            storep t4, [map, t5, 8]
+        elsif X86_64_WIN
+            leap (label - _%kind%_relativePCBase)[t3], t4
+            move index, t0
+            storep t4, [map, t0, 8]
+        elsif X86 or X86_WIN
+            leap (label - _%kind%_relativePCBase)[t3], t4
+            move index, t5
+            storep t4, [map, t5, 4]
+        elsif ARM64 or ARM64E
+            pcrtoaddr label, t3
+            move index, t4
+            storep t3, [map, t4, PtrSize]
+        elsif ARMv7
+            mvlbl (label - _%kind%_relativePCBase), t4
+            addp t4, t3, t4
+            move index, t5
+            storep t4, [map, t5, 4]
+        elsif MIPS
+            la label, t4
+            la _%kind%_relativePCBase, t3
+            subp t3, t4
+            addp t4, t3, t4
+            move index, t5
+            storep t4, [map, t5, 4]
+        end
+    end
 
-macro setEntryAddressWide16(index, label)
-     setEntryAddressCommon(index, label, a1)
-end
 
-macro setEntryAddressWide32(index, label)
-     setEntryAddressCommon(index, label, a2)
-end
 
-macro setEntryAddressCommon(index, label, map)
-    if X86_64
-        leap (label - _relativePCBase)[t3], t4
-        move index, t5
-        storep t4, [map, t5, 8]
-    elsif X86_64_WIN
-        leap (label - _relativePCBase)[t3], t4
-        move index, t0
-        storep t4, [map, t0, 8]
-    elsif X86 or X86_WIN
-        leap (label - _relativePCBase)[t3], t4
-        move index, t5
-        storep t4, [map, t5, 4]
-    elsif ARM64 or ARM64E
-        pcrtoaddr label, t3
-        move index, t4
-        storep t3, [map, t4, PtrSize]
-    elsif ARMv7
-        mvlbl (label - _relativePCBase), t4
-        addp t4, t3, t4
-        move index, t5
-        storep t4, [map, t5, 4]
-    elsif MIPS
-        la label, t4
-        la _relativePCBase, t3
-        subp t3, t4
-        addp t4, t3, t4
-        move index, t5
-        storep t4, [map, t5, 4]
-    end
-end
+    macro includeEntriesAtOffset(kind, fn)
+        macro setEntryAddress(index, label)
+            setEntryAddressCommon(kind, index, label, a0)
+        end
 
-global _llint_entry
-# Entry point for the llint to initialize.
-_llint_entry:
-    functionPrologue()
-    pushCalleeSaves()
-    if X86 or X86_WIN
-        loadp 20[sp], a0
-        loadp 24[sp], a1
-        loadp 28[sp], a2
+        macro setEntryAddressWide16(index, label)
+             setEntryAddressCommon(kind, index, label, a1)
+        end
+
+        macro setEntryAddressWide32(index, label)
+             setEntryAddressCommon(kind, index, label, a2)
+        end
+
+        fn()
     end
 
-    initPCRelative(t3)
 
-    # Include generated bytecode initialization file.
-    include InitBytecodes
+macro entry(kind, initialize)
+    global _%kind%_entry
+    _%kind%_entry:
+        functionPrologue()
+        pushCalleeSaves()
+        if X86 or X86_WIN
+            loadp 20[sp], a0
+            loadp 24[sp], a1
+            loadp 28[sp], a2
+        end
 
-    popCalleeSaves()
-    functionEpilogue()
-    ret
+        initPCRelative(kind, t3)
+
+        # Include generated bytecode initialization file.
+        includeEntriesAtOffset(kind, initialize)
+        popCalleeSaves()
+        functionEpilogue()
+        ret
 end
 
+# Entry point for the llint to initialize.
+entry(llint, macro()
+    include InitBytecodes
+end)
+
+end // not (C_LOOP or C_LOOP_WIN)
+
 _llint_op_wide16:
     nextInstructionWide16()
 
@@ -1462,16 +1506,16 @@ _llint_op_wide32:
     nextInstructionWide32()
 
 macro noWide(label)
-_llint_%label%_wide16:
+_%label%_wide16:
     crash()
 
-_llint_%label%_wide32:
+_%label%_wide32:
     crash()
 end
 
-noWide(op_wide16)
-noWide(op_wide32)
-noWide(op_enter)
+noWide(llint_op_wide16)
+noWide(llint_op_wide32)
+noWide(llint_op_enter)
 
 op(llint_program_prologue, macro ()
     prologue(notFunctionCodeBlockGetter, notFunctionCodeBlockSetter, _llint_entry_osr, _llint_trace_prologue)
@@ -1970,3 +2014,29 @@ macro notSupported()
         break
     end
 end
+
+if WEBASSEMBLY
+
+entry(wasm, macro()
+    include InitWasm
+end)
+
+macro wasmScope()
+    # Wrap the script in a macro since it overwrites some of the LLInt macros,
+    # but we don't want to interfere with the LLInt opcodes
+    include WebAssembly
+end
+wasmScope()
+
+else
+
+# These need to be defined even when WebAssembly is disabled
+op(wasm_function_prologue, macro ()
+    crash()
+end)
+
+op(wasm_function_prologue_no_tls, macro ()
+    crash()
+end)
+
+end
index bc230e5..0cb7ea8 100644 (file)
@@ -1720,7 +1720,7 @@ llintOpWithMetadata(op_jneq_ptr, OpJneqPtr, macro (size, get, dispatch, metadata
     metadata(t5, t2)
     storeb 1, OpJneqPtr::Metadata::m_hasJumped[t5]
     get(m_targetLabel, t0)
-    jumpImpl(t0)
+    jumpImpl(dispatchIndirect, t0)
 .opJneqPtrFallThrough:
     dispatch()
 end)
index f2ddbb2..0443f0c 100644 (file)
@@ -387,7 +387,7 @@ macro traceValue(fromWhere, operand)
     restoreStateAfterCCall()
 end
 
-# Call a slow path for call call opcodes.
+# Call a slow path for call opcodes.
 macro callCallSlowPath(slowPath, action)
     storei PC, ArgumentCount + TagOffset[cfr]
     prepareStateForCCall()
@@ -1019,7 +1019,7 @@ macro binaryOpCustomStore(opcodeName, opcodeStruct, integerOperationAndStore, do
         jmp .op1NotIntReady
     .op1NotIntOp2Int:
         profile(ArithProfileNumberInt)
-        ci2d t1, ft1
+        ci2ds t1, ft1
     .op1NotIntReady:
         get(m_dst, t2)
         addq numberTag, t0
@@ -1035,7 +1035,7 @@ macro binaryOpCustomStore(opcodeName, opcodeStruct, integerOperationAndStore, do
         get(m_dst, t2)
         btqz t1, numberTag, .slow
         profile(ArithProfileIntNumber)
-        ci2d t0, ft0
+        ci2ds t0, ft0
         addq numberTag, t1
         fq2d t1, ft1
         doubleOperation(ft1, ft0)
@@ -1678,7 +1678,7 @@ macro putByValOp(opcodeName, opcodeStruct, osrExitPoint)
             macro (operand, scratch, address)
                 loadConstantOrVariable(size, operand, scratch)
                 bqb scratch, numberTag, .notInt
-                ci2d scratch, ft0
+                ci2ds scratch, ft0
                 jmp .ready
             .notInt:
                 addq numberTag, scratch
@@ -1827,7 +1827,7 @@ llintOpWithMetadata(op_jneq_ptr, OpJneqPtr, macro (size, get, dispatch, metadata
     metadata(t5, t0)
     storeb 1, OpJneqPtr::Metadata::m_hasJumped[t5]
     get(m_targetLabel, t0)
-    jumpImpl(t0)
+    jumpImpl(dispatchIndirect, t0)
 end)
 
 
@@ -1845,7 +1845,7 @@ macro compareJumpOp(opcodeName, opcodeStruct, integerCompare, doubleCompare)
     .op1NotInt:
         btqz t0, numberTag, .slow
         bqb t1, numberTag, .op1NotIntOp2NotInt
-        ci2d t1, ft1
+        ci2ds t1, ft1
         jmp .op1NotIntReady
     .op1NotIntOp2NotInt:
         btqz t1, numberTag, .slow
@@ -1858,7 +1858,7 @@ macro compareJumpOp(opcodeName, opcodeStruct, integerCompare, doubleCompare)
         dispatch()
 
     .op2NotInt:
-        ci2d t0, ft0
+        ci2ds t0, ft0
         btqz t1, numberTag, .slow
         addq numberTag, t1
         fq2d t1, ft1
diff --git a/Source/JavaScriptCore/llint/WebAssembly.asm b/Source/JavaScriptCore/llint/WebAssembly.asm
new file mode 100644 (file)
index 0000000..2e69413
--- /dev/null
@@ -0,0 +1,2000 @@
+# Copyright (C) 2019 Apple Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright
+#    notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+#    notice, this list of conditions and the following disclaimer in the
+#    documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+# THE POSSIBILITY OF SUCH DAMAGE.
+
+# Calling conventions
+const CalleeSaveSpaceAsVirtualRegisters = constexpr Wasm::numberOfLLIntCalleeSaveRegisters
+const CalleeSaveSpaceStackAligned = (CalleeSaveSpaceAsVirtualRegisters * SlotSize + StackAlignment - 1) & ~StackAlignmentMask
+const WasmEntryPtrTag = constexpr WasmEntryPtrTag
+
+if HAVE_FAST_TLS
+    const WTF_WASM_CONTEXT_KEY = constexpr WTF_WASM_CONTEXT_KEY
+end
+
+if X86_64
+    const NumberOfWasmArgumentGPRs = 6
+elsif ARM64 or ARM64E
+    const NumberOfWasmArgumentGPRs = 8
+else
+    error
+end
+const NumberOfWasmArgumentFPRs = 8
+const NumberOfWasmArguments = NumberOfWasmArgumentGPRs + NumberOfWasmArgumentFPRs
+
+# All callee saves must match the definition in WasmCallee.cpp
+
+# These must match the definition in WasmMemoryInformation.cpp
+const wasmInstance = csr0
+const memoryBase = csr3
+const memorySize = csr4
+
+# This must match the definition in LowLevelInterpreter.asm
+if X86_64
+    const PB = csr2
+elsif ARM64 or ARM64E
+    const PB = csr7
+else
+    error
+end
+
+macro forEachArgumentGPR(fn)
+    fn(0 * 8, wa0)
+    fn(1 * 8, wa1)
+    fn(2 * 8, wa2)
+    fn(3 * 8, wa3)
+    fn(4 * 8, wa4)
+    fn(5 * 8, wa5)
+    if ARM64 or ARM64E
+        fn(6 * 8, wa6)
+        fn(7 * 8, wa7)
+    end
+end
+
+macro forEachArgumentFPR(fn)
+    fn((NumberOfWasmArgumentGPRs + 0) * 8, wfa0)
+    fn((NumberOfWasmArgumentGPRs + 1) * 8, wfa1)
+    fn((NumberOfWasmArgumentGPRs + 2) * 8, wfa2)
+    fn((NumberOfWasmArgumentGPRs + 3) * 8, wfa3)
+    fn((NumberOfWasmArgumentGPRs + 4) * 8, wfa4)
+    fn((NumberOfWasmArgumentGPRs + 5) * 8, wfa5)
+    fn((NumberOfWasmArgumentGPRs + 6) * 8, wfa6)
+    fn((NumberOfWasmArgumentGPRs + 7) * 8, wfa7)
+end
+
+# Helper macros
+# FIXME: Eventually this should be unified with the JS versions
+# https://bugs.webkit.org/show_bug.cgi?id=203656
+
+macro wasmDispatch(advanceReg)
+    addp advanceReg, PC
+    wasmNextInstruction()
+end
+
+macro wasmDispatchIndirect(offsetReg)
+    wasmDispatch(offsetReg)
+end
+
+macro wasmNextInstruction()
+    loadb [PB, PC, 1], t0
+    leap _g_opcodeMap, t1
+    jmp NumberOfJSOpcodeIDs * PtrSize[t1, t0, PtrSize], BytecodePtrTag
+end
+
+macro wasmNextInstructionWide16()
+    loadh 1[PB, PC, 1], t0
+    leap _g_opcodeMapWide16, t1
+    jmp NumberOfJSOpcodeIDs * PtrSize[t1, t0, PtrSize], BytecodePtrTag
+end
+
+macro wasmNextInstructionWide32()
+    loadi 1[PB, PC, 1], t0
+    leap _g_opcodeMapWide32, t1
+    jmp NumberOfJSOpcodeIDs * PtrSize[t1, t0, PtrSize], BytecodePtrTag
+end
+
+macro checkSwitchToJIT(increment, action)
+    loadp CodeBlock[cfr], ws0
+    baddis increment, Wasm::FunctionCodeBlock::m_tierUpCounter + Wasm::LLIntTierUpCounter::m_counter[ws0], .continue
+    action()
+    .continue:
+end
+
+macro checkSwitchToJITForPrologue(codeBlockRegister)
+    checkSwitchToJIT(
+        5,
+        macro()
+            move cfr, a0
+            move PC, a1
+            move wasmInstance, a2
+            cCall4(_slow_path_wasm_prologue_osr)
+            btpz r0, .recover
+            move r0, ws0
+
+            forEachArgumentGPR(macro (offset, gpr)
+                loadq -offset - 8 - CalleeSaveSpaceAsVirtualRegisters * 8[cfr], gpr
+            end)
+
+            forEachArgumentFPR(macro (offset, fpr)
+                loadd -offset - 8 - CalleeSaveSpaceAsVirtualRegisters * 8[cfr], fpr
+            end)
+
+            restoreCalleeSavesUsedByWasm()
+            restoreCallerPCAndCFR()
+            untagReturnAddress sp
+            jmp ws0, WasmEntryPtrTag
+        .recover:
+            notFunctionCodeBlockGetter(codeBlockRegister)
+        end)
+end
+
+macro checkSwitchToJITForLoop()
+    checkSwitchToJIT(
+        1,
+        macro()
+            storei PC, ArgumentCount + TagOffset[cfr]
+            prepareStateForCCall()
+            move cfr, a0
+            move PC, a1
+            move wasmInstance, a2
+            cCall4(_slow_path_wasm_loop_osr)
+            btpz r1, .recover
+            restoreCalleeSavesUsedByWasm()
+            restoreCallerPCAndCFR()
+            untagReturnAddress sp
+            move r0, a0
+            jmp r1, WasmEntryPtrTag
+        .recover:
+            loadi ArgumentCount + TagOffset[cfr], PC
+        end)
+end
+
+macro checkSwitchToJITForEpilogue()
+    checkSwitchToJIT(
+        10,
+        macro ()
+            callWasmSlowPath(_slow_path_wasm_epilogue_osr)
+        end)
+end
+
+# Wasm specific helpers
+
+macro preserveCalleeSavesUsedByWasm()
+    subp CalleeSaveSpaceStackAligned, sp
+    if ARM64 or ARM64E
+        emit "stp x23, x26, [x29, #-16]"
+        emit "stp x19, x22, [x29, #-32]"
+    elsif X86_64
+        storep memorySize, -0x08[cfr]
+        storep memoryBase, -0x10[cfr]
+        storep PB, -0x18[cfr]
+        storep wasmInstance, -0x20[cfr]
+    else
+        error
+    end
+end
+
+macro restoreCalleeSavesUsedByWasm()
+    if ARM64 or ARM64E
+        emit "ldp x23, x26, [x29, #-16]"
+        emit "ldp x19, x22, [x29, #-32]"
+    elsif X86_64
+        loadp -0x08[cfr], memorySize
+        loadp -0x10[cfr], memoryBase
+        loadp -0x18[cfr], PB
+        loadp -0x20[cfr], wasmInstance
+    else
+        error
+    end
+end
+
+macro loadWasmInstanceFromTLS()
+if  HAVE_FAST_TLS
+    tls_loadp WTF_WASM_CONTEXT_KEY, wasmInstance
+else
+    crash()
+end
+end
+
+macro storeWasmInstanceToTLS(instance)
+if  HAVE_FAST_TLS
+    tls_storep instance, WTF_WASM_CONTEXT_KEY
+else
+    crash()
+end
+end
+
+macro reloadMemoryRegistersFromInstance(instance, scratch1, scratch2)
+    loadp Wasm::Instance::m_cachedMemory[instance], memoryBase
+    loadi Wasm::Instance::m_cachedMemorySize[instance], memorySize
+    cagedPrimitive(memoryBase, memorySize, scratch1, scratch2)
+end
+
+macro throwException(exception)
+    storei constexpr Wasm::ExceptionType::%exception%, ArgumentCount + PayloadOffset[cfr]
+    jmp _wasm_throw_from_slow_path_trampoline
+end
+
+macro callWasmSlowPath(slowPath)
+    prepareStateForCCall()
+    move cfr, a0
+    move PC, a1
+    move wasmInstance, a2
+    cCall4(slowPath)
+    restoreStateAfterCCall()
+end
+
+macro callWasmCallSlowPath(slowPath, action)
+    storei PC, ArgumentCount + TagOffset[cfr]
+    prepareStateForCCall()
+    move cfr, a0
+    move PC, a1
+    move wasmInstance, a2
+    cCall4(slowPath)
+    action(r0, r1)
+end
+
+macro wasmPrologue(codeBlockGetter, codeBlockSetter, loadWasmInstance)
+    # Set up the call frame and check if we should OSR.
+    tagReturnAddress sp
+    preserveCallerPCAndCFR()
+    preserveCalleeSavesUsedByWasm()
+    loadWasmInstance()
+    reloadMemoryRegistersFromInstance(wasmInstance, ws0, ws1)
+
+    codeBlockGetter(ws0)
+    codeBlockSetter(ws0)
+
+    # Get new sp in ws1 and check stack height.
+    loadi Wasm::FunctionCodeBlock::m_numCalleeLocals[ws0], ws1
+    lshiftp 3, ws1
+    addp maxFrameExtentForSlowPathCall, ws1
+    subp cfr, ws1, ws1
+
+    bpa ws1, cfr, .stackOverflow
+    bpbeq Wasm::Instance::m_cachedStackLimit[wasmInstance], ws1, .stackHeightOK
+
+.stackOverflow:
+    throwException(StackOverflow)
+
+.stackHeightOK:
+    move ws1, sp
+
+    forEachArgumentGPR(macro (offset, gpr)
+        storeq gpr, -offset - 8 - CalleeSaveSpaceAsVirtualRegisters * 8[cfr]
+    end)
+
+    forEachArgumentFPR(macro (offset, fpr)
+        stored fpr, -offset - 8 - CalleeSaveSpaceAsVirtualRegisters * 8[cfr]
+    end)
+
+    checkSwitchToJITForPrologue(ws0)
+
+    # Set up the PC.
+    loadp Wasm::FunctionCodeBlock::m_instructionsRawPointer[ws0], PB
+    move 0, PC
+
+    loadi Wasm::FunctionCodeBlock::m_numVars[ws0], ws1
+    subi NumberOfWasmArguments + CalleeSaveSpaceAsVirtualRegisters, ws1
+    btiz ws1, .zeroInitializeLocalsDone
+    negi ws1
+    sxi2q ws1, ws1
+    leap (NumberOfWasmArguments + CalleeSaveSpaceAsVirtualRegisters) * -8[cfr], ws0
+.zeroInitializeLocalsLoop:
+    addq 1, ws1
+    storeq 0, [ws0, ws1, 8]
+    btqz ws1, .zeroInitializeLocalsDone
+    jmp .zeroInitializeLocalsLoop
+.zeroInitializeLocalsDone:
+end
+
+macro traceExecution()
+    if TRACING
+        callWasmSlowPath(_slow_path_wasm_trace)
+    end
+end
+
+# Less convenient, but required for opcodes that collide with reserved instructions (e.g. wasm_nop)
+macro unprefixedWasmOp(opcodeName, opcodeStruct, fn)
+    commonOp(opcodeName, traceExecution, macro(size)
+        fn(macro(fn2)
+            fn2(opcodeName, opcodeStruct, size)
+        end)
+    end)
+end
+
+macro wasmOp(opcodeName, opcodeStruct, fn)
+    unprefixedWasmOp(wasm_%opcodeName%, opcodeStruct, fn)
+end
+
+# Same as unprefixedWasmOp, necessary for e.g. wasm_call
+macro unprefixedSlowWasmOp(opcodeName)
+    unprefixedWasmOp(opcodeName, unusedOpcodeStruct, macro(ctx)
+        callWasmSlowPath(_slow_path_%opcodeName%)
+        dispatch(ctx)
+    end)
+end
+
+macro slowWasmOp(opcodeName)
+    unprefixedSlowWasmOp(wasm_%opcodeName%)
+end
+
+# Macro version of load operations: mload[suffix]
+# loads field from the instruction stream and performs load[suffix] to dst
+macro firstConstantRegisterIndex(ctx, fn)
+    ctx(macro(opcodeName, opcodeStruct, size)
+        size(FirstConstantRegisterIndexNarrow, FirstConstantRegisterIndexWide16, FirstConstantRegisterIndexWide32, fn)
+    end)
+end
+
+macro loadConstantOrVariable(ctx, index, loader)
+    firstConstantRegisterIndex(ctx, macro (firstConstantIndex)
+        bpgteq index, firstConstantIndex, .constant
+        loader([cfr, index, 8])
+        jmp .done
+    .constant:
+        loadp CodeBlock[cfr], t6
+        loadp Wasm::FunctionCodeBlock::m_constants + VectorBufferOffset[t6], t6
+        subp firstConstantIndex, index
+        loader([t6, index, 8])
+    .done:
+    end)
+end
+
+macro mloadq(ctx, field, dst)
+    wgets(ctx, field, dst)
+    loadConstantOrVariable(ctx, dst, macro (from)
+        loadq from, dst
+    end)
+end
+
+macro mloadi(ctx, field, dst)
+    wgets(ctx, field, dst)
+    loadConstantOrVariable(ctx, dst, macro (from)
+        loadi from, dst
+    end)
+end
+
+macro mloadp(ctx, field, dst)
+    wgets(ctx, field, dst)
+    loadConstantOrVariable(ctx, dst, macro (from)
+        loadp from, dst
+    end)
+end
+
+macro mloadf(ctx, field, dst)
+    wgets(ctx, field, t5)
+    loadConstantOrVariable(ctx, t5, macro (from)
+        loadf from, dst
+    end)
+end
+
+macro mloadd(ctx, field, dst)
+    wgets(ctx, field, t5)
+    loadConstantOrVariable(ctx, t5, macro (from)
+        loadd from, dst
+    end)
+end
+
+# Typed returns
+
+macro returnq(ctx, value)
+    wgets(ctx, m_dst, t5)
+    storeq value, [cfr, t5, 8]
+    dispatch(ctx)
+end
+
+macro returni(ctx, value)
+    wgets(ctx, m_dst, t5)
+    storei value, [cfr, t5, 8]
+    dispatch(ctx)
+end
+
+macro returnf(ctx, value)
+    wgets(ctx, m_dst, t5)
+    storef value, [cfr, t5, 8]
+    dispatch(ctx)
+end
+
+macro returnd(ctx, value)
+    wgets(ctx, m_dst, t5)
+    stored value, [cfr, t5, 8]
+    dispatch(ctx)
+end
+
+# Wasm wrapper of get/getu that operate on ctx
+macro wgets(ctx, field, dst)
+    ctx(macro(opcodeName, opcodeStruct, size)
+        size(getOperandNarrow, getOperandWide16, getOperandWide32, macro (get)
+            get(opcodeStruct, field, dst)
+        end)
+    end)
+end
+
+macro wgetu(ctx, field, dst)
+    ctx(macro(opcodeName, opcodeStruct, size)
+        size(getuOperandNarrow, getuOperandWide16, getuOperandWide32, macro (getu)
+            getu(opcodeStruct, field, dst)
+        end)
+    end)
+end
+
+# Control flow helpers
+
+macro dispatch(ctx)
+    ctx(macro(opcodeName, opcodeStruct, size)
+        genericDispatchOp(wasmDispatch, size, opcodeName)
+    end)
+end
+
+macro jump(ctx, target)
+    wgets(ctx, target, t0)
+    btiz t0, .outOfLineJumpTarget
+    wasmDispatchIndirect(t0)
+.outOfLineJumpTarget:
+    callWasmSlowPath(_slow_path_wasm_out_of_line_jump_target)
+    wasmNextInstruction()
+end
+
+macro doReturn()
+    restoreCalleeSavesUsedByWasm()
+    restoreCallerPCAndCFR()
+    ret
+end
+
+# Entry point
+
+macro wasmCodeBlockGetter(targetRegister)
+    loadp Callee[cfr], targetRegister
+    andp ~3, targetRegister
+    loadp Wasm::LLIntCallee::m_codeBlock[targetRegister], targetRegister
+end
+
+op(wasm_function_prologue, macro ()
+    if not WEBASSEMBLY or not JSVALUE64 or C_LOOP or C_LOOP_WIN
+        error
+    end
+
+    wasmPrologue(wasmCodeBlockGetter, functionCodeBlockSetter, loadWasmInstanceFromTLS)
+    wasmNextInstruction()
+end)
+
+op(wasm_function_prologue_no_tls, macro ()
+    if not WEBASSEMBLY or not JSVALUE64 or C_LOOP or C_LOOP_WIN
+        error
+    end
+
+    wasmPrologue(wasmCodeBlockGetter, functionCodeBlockSetter, macro () end)
+    wasmNextInstruction()
+end)
+
+op(wasm_throw_from_slow_path_trampoline, macro ()
+    loadp Wasm::Instance::m_pointerToTopEntryFrame[wasmInstance], t5
+    loadp [t5], t5
+    copyCalleeSavesToEntryFrameCalleeSavesBuffer(t5)
+
+    move cfr, a0
+    addp PB, PC, a1
+    move wasmInstance, a2
+    # Slow paths and the throwException macro store the exception code in the ArgumentCount slot
+    loadi ArgumentCount + PayloadOffset[cfr], a3
+    cCall4(_slow_path_wasm_throw_exception)
+
+    jmp r0, ExceptionHandlerPtrTag
+end)
+
+# Disable wide version of narrow-only opcodes
+noWide(wasm_enter)
+noWide(wasm_wide16)
+noWide(wasm_wide32)
+
+# Opcodes that always invoke the slow path
+
+slowWasmOp(ref_func)
+slowWasmOp(table_get)
+slowWasmOp(table_set)
+slowWasmOp(table_size)
+slowWasmOp(table_fill)
+slowWasmOp(table_grow)
+slowWasmOp(set_global_ref)
+
+wasmOp(grow_memory, WasmGrowMemory, macro(ctx)
+    callWasmSlowPath(_slow_path_wasm_grow_memory)
+    reloadMemoryRegistersFromInstance(wasmInstance, ws0, ws1)
+    dispatch(ctx)
+end)
+
+# Opcodes that should eventually be shared with JS llint
+
+_wasm_wide16:
+    wasmNextInstructionWide16()
+
+_wasm_wide32:
+    wasmNextInstructionWide32()
+
+_wasm_enter:
+    traceExecution()
+    checkStackPointerAlignment(t2, 0xdead00e1)
+    loadp CodeBlock[cfr], t2                // t2<CodeBlock> = cfr.CodeBlock
+    loadi Wasm::FunctionCodeBlock::m_numVars[t2], t2      // t2<size_t> = t2<CodeBlock>.m_numVars
+    subq CalleeSaveSpaceAsVirtualRegisters + NumberOfWasmArguments, t2
+    btiz t2, .opEnterDone
+    move cfr, t1
+    subq CalleeSaveSpaceAsVirtualRegisters * 8 + NumberOfWasmArguments * 8, t1
+    negi t2
+    sxi2q t2, t2
+.opEnterLoop:
+    storeq 0, [t1, t2, 8]
+    addq 1, t2
+    btqnz t2, .opEnterLoop
+.opEnterDone:
+    wasmDispatchIndirect(1)
+
+unprefixedWasmOp(wasm_nop, WasmNop, macro(ctx)
+    dispatch(ctx)
+end)
+
+wasmOp(loop_hint, WasmLoopHint, macro(ctx)
+    checkSwitchToJITForLoop()
+    dispatch(ctx)
+end)
+
+wasmOp(mov, WasmMov, macro(ctx)
+    mloadq(ctx, m_src, t0)
+    returnq(ctx, t0)
+end)
+
+wasmOp(jtrue, WasmJtrue, macro(ctx)
+    mloadi(ctx, m_condition, t0)
+    btiz t0, .continue
+    jump(ctx, m_targetLabel)
+.continue:
+    dispatch(ctx)
+end)
+
+wasmOp(jfalse, WasmJfalse, macro(ctx)
+    mloadi(ctx, m_condition, t0)
+    btinz t0, .continue
+    jump(ctx, m_targetLabel)
+.continue:
+    dispatch(ctx)
+end)
+
+wasmOp(switch, WasmSwitch, macro(ctx)
+    mloadi(ctx, m_scrutinee, t0)
+    wgetu(ctx, m_tableIndex, t1)
+
+    loadp CodeBlock[cfr], t2
+    loadp Wasm::FunctionCodeBlock::m_jumpTables + VectorBufferOffset[t2], t2
+    muli sizeof Wasm::FunctionCodeBlock::JumpTable, t1
+    addp t1, t2
+
+    loadi VectorSizeOffset[t2], t3
+    biaeq t0, t3, .default
+    loadp VectorBufferOffset[t2], t2
+    loadi [t2, t0, 4], t3
+    assert(macro(ok) btinz t3, .ok end)
+    wasmDispatchIndirect(t3)
+
+.default:
+    jump(ctx, m_defaultTarget)
+end)
+
+unprefixedWasmOp(wasm_jmp, WasmJmp, macro(ctx)
+    jump(ctx, m_targetLabel)
+end)
+
+unprefixedWasmOp(wasm_ret, WasmRet, macro(ctx)
+    checkSwitchToJITForEpilogue()
+    wgetu(ctx, m_stackOffset, ws1)
+    lshifti 3, ws1
+    negi ws1
+    sxi2q ws1, ws1
+    addp cfr, ws1
+    forEachArgumentGPR(macro (offset, gpr)
+        loadq offset[ws1], gpr
+    end)
+    forEachArgumentFPR(macro (offset, fpr)
+        loadd offset[ws1], fpr
+    end)
+    doReturn()
+end)
+
+# Wasm specific bytecodes
+
+wasmOp(unreachable, WasmUnreachable, macro(ctx)
+    throwException(Unreachable)
+end)
+
+wasmOp(ret_void, WasmRetVoid, macro(ctx)
+    checkSwitchToJITForEpilogue()
+    doReturn()
+end)
+
+wasmOp(ref_is_null, WasmRefIsNull, macro(ctx)
+    mloadp(ctx, m_ref, t0)
+    cqeq t0, ValueNull, t0
+    returni(ctx, t0)
+end)
+
+wasmOp(get_global, WasmGetGlobal, macro(ctx)
+    loadp Wasm::Instance::m_globals[wasmInstance], t0
+    wgetu(ctx, m_globalIndex, t1)
+    loadq [t0, t1, 8], t0
+    returnq(ctx, t0)
+end)
+
+wasmOp(set_global, WasmSetGlobal, macro(ctx)
+    loadp Wasm::Instance::m_globals[wasmInstance], t0
+    wgetu(ctx, m_globalIndex, t1)
+    mloadq(ctx, m_value, t2)
+    storeq t2, [t0, t1, 8]
+    dispatch(ctx)
+end)
+
+macro slowPathForWasmCall(ctx, slowPath, storeWasmInstance)
+    callWasmCallSlowPath(
+        slowPath,
+        # callee is r0 and targetWasmInstance is r1
+        macro (callee, targetWasmInstance)
+            move callee, ws0
+
+            loadi ArgumentCount + TagOffset[cfr], PC
+
+            # the call might throw (e.g. indirect call with bad signature)
+            btpz targetWasmInstance, .throw
+
+            wgetu(ctx, m_stackOffset, ws1)
+            lshifti 3, ws1
+            subp cfr, ws1, sp
+
+            wgetu(ctx, m_numberOfStackArgs, ws1)
+
+            # Preserve the current instance
+            move wasmInstance, PB
+
+            storeWasmInstance(targetWasmInstance)
+            reloadMemoryRegistersFromInstance(targetWasmInstance, wa0, wa1)
+
+            # Load registers from stack
+            forEachArgumentGPR(macro (offset, gpr)
+                loadq CallFrameHeaderSize + offset[sp, ws1, 8], gpr
+            end)
+
+            forEachArgumentFPR(macro (offset, fpr)
+                loadd CallFrameHeaderSize + offset[sp, ws1, 8], fpr
+            end)
+
+            addp CallerFrameAndPCSize, sp
+            call ws0, SlowPathPtrTag
+
+            loadp CodeBlock[cfr], ws1
+            loadi Wasm::FunctionCodeBlock::m_numCalleeLocals[ws1], ws1
+            lshiftp 3, ws1
+            addp maxFrameExtentForSlowPathCall, ws1
+            subp cfr, ws1, sp
+
+            # We need to set PC to load information from the instruction stream, but we
+            # need to preserve its current value since it might contain a return value
+            move PC, memoryBase
+            move PB, wasmInstance
+            loadi ArgumentCount + TagOffset[cfr], PC
+            loadp CodeBlock[cfr], PB
+            loadp Wasm::FunctionCodeBlock::m_instructionsRawPointer[PB], PB
+
+            wgetu(ctx, m_stackOffset, ws1)
+            lshifti 3, ws1
+            negi ws1
+            sxi2q ws1, ws1
+            addp cfr, ws1
+
+            # Argument registers are also return registers, so they must be stored to the stack
+            # in case they contain return values.
+            wgetu(ctx, m_numberOfStackArgs, ws0)
+            move memoryBase, PC
+            forEachArgumentGPR(macro (offset, gpr)
+                storeq gpr, CallFrameHeaderSize + offset[ws1, ws0, 8]
+            end)
+
+            forEachArgumentFPR(macro (offset, fpr)
+                stored fpr, CallFrameHeaderSize + offset[ws1, ws0, 8]
+            end)
+
+            loadi ArgumentCount + TagOffset[cfr], PC
+
+            storeWasmInstance(wasmInstance)
+            reloadMemoryRegistersFromInstance(wasmInstance, ws0, ws1)
+
+            # Restore stack limit
+            loadp Wasm::Instance::m_pointerToActualStackLimit[wasmInstance], t5
+            loadp [t5], t5
+            storep t5, Wasm::Instance::m_cachedStackLimit[wasmInstance]
+
+            dispatch(ctx)
+
+        .throw:
+            restoreStateAfterCCall()
+            dispatch(ctx)
+        end)
+end
+
+unprefixedWasmOp(wasm_call, WasmCall, macro(ctx)
+    slowPathForWasmCall(ctx, _slow_path_wasm_call, storeWasmInstanceToTLS)
+end)
+
+unprefixedWasmOp(wasm_call_no_tls, WasmCallNoTls, macro(ctx)
+    slowPathForWasmCall(ctx, _slow_path_wasm_call_no_tls, macro(targetInstance) move targetInstance, wasmInstance end)
+end)
+
+wasmOp(call_indirect, WasmCallIndirect, macro(ctx)
+    slowPathForWasmCall(ctx, _slow_path_wasm_call_indirect, storeWasmInstanceToTLS)
+end)
+
+wasmOp(call_indirect_no_tls, WasmCallIndirectNoTls, macro(ctx)
+    slowPathForWasmCall(ctx, _slow_path_wasm_call_indirect_no_tls, macro(targetInstance) move targetInstance, wasmInstance end)
+end)
+
+wasmOp(current_memory, WasmCurrentMemory, macro(ctx)
+    loadp Wasm::Instance::m_cachedMemorySize[wasmInstance], t0
+    urshiftq 16, t0
+    returnq(ctx, t0)
+end)
+
+wasmOp(select, WasmSelect, macro(ctx)
+    mloadi(ctx, m_condition, t0)
+    btiz t0, .isZero
+    mloadq(ctx, m_nonZero, t0)
+    returnq(ctx, t0)
+.isZero:
+    mloadq(ctx, m_zero, t0)
+    returnq(ctx, t0)
+end)
+
+# uses offset as scratch and returns result on pointer
+macro emitCheckAndPreparePointer(ctx, pointer, offset, size)
+    leap size - 1[pointer, offset], t5
+    bpb t5, memorySize, .continuation
+    throwException(OutOfBoundsMemoryAccess)
+.continuation:
+    addp memoryBase, pointer
+end
+
+macro wasmLoadOp(name, struct, size, fn)
+    wasmOp(name, struct, macro(ctx)
+        mloadi(ctx, m_pointer, t0)
+        wgetu(ctx, m_offset, t1)
+        emitCheckAndPreparePointer(ctx, t0, t1, size)
+        fn([t0, t1], t2)
+        returnq(ctx, t2)
+    end)
+end
+
+wasmLoadOp(load8_u, WasmLoad8U, 1, macro(mem, dst) loadb mem, dst end)
+wasmLoadOp(load16_u, WasmLoad16U, 2, macro(mem, dst) loadh mem, dst end)
+wasmLoadOp(load32_u, WasmLoad32U, 4, macro(mem, dst) loadi mem, dst end)
+wasmLoadOp(load64_u, WasmLoad64U, 8, macro(mem, dst) loadq mem, dst end)
+
+wasmLoadOp(i32_load8_s, WasmI32Load8S, 1, macro(mem, dst) loadbsi mem, dst end)
+wasmLoadOp(i64_load8_s, WasmI64Load8S, 1, macro(mem, dst) loadbsq mem, dst end)
+wasmLoadOp(i32_load16_s, WasmI32Load16S, 2, macro(mem, dst) loadhsi mem, dst end)
+wasmLoadOp(i64_load16_s, WasmI64Load16S, 2, macro(mem, dst) loadhsq mem, dst end)
+wasmLoadOp(i64_load32_s, WasmI64Load32S, 4, macro(mem, dst) loadis mem, dst end)
+
+macro wasmStoreOp(name, struct, size, fn)
+    wasmOp(name, struct, macro(ctx)
+        mloadi(ctx, m_pointer, t0)
+        wgetu(ctx, m_offset, t1)
+        emitCheckAndPreparePointer(ctx, t0, t1, size)
+        mloadq(ctx, m_value, t2)
+        fn(t2, [t0, t1])
+        dispatch(ctx)
+    end)
+end
+
+wasmStoreOp(store8, WasmStore8, 1, macro(value, mem) storeb value, mem end)
+wasmStoreOp(store16, WasmStore16, 2, macro(value, mem) storeh value, mem end)
+wasmStoreOp(store32, WasmStore32, 4, macro(value, mem) storei value, mem end)
+wasmStoreOp(store64, WasmStore64, 8, macro(value, mem) storeq value, mem end)
+
+# Opcodes that don't have the `b3op` entry in wasm.json. This should be kept in sync
+
+wasmOp(i32_div_s, WasmI32DivS, macro (ctx)
+    mloadi(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+
+    btiz t1, .throwDivisionByZero
+
+    bineq t1, -1, .safe
+    bieq t0, constexpr INT32_MIN, .throwIntegerOverflow
+
+.safe:
+    if X86_64
+        # FIXME: Add a way to static_asset that t0 is rax and t2 is rdx
+        # https://bugs.webkit.org/show_bug.cgi?id=203692
+        cdqi
+        idivi t1
+    elsif ARM64 or ARM64E
+        divis t1, t0
+    else
+        error
+    end
+    returni(ctx, t0)
+
+.throwDivisionByZero:
+    throwException(DivisionByZero)
+
+.throwIntegerOverflow:
+    throwException(IntegerOverflow)
+end)
+
+wasmOp(i32_div_u, WasmI32DivU, macro (ctx)
+    mloadi(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+
+    btiz t1, .throwDivisionByZero
+
+    if X86_64
+        xori t2, t2
+        udivi t1
+    elsif ARM64 or ARM64E
+        divi t1, t0
+    else
+        error
+    end
+    returni(ctx, t0)
+
+.throwDivisionByZero:
+    throwException(DivisionByZero)
+end)
+
+wasmOp(i32_rem_s, WasmI32RemS, macro (ctx)
+    mloadi(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+
+    btiz t1, .throwDivisionByZero
+
+    bineq t1, -1, .safe
+    bineq t0, constexpr INT32_MIN, .safe
+
+    move 0, t2
+    jmp .return
+
+.safe:
+    if X86_64
+        # FIXME: Add a way to static_asset that t0 is rax and t2 is rdx
+        # https://bugs.webkit.org/show_bug.cgi?id=203692
+        cdqi
+        idivi t1
+    elsif ARM64 or ARM64E
+        divis t1, t0, t2
+        muli t1, t2
+        subi t0, t2, t2
+    else
+        error
+    end
+
+.return:
+    returni(ctx, t2)
+
+.throwDivisionByZero:
+    throwException(DivisionByZero)
+end)
+
+wasmOp(i32_rem_u, WasmI32RemU, macro (ctx)
+    mloadi(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+
+    btiz t1, .throwDivisionByZero
+
+    if X86_64
+        xori t2, t2
+        udivi t1
+    elsif ARM64 or ARM64E
+        divi t1, t0, t2
+        muli t1, t2
+        subi t0, t2, t2
+    else
+        error
+    end
+    returni(ctx, t2)
+
+.throwDivisionByZero:
+    throwException(DivisionByZero)
+end)
+
+wasmOp(i32_ctz, WasmI32Ctz, macro (ctx)
+    mloadq(ctx, m_operand, t0)
+    tzcnti t0, t0
+    returni(ctx, t0)
+end)
+
+wasmOp(i32_popcnt, WasmI32Popcnt, macro (ctx)
+    mloadi(ctx, m_operand, a1)
+    prepareStateForCCall()
+    move PC, a0
+    cCall2(_slow_path_wasm_popcount)
+    restoreStateAfterCCall()
+    returni(ctx, r1)
+end)
+
+wasmOp(i64_div_s, WasmI64DivS, macro (ctx)
+    mloadq(ctx, m_lhs, t0)
+    mloadq(ctx, m_rhs, t1)
+
+    btqz t1, .throwDivisionByZero
+
+    bqneq t1, -1, .safe
+    bqeq t0, constexpr INT64_MIN, .throwIntegerOverflow
+
+.safe:
+    if X86_64
+        # FIXME: Add a way to static_asset that t0 is rax and t2 is rdx
+        # https://bugs.webkit.org/show_bug.cgi?id=203692
+        cqoq
+        idivq t1
+    elsif ARM64 or ARM64E
+        divqs t1, t0
+    else
+        error
+    end
+    returnq(ctx, t0)
+
+.throwDivisionByZero:
+    throwException(DivisionByZero)
+
+.throwIntegerOverflow:
+    throwException(IntegerOverflow)
+end)
+
+wasmOp(i64_div_u, WasmI64DivU, macro (ctx)
+    mloadq(ctx, m_lhs, t0)
+    mloadq(ctx, m_rhs, t1)
+
+    btqz t1, .throwDivisionByZero
+
+    if X86_64
+        xorq t2, t2
+        udivq t1
+    elsif ARM64 or ARM64E
+        divq t1, t0
+    else
+        error
+    end
+    returnq(ctx, t0)
+
+.throwDivisionByZero:
+    throwException(DivisionByZero)
+end)
+
+wasmOp(i64_rem_s, WasmI64RemS, macro (ctx)
+    mloadq(ctx, m_lhs, t0)
+    mloadq(ctx, m_rhs, t1)
+
+    btqz t1, .throwDivisionByZero
+
+    bqneq t1, -1, .safe
+    bqneq t0, constexpr INT64_MIN, .safe
+
+    move 0, t2
+    jmp .return
+
+.safe:
+    if X86_64
+        # FIXME: Add a way to static_asset that t0 is rax and t2 is rdx
+        # https://bugs.webkit.org/show_bug.cgi?id=203692
+        cqoq
+        idivq t1
+    elsif ARM64 or ARM64E
+        divqs t1, t0, t2
+        mulq t1, t2
+        subq t0, t2, t2
+    else
+        error
+    end
+
+.return:
+    returnq(ctx, t2) # rdx has the remainder
+
+.throwDivisionByZero:
+    throwException(DivisionByZero)
+end)
+
+wasmOp(i64_rem_u, WasmI64RemU, macro (ctx)
+    mloadq(ctx, m_lhs, t0)
+    mloadq(ctx, m_rhs, t1)
+
+    btqz t1, .throwDivisionByZero
+
+    if X86_64
+        xorq t2, t2
+        udivq t1
+    elsif ARM64 or ARM64E
+        divq t1, t0, t2
+        mulq t1, t2
+        subq t0, t2, t2
+    else
+        error
+    end
+    returnq(ctx, t2)
+
+.throwDivisionByZero:
+    throwException(DivisionByZero)
+end)
+
+wasmOp(i64_ctz, WasmI64Ctz, macro (ctx)
+    mloadq(ctx, m_operand, t0)
+    tzcntq t0, t0
+    returnq(ctx, t0)
+end)
+
+wasmOp(i64_popcnt, WasmI64Popcnt, macro (ctx)
+    mloadq(ctx, m_operand, a1)
+    prepareStateForCCall()
+    move PC, a0
+    cCall2(_slow_path_wasm_popcountll)
+    restoreStateAfterCCall()
+    returnq(ctx, r1)
+end)
+
+wasmOp(f32_trunc, WasmF32Trunc, macro (ctx)
+    mloadf(ctx, m_operand, ft0)
+    truncatef ft0, ft0
+    returnf(ctx, ft0)
+end)
+
+wasmOp(f32_nearest, WasmF32Nearest, macro (ctx)
+    mloadf(ctx, m_operand, ft0)
+    roundf ft0, ft0
+    returnf(ctx, ft0)
+end)
+
+wasmOp(f64_trunc, WasmF64Trunc, macro (ctx)
+    mloadd(ctx, m_operand, ft0)
+    truncated ft0, ft0
+    returnd(ctx, ft0)
+end)
+
+wasmOp(f64_nearest, WasmF64Nearest, macro (ctx)
+    mloadd(ctx, m_operand, ft0)
+    roundd ft0, ft0
+    returnd(ctx, ft0)
+end)
+
+wasmOp(i32_trunc_s_f32, WasmI32TruncSF32, macro (ctx)
+    mloadf(ctx, m_operand, ft0)
+
+    move 0xcf000000, t0 # INT32_MIN
+    fi2f t0, ft1
+    bfltun ft0, ft1, .outOfBoundsTrunc
+
+    move 0x4f000000, t0 # -INT32_MIN
+    fi2f t0, ft1
+    bfgtequn ft0, ft1, .outOfBoundsTrunc
+
+    truncatef2is ft0, t0
+    returni(ctx, t0)
+
+.outOfBoundsTrunc:
+    throwException(OutOfBoundsTrunc)
+end)
+
+wasmOp(i32_trunc_s_f64, WasmI32TruncSF64, macro (ctx)
+    mloadd(ctx, m_operand, ft0)
+
+    move 0xc1e0000000000000, t0 # INT32_MIN
+    fq2d t0, ft1
+    bdltun ft0, ft1, .outOfBoundsTrunc
+
+    move 0x41e0000000000000, t0 # -INT32_MIN
+    fq2d t0, ft1
+    bdgtequn ft0, ft1, .outOfBoundsTrunc
+
+    truncated2is ft0, t0
+    returni(ctx, t0)
+
+.outOfBoundsTrunc:
+    throwException(OutOfBoundsTrunc)
+end)
+
+wasmOp(i32_trunc_u_f32, WasmI32TruncUF32, macro (ctx)
+    mloadf(ctx, m_operand, ft0)
+
+    move 0xbf800000, t0 # -1.0
+    fi2f t0, ft1
+    bfltequn ft0, ft1, .outOfBoundsTrunc
+
+    move 0x4f800000, t0 # INT32_MIN * -2.0
+    fi2f t0, ft1
+    bfgtequn ft0, ft1, .outOfBoundsTrunc
+
+    truncatef2i ft0, t0
+    returni(ctx, t0)
+
+.outOfBoundsTrunc:
+    throwException(OutOfBoundsTrunc)
+end)
+
+wasmOp(i32_trunc_u_f64, WasmI32TruncUF64, macro (ctx)
+    mloadd(ctx, m_operand, ft0)
+
+    move 0xbff0000000000000, t0 # -1.0
+    fq2d t0, ft1
+    bdltequn ft0, ft1, .outOfBoundsTrunc
+
+    move 0x41f0000000000000, t0 # INT32_MIN * -2.0
+    fq2d t0, ft1
+    bdgtequn ft0, ft1, .outOfBoundsTrunc
+
+    truncated2i ft0, t0
+    returni(ctx, t0)
+
+.outOfBoundsTrunc:
+    throwException(OutOfBoundsTrunc)
+end)
+
+wasmOp(i64_trunc_s_f32, WasmI64TruncSF32, macro (ctx)
+    mloadd(ctx, m_operand, ft0)
+
+    move 0xdf000000, t0 # INT64_MIN
+    fi2f t0, ft1
+    bfltun ft0, ft1, .outOfBoundsTrunc
+
+    move 0x5f000000, t0 # -INT64_MIN
+    fi2f t0, ft1
+    bfgtequn ft0, ft1, .outOfBoundsTrunc
+
+    truncatef2qs ft0, t0
+    returnq(ctx, t0)
+
+.outOfBoundsTrunc:
+    throwException(OutOfBoundsTrunc)
+end)
+
+wasmOp(i64_trunc_s_f64, WasmI64TruncSF64, macro (ctx)
+    mloadd(ctx, m_operand, ft0)
+
+    move 0xc3e0000000000000, t0 # INT64_MIN
+    fq2d t0, ft1
+    bdltun ft0, ft1, .outOfBoundsTrunc
+
+    move 0x43e0000000000000, t0 # -INT64_MIN
+    fq2d t0, ft1
+    bdgtequn ft0, ft1, .outOfBoundsTrunc
+
+    truncated2qs ft0, t0
+    returnq(ctx, t0)
+
+.outOfBoundsTrunc:
+    throwException(OutOfBoundsTrunc)
+end)
+
+wasmOp(i64_trunc_u_f32, WasmI64TruncUF32, macro (ctx)
+    mloadf(ctx, m_operand, ft0)
+
+    move 0xbf800000, t0 # -1.0
+    fi2f t0, ft1
+    bfltequn ft0, ft1, .outOfBoundsTrunc
+
+    move 0x5f800000, t0 # INT64_MIN * -2.0
+    fi2f t0, ft1
+    bfgtequn ft0, ft1, .outOfBoundsTrunc
+
+    truncatef2q ft0, t0
+    returnq(ctx, t0)
+
+.outOfBoundsTrunc:
+    throwException(OutOfBoundsTrunc)
+end)
+
+wasmOp(i64_trunc_u_f64, WasmI64TruncUF64, macro (ctx)
+    mloadd(ctx, m_operand, ft0)
+
+    move 0xbff0000000000000, t0 # -1.0
+    fq2d t0, ft1
+    bdltequn ft0, ft1, .outOfBoundsTrunc
+
+    move 0x43f0000000000000, t0 # INT64_MIN * -2.0
+    fq2d t0, ft1
+    bdgtequn ft0, ft1, .outOfBoundsTrunc
+
+    truncated2q ft0, t0
+    returnq(ctx, t0)
+
+.outOfBoundsTrunc:
+    throwException(OutOfBoundsTrunc)
+end)
+
+wasmOp(f32_convert_u_i64, WasmF32ConvertUI64, macro (ctx)
+    mloadq(ctx, m_operand, t0)
+    if X86_64
+        cq2f t0, t1, ft0
+    elsif ARM64 or ARM64E
+        cq2f t0, ft0
+    else
+        error
+    end
+    returnf(ctx, ft0)
+end)
+
+wasmOp(f64_convert_u_i64, WasmF64ConvertUI64, macro (ctx)
+    mloadq(ctx, m_operand, t0)
+    if X86_64
+        cq2d t0, t1, ft0
+    elsif ARM64 or ARM64E
+        cq2d t0, ft0
+    else
+        error
+    end
+    returnd(ctx, ft0)
+end)
+
+wasmOp(i32_eqz, WasmI32Eqz, macro(ctx)
+    mloadi(ctx, m_operand, t0)
+    cieq t0, 0, t0
+    returni(ctx, t0)
+end)
+
+wasmOp(i64_shl, WasmI64Shl, macro(ctx)
+    mloadq(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    lshiftq t1, t0
+    returnq(ctx, t0)
+end)
+
+wasmOp(i64_shr_u, WasmI64ShrU, macro(ctx)
+    mloadq(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    urshiftq t1, t0
+    returnq(ctx, t0)
+end)
+
+wasmOp(i64_shr_s, WasmI64ShrS, macro(ctx)
+    mloadq(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    rshiftq t1, t0
+    returnq(ctx, t0)
+end)
+
+wasmOp(i64_rotr, WasmI64Rotr, macro(ctx)
+    mloadq(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    rrotateq t1, t0
+    returnq(ctx, t0)
+end)
+
+wasmOp(i64_rotl, WasmI64Rotl, macro(ctx)
+    mloadq(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    lrotateq t1, t0
+    returnq(ctx, t0)
+end)
+
+wasmOp(i64_eqz, WasmI64Eqz, macro(ctx)
+    mloadq(ctx, m_operand, t0)
+    cqeq t0, 0, t0
+    returni(ctx, t0)
+end)
+
+wasmOp(f32_min, WasmF32Min, macro(ctx)
+    mloadf(ctx, m_lhs, ft0)
+    mloadf(ctx, m_rhs, ft1)
+
+    bfeq ft0, ft1, .equal
+    bflt ft0, ft1, .lt
+    bfgt ft0, ft1, .return
+
+.NaN:
+    addf ft0, ft1
+    jmp .return
+
+.equal:
+    orf ft0, ft1
+    jmp .return
+
+.lt:
+    moved ft0, ft1
+
+.return:
+    returnf(ctx, ft1)
+end)
+
+wasmOp(f32_max, WasmF32Max, macro(ctx)
+    mloadf(ctx, m_lhs, ft0)
+    mloadf(ctx, m_rhs, ft1)
+
+    bfeq ft1, ft0, .equal
+    bflt ft1, ft0, .lt
+    bfgt ft1, ft0, .return
+
+.NaN:
+    addf ft0, ft1
+    jmp .return
+
+.equal:
+    andf ft0, ft1
+    jmp .return
+
+.lt:
+    moved ft0, ft1
+
+.return:
+    returnf(ctx, ft1)
+end)
+
+wasmOp(f32_copysign, WasmF32Copysign, macro(ctx)
+    mloadf(ctx, m_lhs, ft0)
+    mloadf(ctx, m_rhs, ft1)
+
+    ff2i ft1, t1
+    move 0x80000000, t2
+    andi t2, t1
+
+    ff2i ft0, t0
+    move 0x7fffffff, t2
+    andi t2, t0
+
+    ori t1, t0
+    fi2f t0, ft0
+    returnf(ctx, ft0)
+end)
+
+wasmOp(f64_min, WasmF64Min, macro(ctx)
+    mloadd(ctx, m_lhs, ft0)
+    mloadd(ctx, m_rhs, ft1)
+
+    bdeq ft0, ft1, .equal
+    bdlt ft0, ft1, .lt
+    bdgt ft0, ft1, .return
+
+.NaN:
+    addd ft0, ft1
+    jmp .return
+
+.equal:
+    ord ft0, ft1
+    jmp .return
+
+.lt:
+    moved ft0, ft1
+
+.return:
+    returnd(ctx, ft1)
+end)
+
+wasmOp(f64_max, WasmF64Max, macro(ctx)
+    mloadd(ctx, m_lhs, ft0)
+    mloadd(ctx, m_rhs, ft1)
+
+    bdeq ft1, ft0, .equal
+    bdlt ft1, ft0, .lt
+    bdgt ft1, ft0, .return
+
+.NaN:
+    addd ft0, ft1
+    jmp .return
+
+.equal:
+    andd ft0, ft1
+    jmp .return
+
+.lt:
+    moved ft0, ft1
+
+.return:
+    returnd(ctx, ft1)
+end)
+
+wasmOp(f64_copysign, WasmF64Copysign, macro(ctx)
+    mloadd(ctx, m_lhs, ft0)
+    mloadd(ctx, m_rhs, ft1)
+
+    fd2q ft1, t1
+    move 0x8000000000000000, t2
+    andq t2, t1
+
+    fd2q ft0, t0
+    move 0x7fffffffffffffff, t2
+    andq t2, t0
+
+    orq t1, t0
+    fq2d t0, ft0
+    returnd(ctx, ft0)
+end)
+
+wasmOp(f32_convert_u_i32, WasmF32ConvertUI32, macro(ctx)
+    mloadi(ctx, m_operand, t0)
+    ci2f t0, ft0
+    returnf(ctx, ft0)
+end)
+
+wasmOp(f64_convert_u_i32, WasmF64ConvertUI32, macro(ctx)
+    mloadi(ctx, m_operand, t0)
+    ci2d t0, ft0
+    returnd(ctx, ft0)
+end)
+
+wasmOp(i32_add, WasmI32Add, macro(ctx)
+    mloadi(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    addi t0, t1, t2
+    returni(ctx, t2)
+end)
+
+wasmOp(i32_sub, WasmI32Sub, macro(ctx)
+    mloadi(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    subi t1, t0
+    returni(ctx, t0)
+end)
+
+wasmOp(i32_mul, WasmI32Mul, macro(ctx)
+    mloadi(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    muli t0, t1
+    returni(ctx, t1)
+end)
+
+wasmOp(i32_and, WasmI32And, macro(ctx)
+    mloadi(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    andi t0, t1
+    returni(ctx, t1)
+end)
+
+wasmOp(i32_or, WasmI32Or, macro(ctx)
+    mloadi(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    ori t0, t1
+    returni(ctx, t1)
+end)
+
+wasmOp(i32_xor, WasmI32Xor, macro(ctx)
+    mloadi(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    xori t0, t1
+    returni(ctx, t1)
+end)
+
+wasmOp(i32_shl, WasmI32Shl, macro(ctx)
+    mloadi(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    lshifti t1, t0
+    returni(ctx, t0)
+end)
+
+wasmOp(i32_shr_u, WasmI32ShrU, macro(ctx)
+    mloadi(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    urshifti t1, t0
+    returni(ctx, t0)
+end)
+
+wasmOp(i32_shr_s, WasmI32ShrS, macro(ctx)
+    mloadi(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    rshifti t1, t0
+    returni(ctx, t0)
+end)
+
+wasmOp(i32_rotr, WasmI32Rotr, macro(ctx)
+    mloadi(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    rrotatei t1, t0
+    returni(ctx, t0)
+end)
+
+wasmOp(i32_rotl, WasmI32Rotl, macro(ctx)
+    mloadi(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    lrotatei t1, t0
+    returni(ctx, t0)
+end)
+
+wasmOp(i32_eq, WasmI32Eq, macro(ctx)
+    mloadi(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    cieq t0, t1, t2
+    andi 1, t2
+    returni(ctx, t2)
+end)
+
+wasmOp(i32_ne, WasmI32Ne, macro(ctx)
+    mloadi(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    cineq t0, t1, t2
+    andi 1, t2
+    returni(ctx, t2)
+end)
+
+wasmOp(i32_lt_s, WasmI32LtS, macro(ctx)
+    mloadi(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    cilt t0, t1, t2
+    andi 1, t2
+    returni(ctx, t2)
+end)
+
+wasmOp(i32_le_s, WasmI32LeS, macro(ctx)
+    mloadi(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    cilteq t0, t1, t2
+    andi 1, t2
+    returni(ctx, t2)
+end)
+
+wasmOp(i32_lt_u, WasmI32LtU, macro(ctx)
+    mloadi(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    cib t0, t1, t2
+    andi 1, t2
+    returni(ctx, t2)
+end)
+
+wasmOp(i32_le_u, WasmI32LeU, macro(ctx)
+    mloadi(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    cibeq t0, t1, t2
+    andi 1, t2
+    returni(ctx, t2)
+end)
+
+wasmOp(i32_gt_s, WasmI32GtS, macro(ctx)
+    mloadi(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    cigt t0, t1, t2
+    andi 1, t2
+    returni(ctx, t2)
+end)
+
+wasmOp(i32_ge_s, WasmI32GeS, macro(ctx)
+    mloadi(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    cigteq t0, t1, t2
+    andi 1, t2
+    returni(ctx, t2)
+end)
+
+wasmOp(i32_gt_u, WasmI32GtU, macro(ctx)
+    mloadi(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    cia t0, t1, t2
+    andi 1, t2
+    returni(ctx, t2)
+end)
+
+wasmOp(i32_ge_u, WasmI32GeU, macro(ctx)
+    mloadi(ctx, m_lhs, t0)
+    mloadi(ctx, m_rhs, t1)
+    ciaeq t0, t1, t2
+    andi 1, t2
+    returni(ctx, t2)
+end)
+
+wasmOp(i32_clz, WasmI32Clz, macro(ctx)
+    mloadi(ctx, m_operand, t0)
+    lzcnti t0, t1
+    returni(ctx, t1)
+end)
+
+wasmOp(i64_add, WasmI64Add, macro(ctx)
+    mloadq(ctx, m_lhs, t0)
+    mloadq(ctx, m_rhs, t1)
+    addq t0, t1
+    returnq(ctx, t1)
+end)
+
+wasmOp(i64_sub, WasmI64Sub, macro(ctx)
+    mloadq(ctx, m_lhs, t0)
+    mloadq(ctx, m_rhs, t1)
+    subq t1, t0
+    returnq(ctx, t0)
+end)
+
+wasmOp(i64_mul, WasmI64Mul, macro(ctx)
+    mloadq(ctx, m_lhs, t0)
+    mloadq(ctx, m_rhs, t1)
+    mulq t0, t1
+    returnq(ctx, t1)
+end)
+
+wasmOp(i64_and, WasmI64And, macro(ctx)
+    mloadq(ctx, m_lhs, t0)
+    mloadq(ctx, m_rhs, t1)
+    andq t0, t1
+    returnq(ctx, t1)
+end)
+
+wasmOp(i64_or, WasmI64Or, macro(ctx)
+    mloadq(ctx, m_lhs, t0)
+    mloadq(ctx, m_rhs, t1)
+    orq t0, t1
+    returnq(ctx, t1)
+end)
+
+wasmOp(i64_xor, WasmI64Xor, macro(ctx)
+    mloadq(ctx, m_lhs, t0)
+    mloadq(ctx, m_rhs, t1)
+    xorq t0, t1
+    returnq(ctx, t1)
+end)
+
+wasmOp(i64_eq, WasmI64Eq, macro(ctx)
+    mloadq(ctx, m_lhs, t0)
+    mloadq(ctx, m_rhs, t1)
+    cqeq t0, t1, t2
+    andi 1, t2
+    returni(ctx, t2)
+end)
+
+wasmOp(i64_ne, WasmI64Ne, macro(ctx)
+    mloadq(ctx, m_lhs, t0)
+    mloadq(ctx, m_rhs, t1)
+    cqneq t0, t1, t2
+    andi 1, t2
+    returni(ctx, t2)
+end)
+
+wasmOp(i64_lt_s, WasmI64LtS, macro(ctx)
+    mloadq(ctx, m_lhs, t0)
+    mloadq(ctx, m_rhs, t1)
+    cqlt t0, t1, t2
+    andi 1, t2
+    returni(ctx, t2)
+end)
+
+wasmOp(i64_le_s, WasmI64LeS, macro(ctx)
+    mloadq(ctx, m_lhs, t0)
+    mloadq(ctx, m_rhs, t1)
+    cqlteq t0, t1, t2
+    andi 1, t2
+    returni(ctx, t2)
+end)
+
+wasmOp(i64_lt_u, WasmI64LtU, macro(ctx)
+    mloadq(ctx, m_lhs, t0)
+    mloadq(ctx, m_rhs, t1)
+    cqb t0, t1, t2
+    andi 1, t2
+    returni(ctx, t2)
+end)
+
+wasmOp(i64_le_u, WasmI64LeU, macro(ctx)
+    mloadq(ctx, m_lhs, t0)
+    mloadq(ctx, m_rhs, t1)
+    cqbeq t0, t1, t2
+    andi 1, t2
+    returni(ctx, t2)
+end)
+
+wasmOp(i64_gt_s, WasmI64GtS, macro(ctx)
+    mloadq(ctx, m_lhs, t0)
+    mloadq(ctx, m_rhs, t1)
+    cqgt t0, t1, t2
+    andi 1, t2
+    returni(ctx, t2)
+end)
+
+wasmOp(i64_ge_s, WasmI64GeS, macro(ctx)
+    mloadq(ctx, m_lhs, t0)
+    mloadq(ctx, m_rhs, t1)
+    cqgteq t0, t1, t2
+    andi 1, t2
+    returni(ctx, t2)
+end)
+
+wasmOp(i64_gt_u, WasmI64GtU, macro(ctx)
+    mloadq(ctx, m_lhs, t0)
+    mloadq(ctx, m_rhs, t1)
+    cqa t0, t1, t2
+    andi 1, t2
+    returni(ctx, t2)
+end)
+
+wasmOp(i64_ge_u, WasmI64GeU, macro(ctx)
+    mloadq(ctx, m_lhs, t0)
+    mloadq(ctx, m_rhs, t1)
+    cqaeq t0, t1, t2
+    andi 1, t2
+    returni(ctx, t2)
+end)
+
+wasmOp(i64_clz, WasmI64Clz, macro(ctx)
+    mloadq(ctx, m_operand, t0)
+    lzcntq t0, t1
+    returnq(ctx, t1)
+end)
+
+wasmOp(f32_add, WasmF32Add, macro(ctx)
+    mloadf(ctx, m_lhs, ft0)
+    mloadf(ctx, m_rhs, ft1)
+    addf ft0, ft1
+    returnf(ctx, ft1)
+end)
+
+wasmOp(f32_sub, WasmF32Sub, macro(ctx)
+    mloadf(ctx, m_lhs, ft0)
+    mloadf(ctx, m_rhs, ft1)
+    subf ft1, ft0
+    returnf(ctx, ft0)
+end)
+
+wasmOp(f32_mul, WasmF32Mul, macro(ctx)
+    mloadf(ctx, m_lhs, ft0)
+    mloadf(ctx, m_rhs, ft1)
+    mulf ft0, ft1
+    returnf(ctx, ft1)
+end)
+
+wasmOp(f32_div, WasmF32Div, macro(ctx)
+    mloadf(ctx, m_lhs, ft0)
+    mloadf(ctx, m_rhs, ft1)
+    divf ft1, ft0
+    returnf(ctx, ft0)
+end)
+
+wasmOp(f32_abs, WasmF32Abs, macro(ctx)
+    mloadf(ctx, m_operand, ft0)
+    absf ft0, ft1
+    returnf(ctx, ft1)
+end)
+
+wasmOp(f32_neg, WasmF32Neg, macro(ctx)
+    mloadf(ctx, m_operand, ft0)
+    negf ft0, ft1
+    returnf(ctx, ft1)
+end)
+
+wasmOp(f32_ceil, WasmF32Ceil, macro(ctx)
+    mloadf(ctx, m_operand, ft0)
+    ceilf ft0, ft1
+    returnf(ctx, ft1)
+end)
+
+wasmOp(f32_floor, WasmF32Floor, macro(ctx)
+    mloadf(ctx, m_operand, ft0)
+    floorf ft0, ft1
+    returnf(ctx, ft1)
+end)
+
+wasmOp(f32_sqrt, WasmF32Sqrt, macro(ctx)
+    mloadf(ctx, m_operand, ft0)
+    sqrtf ft0, ft1
+    returnf(ctx, ft1)
+end)
+
+wasmOp(f32_eq, WasmF32Eq, macro(ctx)
+    mloadf(ctx, m_lhs, ft0)
+    mloadf(ctx, m_rhs, ft1)
+    cfeq ft0, ft1, t0
+    returni(ctx, t0)
+end)
+
+wasmOp(f32_ne, WasmF32Ne, macro(ctx)
+    mloadf(ctx, m_lhs, ft0)
+    mloadf(ctx, m_rhs, ft1)
+    cfnequn ft0, ft1, t0
+    returni(ctx, t0)
+end)
+
+wasmOp(f32_lt, WasmF32Lt, macro(ctx)
+    mloadf(ctx, m_lhs, ft0)
+    mloadf(ctx, m_rhs, ft1)
+    cflt ft0, ft1, t0
+    returni(ctx, t0)
+end)
+
+wasmOp(f32_le, WasmF32Le, macro(ctx)
+    mloadf(ctx, m_lhs, ft0)
+    mloadf(ctx, m_rhs, ft1)
+    cflteq ft0, ft1, t0
+    returni(ctx, t0)
+end)
+
+wasmOp(f32_gt, WasmF32Gt, macro(ctx)
+    mloadf(ctx, m_lhs, ft0)
+    mloadf(ctx, m_rhs, ft1)
+    cfgt ft0, ft1, t0
+    returni(ctx, t0)
+end)
+
+wasmOp(f32_ge, WasmF32Ge, macro(ctx)
+    mloadf(ctx, m_lhs, ft0)
+    mloadf(ctx, m_rhs, ft1)
+    cfgteq ft0, ft1, t0
+    returni(ctx, t0)
+end)
+
+wasmOp(f64_add, WasmF64Add, macro(ctx)
+    mloadd(ctx, m_lhs, ft0)
+    mloadd(ctx, m_rhs, ft1)
+    addd ft0, ft1
+    returnd(ctx, ft1)
+end)
+
+wasmOp(f64_sub, WasmF64Sub, macro(ctx)
+    mloadd(ctx, m_lhs, ft0)
+    mloadd(ctx, m_rhs, ft1)
+    subd ft1, ft0
+    returnd(ctx, ft0)
+end)
+
+wasmOp(f64_mul, WasmF64Mul, macro(ctx)
+    mloadd(ctx, m_lhs, ft0)
+    mloadd(ctx, m_rhs, ft1)
+    muld ft0, ft1
+    returnd(ctx, ft1)
+end)
+
+wasmOp(f64_div, WasmF64Div, macro(ctx)
+    mloadd(ctx, m_lhs, ft0)
+    mloadd(ctx, m_rhs, ft1)
+    divd ft1, ft0
+    returnd(ctx, ft0)
+end)
+
+wasmOp(f64_abs, WasmF64Abs, macro(ctx)
+    mloadd(ctx, m_operand, ft0)
+    absd ft0, ft1
+    returnd(ctx, ft1)
+end)
+
+wasmOp(f64_neg, WasmF64Neg, macro(ctx)
+    mloadd(ctx, m_operand, ft0)
+    negd ft0, ft1
+    returnd(ctx, ft1)
+end)
+
+wasmOp(f64_ceil, WasmF64Ceil, macro(ctx)
+    mloadd(ctx, m_operand, ft0)
+    ceild ft0, ft1
+    returnd(ctx, ft1)
+end)
+
+wasmOp(f64_floor, WasmF64Floor, macro(ctx)
+    mloadd(ctx, m_operand, ft0)
+    floord ft0, ft1
+    returnd(ctx, ft1)
+end)
+
+wasmOp(f64_sqrt, WasmF64Sqrt, macro(ctx)
+    mloadd(ctx, m_operand, ft0)
+    sqrtd ft0, ft1
+    returnd(ctx, ft1)
+end)
+
+wasmOp(f64_eq, WasmF64Eq, macro(ctx)
+    mloadd(ctx, m_lhs, ft0)
+    mloadd(ctx, m_rhs, ft1)
+    cdeq ft0, ft1, t0
+    returni(ctx, t0)
+end)
+
+wasmOp(f64_ne, WasmF64Ne, macro(ctx)
+    mloadd(ctx, m_lhs, ft0)
+    mloadd(ctx, m_rhs, ft1)
+    cdnequn ft0, ft1, t0
+    returni(ctx, t0)
+end)
+
+wasmOp(f64_lt, WasmF64Lt, macro(ctx)
+    mloadd(ctx, m_lhs, ft0)
+    mloadd(ctx, m_rhs, ft1)
+    cdlt ft0, ft1, t0
+    returni(ctx, t0)
+end)
+
+wasmOp(f64_le, WasmF64Le, macro(ctx)
+    mloadd(ctx, m_lhs, ft0)
+    mloadd(ctx, m_rhs, ft1)
+    cdlteq ft0, ft1, t0
+    returni(ctx, t0)
+end)
+
+wasmOp(f64_gt, WasmF64Gt, macro(ctx)
+    mloadd(ctx, m_lhs, ft0)
+    mloadd(ctx, m_rhs, ft1)
+    cdgt ft0, ft1, t0
+    returni(ctx, t0)
+end)
+
+wasmOp(f64_ge, WasmF64Ge, macro(ctx)
+    mloadd(ctx, m_lhs, ft0)
+    mloadd(ctx, m_rhs, ft1)
+    cdgteq ft0, ft1, t0
+    returni(ctx, t0)
+end)
+
+wasmOp(i32_wrap_i64, WasmI32WrapI64, macro(ctx)
+    mloadq(ctx, m_operand, t0)
+    returni(ctx, t0)
+end)
+
+wasmOp(i64_extend_s_i32, WasmI64ExtendSI32, macro(ctx)
+    mloadi(ctx, m_operand, t0)
+    sxi2q t0, t1
+    returnq(ctx, t1)
+end)
+
+wasmOp(i64_extend_u_i32, WasmI64ExtendUI32, macro(ctx)
+    mloadi(ctx, m_operand, t0)
+    zxi2q t0, t1
+    returnq(ctx, t1)
+end)
+
+wasmOp(f32_convert_s_i32, WasmF32ConvertSI32, macro(ctx)
+    mloadi(ctx, m_operand, t0)
+    ci2fs t0, ft0
+    returnf(ctx, ft0)
+end)
+
+wasmOp(f32_convert_s_i64, WasmF32ConvertSI64, macro(ctx)
+    mloadq(ctx, m_operand, t0)
+    cq2fs t0, ft0
+    returnf(ctx, ft0)
+end)
+
+wasmOp(f32_demote_f64, WasmF32DemoteF64, macro(ctx)
+    mloadd(ctx, m_operand, ft0)
+    cd2f ft0, ft1
+    returnf(ctx, ft1)
+end)
+
+wasmOp(f32_reinterpret_i32, WasmF32ReinterpretI32, macro(ctx)
+    mloadi(ctx, m_operand, t0)
+    fi2f t0, ft0
+    returnf(ctx, ft0)
+end)
+
+wasmOp(f64_convert_s_i32, WasmF64ConvertSI32, macro(ctx)
+    mloadi(ctx, m_operand, t0)
+    ci2ds t0, ft0
+    returnd(ctx, ft0)
+end)
+
+wasmOp(f64_convert_s_i64, WasmF64ConvertSI64, macro(ctx)
+    mloadq(ctx, m_operand, t0)
+    cq2ds t0, ft0
+    returnd(ctx, ft0)
+end)
+
+wasmOp(f64_promote_f32, WasmF64PromoteF32, macro(ctx)
+    mloadf(ctx, m_operand, ft0)
+    cf2d ft0, ft1
+    returnd(ctx, ft1)
+end)
+
+wasmOp(f64_reinterpret_i64, WasmF64ReinterpretI64, macro(ctx)
+    mloadq(ctx, m_operand, t0)
+    fq2d t0, ft0
+    returnd(ctx, ft0)
+end)
+
+wasmOp(i32_reinterpret_f32, WasmI32ReinterpretF32, macro(ctx)
+    mloadf(ctx, m_operand, ft0)
+    ff2i ft0, t0
+    returni(ctx, t0)
+end)
+
+wasmOp(i64_reinterpret_f64, WasmI64ReinterpretF64, macro(ctx)
+    mloadd(ctx, m_operand, ft0)
+    fd2q ft0, t0
+    returnq(ctx, t0)
+end)
index 4cd8f66..6f26fb8 100644 (file)
@@ -88,9 +88,17 @@ def arm64GPRName(name, kind)
 end
 
 def arm64FPRName(name, kind)
-    raise "bad FPR kind #{kind}" unless kind == :double
     raise "bad FPR name #{name}" unless name =~ /^q/
-    "d" + name[1..-1]
+    case kind
+    when :double
+        "d" + name[1..-1]
+    when :float
+        "s" + name[1..-1]
+    when :vector
+        "v" + name[1..-1]
+    else
+        raise "bad FPR kind #{kind}"
+    end
 end
 
 class SpecialRegister
@@ -112,22 +120,26 @@ ARM64_EXTRA_FPRS = [SpecialRegister.new("q31")]
 class RegisterID
     def arm64Operand(kind)
         case @name
-        when 't0', 'a0', 'r0'
+        when 't0', 'a0', 'r0', 'wa0'
             arm64GPRName('x0', kind)
-        when 't1', 'a1', 'r1'
+        when 't1', 'a1', 'r1', 'wa1'
             arm64GPRName('x1', kind)
-        when 't2', 'a2'
+        when 't2', 'a2', 'wa2'
             arm64GPRName('x2', kind)
-        when 't3', 'a3'
+        when 't3', 'a3', 'wa3'
             arm64GPRName('x3', kind)
-        when 't4'
+        when 't4', 'wa4'
             arm64GPRName('x4', kind)
-        when 't5'
+        when 't5', 'wa5'
           arm64GPRName('x5', kind)
-        when 't6'
+        when 't6', 'wa6'
           arm64GPRName('x6', kind)
-        when 't7'
+        when 't7', 'wa7'
           arm64GPRName('x7', kind)
+        when 'ws0'
+          arm64GPRName('x9', kind)
+        when 'ws1'
+          arm64GPRName('x10', kind)
         when 'cfr'
             arm64GPRName('x29', kind)
         when 'csr0'
@@ -163,18 +175,22 @@ end
 class FPRegisterID
     def arm64Operand(kind)
         case @name
-        when 'ft0', 'fr', 'fa0'
+        when 'ft0', 'fr', 'fa0', 'wfa0'
             arm64FPRName('q0', kind)
-        when 'ft1', 'fa1'
+        when 'ft1', 'fa1', 'wfa1'
             arm64FPRName('q1', kind)
-        when 'ft2', 'fa2'
+        when 'ft2', 'fa2', 'wfa2'
             arm64FPRName('q2', kind)
-        when 'ft3', 'fa3'
+        when 'ft3', 'fa3', 'wfa3'
             arm64FPRName('q3', kind)
-        when 'ft4'
+        when 'ft4', 'wfa4'
             arm64FPRName('q4', kind)
-        when 'ft5'
+        when 'ft5', 'wfa5'
             arm64FPRName('q5', kind)
+        when 'wfa6'
+            arm64FPRName('q6', kind)
+        when 'wfa7'
+            arm64FPRName('q7', kind)
         when 'csfr0'
             arm64FPRName('q8', kind)
         when 'csfr1'
@@ -378,11 +394,11 @@ class Sequence
             case node.opcode
             when "loadb", "loadbsi", "loadbsq", "storeb", /^bb/, /^btb/, /^cb/, /^tb/
                 size = 1
-            when "loadh", "loadhsi", "loadhsq"
+            when "loadh", "loadhsi", "loadhsq", "storeh"
                 size = 2
             when "loadi", "loadis", "storei", "addi", "andi", "lshifti", "muli", "negi",
                 "noti", "ori", "rshifti", "urshifti", "subi", "xori", /^bi/, /^bti/,
-                /^ci/, /^ti/, "addis", "subis", "mulis", "smulli", "leai"
+                /^ci/, /^ti/, "addis", "subis", "mulis", "smulli", "leai", "loadf", "storef"
                 size = 4
             when "loadp", "storep", "loadq", "storeq", "loadd", "stored", "lshiftp", "lshiftq", "negp", "negq", "rshiftp", "rshiftq",
                 "urshiftp", "urshiftq", "addp", "addq", "mulp", "mulq", "andp", "andq", "orp", "orq", "subp", "subq", "xorp", "xorq", "addd",
@@ -531,6 +547,33 @@ def emitARM64TAC(opcode, operands, kind)
     $asm.puts "#{opcode} #{arm64TACOperands(operands, kind)}"
 end
 
+def emitARM64Div(opcode, operands, kind)
+    if operands.size == 2
+        operands = [operands[1], operands[1], operands[0]]
+    elsif operands.size == 3
+        operands = [operands[2], operands[1], operands[0]]
+    else
+        raise
+    end
+    $asm.puts "#{opcode} #{arm64Operands(operands, kind)}"
+end
+
+def emitARM64TACWithOperandSuffix(opcode, operands, kind)
+    raise unless [:float, :double].include? kind
+    size = kind == :float ? 8 : 16
+    operands = operands.map { |operand|
+        raise unless operand.is_a? FPRegisterID
+        "#{operand.arm64Operand(:vector)}.#{size}b"
+    }
+    if operands.length == 2
+      operands = [operands[1], operands[1], operands[0]]
+    else
+      raise unless operands.length == 3
+      operands = [operands[2], operands[0], operands[1]]
+    end
+    $asm.puts "#{opcode} #{operands.join(", ")}"
+end
+
 def emitARM64(opcode, operands, kind)
     $asm.puts "#{opcode} #{arm64FlippedOperands(operands, kind)}"
 end
@@ -581,6 +624,11 @@ def emitARM64Branch(opcode, operands, kind, branchOpcode)
     $asm.puts "#{branchOpcode} #{operands[-1].asmLabel}"
 end
 
+def emitARM64CompareFP(operands, kind, compareCode)
+    emitARM64Unflipped("fcmp", operands[0..-2], kind)
+    $asm.puts "cset #{operands[-1].arm64Operand(:word)}, #{compareCode}"
+end
+
 def emitARM64Compare(operands, kind, compareCode)
     emitARM64Unflipped("subs #{arm64GPRName('xzr', kind)}, ", operands[0..-2], kind)
     $asm.puts "csinc #{operands[-1].arm64Operand(:word)}, wzr, wzr, #{compareCode}"
@@ -637,6 +685,14 @@ class Instruction
             emitARM64TAC("eor", operands, :ptr)
         when "xorq"
             emitARM64TAC("eor", operands, :quad)
+        when 'divi'
+            emitARM64Div("udiv", operands, :word)
+        when 'divis'
+            emitARM64Div("sdiv", operands, :word)
+        when 'divq'
+            emitARM64Div("udiv", operands, :quad)
+        when 'divqs'
+            emitARM64Div("sdiv", operands, :quad)
         when "lshifti"
             emitARM64LShift(operands, :word)
         when "lshiftp"
@@ -739,8 +795,6 @@ class Instruction
             emitARM64TAC("fmul", operands, :double)
         when "sqrtd"
             emitARM64("fsqrt", operands, :double)
-        when "ci2d"
-            emitARM64("scvtf", operands, [:word, :double])
         when "bdeq"
             emitARM64Branch("fcmp", operands, :double, "b.eq")
         when "bdneq"
@@ -748,7 +802,7 @@ class Instruction
             isUnordered = LocalLabel.unique("bdneq")
             $asm.puts "b.vs #{LocalLabelReference.new(codeOrigin, isUnordered).asmLabel}"
             $asm.puts "b.ne #{operands[2].asmLabel}"
-            isUnordered.lower("ARM64")
+            isUnordered.lower($activeBackend)
         when "bdgt"
             emitARM64Branch("fcmp", operands, :double, "b.gt")
         when "bdgteq"
@@ -809,6 +863,8 @@ class Instruction
             else
                 emitARM64("mov", operands, :quad)
             end
+        when "moved"
+            emitARM64("fmov", operands, :double)
         when "sxi2p"
             emitARM64("sxtw", operands, [:word, :ptr])
         when "sxi2q"
@@ -1056,6 +1112,189 @@ class Instruction
                 $asm.puts ".loh AdrpLdrGot L_offlineasm_loh_adrp_#{uid}, L_offlineasm_loh_ldr_#{uid}"
                 $asm.putStr("#endif")
             }
+
+        when "andf", "andd"
+            emitARM64TACWithOperandSuffix("and", operands, :double)
+        when "orf", "ord"
+            emitARM64TACWithOperandSuffix("orr", operands, :double)
+        when "lrotatei"
+            tmp = Tmp.new(codeOrigin, :gpr)
+            Sequence.new(codeOrigin, [
+                Instruction.new(codeOrigin, "move", [operands[0], tmp]),
+                Instruction.new(codeOrigin, "negq", [tmp]),
+                Instruction.new(codeOrigin, "rrotatei", [tmp, operands[1]]),
+            ]).lower($activeBackend)
+        when "lrotateq"
+            tmp = Tmp.new(codeOrigin, :gpr)
+            Sequence.new(codeOrigin, [
+                Instruction.new(codeOrigin, "move", [operands[0], tmp]),
+                Instruction.new(codeOrigin, "negq", [tmp]),
+                Instruction.new(codeOrigin, "rrotateq", [tmp, operands[1]]),
+            ]).lower($activeBackend)
+        when "rrotatei"
+            emitARM64TAC("ror", operands, :word)
+        when "rrotateq"
+            emitARM64TAC("ror", operands, :quad)
+        when "loadf"
+            emitARM64Access("ldr", "ldur", operands[1], operands[0], :float)
+        when "storef"
+            emitARM64Unflipped("str", operands, :float)
+        when "addf"
+            emitARM64TAC("fadd", operands, :float)
+        when "divf"
+            emitARM64TAC("fdiv", operands, :float)
+        when "subf"
+            emitARM64TAC("fsub", operands, :float)
+        when "mulf"
+            emitARM64TAC("fmul", operands, :float)
+        when "sqrtf"
+            emitARM64("fsqrt", operands, :float)
+        when "floorf"
+            emitARM64("frintm", operands, :float)
+        when "floord"
+            emitARM64("frintm", operands, :double)
+        when "roundf"
+            emitARM64("frintn", operands, :float)
+        when "roundd"
+            emitARM64("frintn", operands, :double)
+        when "truncatef"
+            emitARM64("frintz", operands, :float)
+        when "truncated"
+            emitARM64("frintz", operands, :double)
+        when "truncatef2i"
+            emitARM64("fcvtzu", operands, [:float, :word])
+        when "truncatef2q"
+            emitARM64("fcvtzu", operands, [:float, :quad])
+        when "truncated2q"
+            emitARM64("fcvtzu", operands, [:double, :quad])
+        when "truncated2i"
+            emitARM64("fcvtzu", operands, [:double, :word])
+        when "truncatef2is"
+            emitARM64("fcvtzs", operands, [:float, :word])
+        when "truncated2is"
+            emitARM64("fcvtzs", operands, [:double, :word])
+        when "truncatef2qs"
+            emitARM64("fcvtzs", operands, [:float, :quad])
+        when "truncated2qs"
+            emitARM64("fcvtzs", operands, [:double, :quad])
+        when "ci2d"
+            emitARM64("ucvtf", operands, [:word, :double])
+        when "ci2ds"
+            emitARM64("scvtf", operands, [:word, :double])
+        when "ci2f"
+            emitARM64("ucvtf", operands, [:word, :float])
+        when "ci2fs"
+            emitARM64("scvtf", operands, [:word, :float])
+        when "cq2f"
+            emitARM64("ucvtf", operands, [:quad, :float])
+        when "cq2fs"
+            emitARM64("scvtf", operands, [:quad, :float])
+        when "cq2d"
+            emitARM64("ucvtf", operands, [:quad, :double])
+        when "cq2ds"
+            emitARM64("scvtf", operands, [:quad, :double])
+        when "cd2f"
+            emitARM64("fcvt", operands, [:double, :float])
+        when "cf2d"
+            emitARM64("fcvt", operands, [:float, :double])
+        when "bfeq"
+            emitARM64Branch("fcmp", operands, :float, "b.eq")
+        when "bfgt"
+            emitARM64Branch("fcmp", operands, :float, "b.gt")
+        when "bflt"
+            emitARM64Branch("fcmp", operands, :float, "b.mi")
+        when "bfgtun"
+            emitARM64Branch("fcmp", operands, :float, "b.hi")
+        when "bfgtequn"
+            emitARM64Branch("fcmp", operands, :float, "b.pl")
+        when "bfltun"
+            emitARM64Branch("fcmp", operands, :float, "b.lt")
+        when "bfltequn"
+            emitARM64Branch("fcmp", operands, :float, "b.le")
+        when "tzcnti"
+            emitARM64("rbit", operands, :word)
+            emitARM64("clz", [operands[1], operands[1]], :word)
+        when "tzcntq"
+            emitARM64("rbit", operands, :quad)
+            emitARM64("clz", [operands[1], operands[1]], :quad)
+        when "lzcnti"
+            emitARM64("clz", operands, :word)
+        when "lzcntq"
+            emitARM64("clz", operands, :quad)
+        when "absf"
+            emitARM64("fabs", operands, :float)
+        when "absd"
+            emitARM64("fabs", operands, :double)
+        when "negf"
+            emitARM64("fneg", operands, :float)
+        when "negd"
+            emitARM64("fneg", operands, :double)
+        when "ceilf"
+            emitARM64("frintp", operands, :float)
+        when "ceild"
+            emitARM64("frintp", operands, :double)
+        when "cfeq"
+            emitARM64CompareFP(operands, :float, "eq")
+        when "cdeq"
+            emitARM64CompareFP(operands, :double, "eq")
+        when "cfneq"
+            $asm.puts "move $0, #{operands[2].arm64Operand(:word)}"
+            emitARM64Unflipped("fcmp", operands[0..1], :float)
+            isUnordered = LocalLabel.unique("cdneq")
+            $asm.puts "b.vs #{LocalLabelReference.new(codeOrigin, isUnordered).asmLabel}"
+            $asm.puts "cset #{operands[2].arm64Operand(:word)}, ne"
+            isUnordered.lower($activeBackend)
+        when "cdneq"
+            $asm.puts "move $0, #{operands[2].arm64Operand(:word)}"
+            emitARM64Unflipped("fcmp", operands[0..1], :double)
+            isUnordered = LocalLabel.unique("cdneq")
+            $asm.puts "b.vs #{LocalLabelReference.new(codeOrigin, isUnordered).asmLabel}"
+            $asm.puts "cset #{operands[2].arm64Operand(:word)}, ne"
+            isUnordered.lower($activeBackend)
+        when "cfnequn"
+            emitARM64CompareFP(operands, :float, "ne")
+        when "cdnequn"
+            emitARM64CompareFP(operands, :double, "ne")
+        when "cflt"
+            emitARM64CompareFP(operands, :float, "mi")
+        when "cdlt"
+            emitARM64CompareFP(operands, :double, "mi")
+        when "cflteq"
+            emitARM64CompareFP(operands, :float, "ls")
+        when "cdlteq"
+            emitARM64CompareFP(operands, :double, "ls")
+        when "cfgt"
+            emitARM64CompareFP(operands, :float, "gt")
+        when "cdgt"
+            emitARM64CompareFP(operands, :double, "gt")
+        when "cfgteq"
+            emitARM64CompareFP(operands, :float, "ge")
+        when "cdgteq"
+            emitARM64CompareFP(operands, :double, "ge")
+        when "fi2f"
+            emitARM64("fmov", operands, [:word, :float])
+        when "ff2i"
+            emitARM64("fmov", operands, [:float, :word])
+        when "tls_loadp"
+            tmp = ARM64_EXTRA_GPRS[0].arm64Operand(:ptr)
+            if operands[0].immediate?
+              offset = "\##{operands[0].value * 8}"
+            else
+              offset = operands[0].arm64Operand(:word)
+            end
+            $asm.puts "mrs #{tmp}, tpidrro_el0"
+            $asm.puts "bic #{tmp}, #{tmp}, #7"
+            $asm.puts "ldr #{operands[1].arm64Operand(:ptr)}, [#{tmp}, #{offset}]"
+        when "tls_storep"
+            tmp = ARM64_EXTRA_GPRS[0].arm64Operand(:ptr)
+            if operands[1].immediate?
+              offset = "\##{operands[1].value * 8}"
+            else
+              offset = operands[1].arm64Operand(:word)
+            end
+            $asm.puts "mrs #{tmp}, tpidrro_el0"
+            $asm.puts "bic #{tmp}, #{tmp}, #7"
+            $asm.puts "str #{operands[0].arm64Operand(:ptr)}, [#{tmp}, #{offset}]"
         else
             lowerDefault
         end
index 71788b1..7a212b2 100644 (file)
@@ -33,6 +33,8 @@ MACRO_INSTRUCTIONS =
      "emit",
      "addi",
      "andi",
+     "andf",
+     "andd",
      "lshifti",
      "lshiftp",
      "lshiftq",
@@ -42,12 +44,18 @@ MACRO_INSTRUCTIONS =
      "negq",
      "noti",
      "ori",
+     "orf",
+     "ord",
      "rshifti",
      "urshifti",
      "rshiftp",
      "urshiftp",
      "rshiftq",
      "urshiftq",
+     "lrotatei",
+     "lrotateq",
+     "rrotatei",
+     "rrotateq",
      "subi",
      "xori",
      "loadi",
@@ -59,16 +67,47 @@ MACRO_INSTRUCTIONS =
      "loadhsi",
      "loadhsq",
      "storei",
+     "storeh",
      "storeb",
+     "loadf",
      "loadd",
      "moved",
+     "storef",
      "stored",
+     "addf",
      "addd",
+     "divf",
      "divd",
+     "subf",
      "subd",
+     "mulf",
      "muld",
+     "sqrtf",
      "sqrtd",
+     "floorf",
+     "floord",
+     "roundf",
+     "roundd",
+     "truncatef",
+     "truncated",
+     "truncatef2i",
+     "truncatef2q",
+     "truncated2q",
+     "truncated2i",
+     "truncatef2is",
+     "truncated2is",
+     "truncatef2qs",
+     "truncated2qs",
      "ci2d",
+     "ci2ds",
+     "ci2f",
+     "ci2fs",
+     "cq2f",
+     "cq2fs",
+     "cq2d",
+     "cq2ds",
+     "cd2f",
+     "cf2d",
      "fii2d", # usage: fii2d <gpr with least significant bits>, <gpr with most significant bits>, <fpr>
      "fd2ii", # usage: fd2ii <fpr>, <gpr with least significant bits>, <gpr with most significant bits>
      "fq2d",
@@ -85,6 +124,13 @@ MACRO_INSTRUCTIONS =
      "bdgtequn",
      "bdltun",
      "bdltequn",
+     "bfeq",
+     "bfgt",
+     "bflt",
+     "bfgtun",
+     "bfgtequn",
+     "bfltun",
+     "bfltequn",
      "btd2i",
      "td2i",
      "bcd2i",
@@ -256,12 +302,44 @@ MACRO_INSTRUCTIONS =
      "untagReturnAddress",
      "removeCodePtrTag",
      "untagArrayPtr",    
+     "tzcnti",
+     "tzcntq",
+     "lzcnti",
+     "lzcntq",
+     "absf",
+     "absd",
+     "negf",
+     "negd",
+     "ceilf",
+     "ceild",
+     "cfeq",
+     "cdeq",
+     "cfneq",
+     "cfnequn",
+     "cdneq",
+     "cdnequn",
+     "cflt",
+     "cdlt",
+     "cflteq",
+     "cdlteq",
+     "cfgt",
+     "cdgt",
+     "cfgteq",
+     "cdgteq",
+     "fi2f",
+     "ff2i",
+     "tls_loadp",
+     "tls_storep",
     ]
 
 X86_INSTRUCTIONS =
     [
      "cdqi",
-     "idivi"
+     "idivi",
+     "udivi",
+     "cqoq",
+     "idivq",
+     "udivq",
     ]
 
 ARM_INSTRUCTIONS =
@@ -276,7 +354,11 @@ ARM64_INSTRUCTIONS =
      "bfiq", # Bit field insert <source reg> <last bit written> <width immediate> <dest reg>
      "pcrtoaddr",   # Address from PC relative offset - adr instruction
      "nopFixCortexA53Err835769", # nop on Cortex-A53 (nothing otherwise)
-     "globaladdr"
+     "globaladdr",
+     "divi",
+     "divis",
+     "divq",
+     "divqs",
     ]
 
 RISC_INSTRUCTIONS =
index 632cafc..791c81a 100644 (file)
@@ -350,7 +350,7 @@ class Parser
     
     def parseVariable
         if isRegister(@tokens[@idx])
-            if @tokens[@idx] =~ FPR_PATTERN
+            if @tokens[@idx] =~ FPR_PATTERN || @tokens[@idx] =~ WASM_FPR_PATTERN
                 result = FPRegisterID.forName(@tokens[@idx].codeOrigin, @tokens[@idx].string)
             else
                 result = RegisterID.forName(@tokens[@idx].codeOrigin, @tokens[@idx].string)
index 158d141..bd77ac6 100644 (file)
@@ -79,9 +79,41 @@ FPRS =
      "fr"
     ]
 
-REGISTERS = GPRS + FPRS
+WASM_GPRS =
+    [
+     "wa0",
+     "wa1",
+     "wa2",
+     "wa3",
+     "wa4",
+     "wa5",
+     "wa6",
+     "wa7",
+    ]
+
+WASM_FPRS =
+    [
+     "wfa0",
+     "wfa1",
+     "wfa2",
+     "wfa3",
+     "wfa4",
+     "wfa5",
+     "wfa6",
+     "wfa7",
+    ]
+
+WASM_SCRATCHS =
+    [
+     "ws0",
+     "ws1",
+    ]
+
+REGISTERS = GPRS + FPRS + WASM_GPRS + WASM_FPRS + WASM_SCRATCHS
 
 GPR_PATTERN = Regexp.new('\\A((' + GPRS.join(')|(') + '))\\Z')
 FPR_PATTERN = Regexp.new('\\A((' + FPRS.join(')|(') + '))\\Z')
+WASM_GPR_PATTERN = Regexp.new('\\A((' + WASM_GPRS.join(')|(') + '))\\Z')
+WASM_FPR_PATTERN = Regexp.new('\\A((' + WASM_FPRS.join(')|(') + '))\\Z')
 
 REGISTER_PATTERN = Regexp.new('\\A((' + REGISTERS.join(')|(') + '))\\Z')
index 5a48e40..8d76711 100644 (file)
@@ -205,8 +205,11 @@ class Variable
         if @name =~ $concatenation
             name = @name.gsub($concatenation) { |match|
                 var = Variable.forName(codeOrigin, match[1...-1])
-                raise "Unknown variable `#{var.originalName}` in substitution at #{codeOrigin} - #{mapping} " unless mapping[var]
-                mapping[var].name
+                if mapping[var]
+                    mapping[var].name
+                else
+                    match
+                end
             }
             Variable.forName(codeOrigin, name)
         elsif mapping[self]
@@ -238,8 +241,11 @@ class StructOffset
         if dump =~ $concatenation
             names = dump.gsub($concatenation) { |match|
                 var = Variable.forName(codeOrigin, match[1...-1])
-                raise "Unknown variable `#{var.originalName}` in substitution at #{codeOrigin}" unless mapping[var]
-                mapping[var].name
+                if mapping[var]
+                    mapping[var].name
+                else
+                    match
+                end
             }.split('::')
             StructOffset.forField(codeOrigin, names[0..-2].join('::'), names[-1])
         else
@@ -272,8 +278,11 @@ class Label
         if @name =~ $concatenation
             name = @name.gsub($concatenation) { |match|
                 var = Variable.forName(codeOrigin, match[1...-1])
-                raise "Unknown variable `#{var.originalName}` in substitution at #{codeOrigin}" unless mapping[var]
-                mapping[var].name
+                if mapping[var]
+                    mapping[var].name
+                else
+                    match
+                end
             }
             result = Label.forName(codeOrigin, name, @definedInFile)
             result.setGlobal() if global?
@@ -306,8 +315,11 @@ class ConstExpr
         if @value =~ $concatenation
             value = @value.gsub($concatenation) { |match|
                 var = Variable.forName(codeOrigin, match[1...-1])
-                raise "Unknown variable `#{var.originalName}` in substitution at #{codeOrigin}" unless mapping[var]
-                mapping[var].name
+                if mapping[var]
+                    mapping[var].name
+                else
+                    match
+                end
             }
             ConstExpr.forName(codeOrigin, value)
         else
@@ -337,8 +349,11 @@ class Sizeof
         if struct =~ $concatenation
             value = struct.gsub($concatenation) { |match|
                 var = Variable.forName(codeOrigin, match[1...-1])
-                raise "Unknown variable `#{var.originalName}` in substitution at #{codeOrigin}" unless mapping[var]
-                mapping[var].name
+                if mapping[var]
+                    mapping[var].name
+                else
+                    match
+                end
             }
             Sizeof.forName(codeOrigin, value)
         else
@@ -357,13 +372,7 @@ class LocalLabel
     end
 end
 
-class MacroError < RuntimeError
-    attr_reader :message
-    attr_reader :backtrace
-    def initialize(message, backtrace)
-        @message = message
-        @backtrace = backtrace
-    end
+class MacroError < StandardError
 end
 
 class Sequence
@@ -399,11 +408,12 @@ class Sequence
         backtrace = @@demacroifyStack.reverse.map { |macroCall|
             "#{macroCall.codeOrigin} in call to #{macroCall.originalName}"
         }
-        raise MacroError.new(msg, backtrace)
+        raise MacroError, msg, backtrace
     end
 
     def demacroify(macros)
         myMacros = macros.dup
+        # We do an initial pass looking for all macros in order to allow forward references
         @list.each {
             | item |
             if item.is_a? Macro
@@ -414,7 +424,7 @@ class Sequence
         @list.each {
             | item |
             if item.is_a? Macro
-                # Ignore.
+                # Ignore. We already looked for macros above and they should not be part of the final output
             elsif item.is_a? MacroCall
                 @@demacroifyStack << item
                 mapping = {}
@@ -439,6 +449,7 @@ class Sequence
                     newList << Instruction.new(item.codeOrigin, "localAnnotation", [], item.annotation)
                 end
                 newList += macro.body.substitute(mapping).demacroify(myMyMacros).renameLabels(item.originalName).list
+
                 @@demacroifyStack.pop
             else
                 newList << item.demacroify(myMacros)
index 166f4bd..8fd2bde 100644 (file)
@@ -51,11 +51,11 @@ require "config"
 #  r8 => t4
 #  r9 => t5
 # r10 => t6
-# rbx =>             csr0 (callee-save, PB, unused in baseline)
-# r12 =>             csr1 (callee-save)
-# r13 =>             csr2 (callee-save)
-# r14 =>             csr3 (callee-save, numberTag)
-# r15 =>             csr4 (callee-save, notCellMask)
+# rbx =>             csr0 (callee-save, wasmInstance)
+# r12 =>             csr1 (callee-save, metadataTable)
+# r13 =>             csr2 (callee-save, PB)
+# r14 =>             csr3 (callee-save, tagTypeNumber)
+# r15 =>             csr4 (callee-save, tagMask)
 # rsp => sp
 # rbp => cfr
 # r11 =>                  (scratch)
@@ -131,8 +131,8 @@ def callPrefix
     isIntelSyntax ? "" : "*"
 end
 
-def orderOperands(opA, opB)
-    isIntelSyntax ? "#{opB}, #{opA}" : "#{opA}, #{opB}"
+def orderOperands(*operands)
+    (isIntelSyntax ? operands.reverse : operands).join(", ")
 end
 
 def const(c)
@@ -154,6 +154,8 @@ def getSizeString(kind)
         size = "dword"
     when :ptr
         size =  isX64 ? "qword" : "dword"
+    when :float
+        size = "dword"
     when :double
         size = "qword"
     when :quad
@@ -179,7 +181,7 @@ class SpecialRegister < NoChildren
         when :quad
             register(@name)
         else
-            raise
+            raise codeOriginString
         end
     end
     def x86CallOperand(kind)
@@ -254,22 +256,25 @@ class RegisterID
     def x86GPR
         if isX64
             case name
-            when "t0", "r0"
+            when "t0", "r0", "ws0"
                 "eax"
             when "r1"
                 "edx" # t1 = a1 when isWin, t2 = a2 otherwise
-            when "a0"
+            when "a0", "wa0"
                 isWin ? "ecx" : "edi"
-            when "t1", "a1"
+            when "t1", "a1", "wa1"
                 isWin ? "edx" : "esi"
-            when "t2", "a2"
+            when "t2", "a2", "wa2"
                 isWin ? "r8" : "edx"
-            when "t3", "a3"
+            when "t3", "a3", "wa3"
                 isWin ? "r9" : "ecx"
-            when "t4"
+            when "t4", "wa4"
                 isWin ? "r10" : "r8"
-            when "t5"
-                isWin ? "ecx" : "r10"
+            when "t5", "wa5"
+                isWin ? "ecx" : "r9"
+            when "t6", "ws1"
+                raise "cannot use register #{name} on X86-64 Windows" if isWin
+                "r10"
             when "csr0"
                 "ebx"
             when "csr1"
@@ -326,20 +331,24 @@ end
 
 class FPRegisterID
     def x86Operand(kind)
-        raise unless kind == :double
+        raise unless [:float, :double].include? kind
         case name
-        when "ft0", "fa0", "fr"
+        when "ft0", "fa0", "fr", "wfa0"
             register("xmm0")
-        when "ft1", "fa1"
+        when "ft1", "fa1", "wfa1"
             register("xmm1")
-        when "ft2", "fa2"
+        when "ft2", "fa2", "wfa2"
             register("xmm2")
-        when "ft3", "fa3"
+        when "ft3", "fa3", "wfa3"
             register("xmm3")
-        when "ft4"
+        when "ft4", "wfa4"
             register("xmm4")
-        when "ft5"
+        when "ft5", "wfa5"
             register("xmm5")
+        when "wfa6"
+            register("xmm6")
+        when "wfa7"
+            register("xmm7")
         else
             raise "Bad register #{name} for X86 at #{codeOriginString}"
         end
@@ -498,7 +507,7 @@ end
 class Instruction
     
     def x86Operands(*kinds)
-        raise unless kinds.size == operands.size
+        raise "Expected size of kinds to be #{operands.size}, but it was #{kinds.size}" unless kinds.size == operands.size
         result = []
         kinds.size.times {
             | idx |
@@ -528,6 +537,8 @@ class Instruction
             isX64 ? "q" : "l"
         when :quad
             isX64 ? "q" : raise
+        when :float
+            "ss"
         when :double
             "sd"
         else
@@ -547,6 +558,8 @@ class Instruction
             isX64 ? 8 : 4
         when :quad
             isX64 ? 8 : raise
+        when :float
+            4
         when :double
             8
         else
@@ -600,12 +613,12 @@ class Instruction
         end
     end
     
-    def handleX86DoubleBranch(branchOpcode, mode)
+    def handleX86FPBranch(kind, branchOpcode, mode)
         case mode
         when :normal
-            $asm.puts "ucomisd #{orderOperands(operands[1].x86Operand(:double), operands[0].x86Operand(:double))}"
+            $asm.puts "ucomi#{x86Suffix(kind)} #{orderOperands(operands[1].x86Operand(:double), operands[0].x86Operand(:double))}"
         when :reverse
-            $asm.puts "ucomisd #{orderOperands(operands[0].x86Operand(:double), operands[1].x86Operand(:double))}"
+            $asm.puts "ucomi#{x86Suffix(kind)} #{orderOperands(operands[0].x86Operand(:double), operands[1].x86Operand(:double))}"
         else
             raise mode.inspect
         end
@@ -622,6 +635,16 @@ class Instruction
         end
     end
     
+    def handleX86IntCompare(opcodeSuffix, kind)
+        if operands[0].is_a? Immediate and operands[0].value == 0 and operands[1].is_a? RegisterID and (opcodeSuffix == "e" or opcodeSuffix == "ne")
+            $asm.puts "test#{x86Suffix(kind)} #{orderOperands(operands[1].x86Operand(kind), operands[1].x86Operand(kind))}"
+        elsif operands[1].is_a? Immediate and operands[1].value == 0 and operands[0].is_a? RegisterID and (opcodeSuffix == "e" or opcodeSuffix == "ne")
+            $asm.puts "test#{x86Suffix(kind)}  #{orderOperands(operands[0].x86Operand(kind), operands[0].x86Operand(kind))}"
+        else
+            $asm.puts "cmp#{x86Suffix(kind)} #{orderOperands(operands[1].x86Operand(kind), operands[0].x86Operand(kind))}"
+        end
+    end
+
     def handleX86IntBranch(branchOpcode, kind)
         handleX86IntCompare(branchOpcode[1..-1], kind)
         $asm.puts "#{branchOpcode} #{operands[2].asmLabel}"
@@ -652,6 +675,59 @@ class Instruction
         handleX86IntCompare(setOpcode[3..-1], kind)
         handleX86Set(setOpcode, operands[2])
     end
+
+    def handleX86FPCompareSet(kind, setOpcode, order = :normal)
+        is_special = setOpcode.is_a? Symbol
+        left = operands[0]
+        right = operands[1]
+        target = operands[2]
+
+        compare = lambda do |lhs, rhs|
+            $asm.puts "ucomi#{x86Suffix(kind)} #{orderOperands(lhs.x86Operand(:double), rhs.x86Operand(:double))}"
+        end
+
+        if is_special
+            case setOpcode
+            when :eq
+                if left == right
+                    compare.call(right, left)
+                    handleX86Set("setnp", operands[2])
+                    return
+                end
+
+                isUnordered = LocalLabel.unique("isUnordered")
+                $asm.puts "movq $0, #{target.x86Operand(:quad)}"
+                compare.call(right, left)
+                $asm.puts "jp #{LocalLabelReference.new(codeOrigin, isUnordered).asmLabel}"
+                handleX86Set("sete", target)
+                isUnordered.lower($activeBackend)
+                return
+            when :nequn
+                if left == right
+                    compare.call(right, left)
+                    handleX86Set("setp", target)
+                    return
+                end
+
+                isUnordered = LocalLabel.unique("isUnordered")
+                $asm.puts "movq $1, #{target.x86Operand(:quad)}"
+                compare.call(right, left);
+                $asm.puts "jp #{LocalLabelReference.new(codeOrigin, isUnordered).asmLabel}"
+                handleX86Set("setne", target)
+                isUnordered.lower($activeBackend)
+                return
+            else
+                raise "Uhandled special opcode: #{setOpcode}"
+            end
+        end
+
+        if order == :normal
+            compare.call(right, left)
+        else
+            compare.call(left, right)
+        end
+        handleX86Set(setOpcode, target)
+    end
     
     def handleX86Test(kind)
         value = operands[0]
@@ -749,7 +825,7 @@ class Instruction
             end
         end
     end
-    
+
     def handleX86Sub(kind)
         if operands.size == 3
             if Immediate.new(nil, 0) == operands[1]
@@ -795,6 +871,46 @@ class Instruction
         handleX86Op("imul#{x86Suffix(kind)}", kind)
     end
     
+    def handleX86AddFP(kind)
+        if operands.size == 2
+            $asm.puts "add#{x86Suffix(kind)} #{orderOperands(operands[0].x86Operand(kind), operands[1].x86Operand(kind))}"
+        elsif operands.size == 3
+            $asm.puts "vadd#{x86Suffix(kind)} #{orderOperands(operands[1].x86Operand(kind), operands[0].x86Operand(kind), operands[2].x86Operand(kind))}"
+        else
+            raise "Unexpected number of operands for floating point addition: #{operands.size}"
+        end
+    end
+
+    def handleX86SubFP(kind)
+        if operands.size == 2
+            $asm.puts "sub#{x86Suffix(kind)} #{orderOperands(operands[0].x86Operand(kind), operands[1].x86Operand(kind))}"
+        elsif operands.size == 3
+            $asm.puts "vsub#{x86Suffix(kind)} #{orderOperands(operands[1].x86Operand(kind), operands[0].x86Operand(kind), operands[2].x86Operand(kind))}"
+        else
+            raise "Unexpected number of operands for floating point addition: #{operands.size}"
+        end
+    end
+
+    def handleX86MulFP(kind)
+        if operands.size == 2
+            $asm.puts "mul#{x86Suffix(kind)} #{orderOperands(operands[0].x86Operand(kind), operands[1].x86Operand(kind))}"
+        elsif operands.size == 3
+            $asm.puts "vmul#{x86Suffix(kind)} #{orderOperands(operands[1].x86Operand(kind), operands[0].x86Operand(kind), operands[2].x86Operand(kind))}"
+        else
+            raise "Unexpected number of operands for floating point addition: #{operands.size}"
+        end
+    end
+
+    def handleX86DivFP(kind)
+        if operands.size == 2
+            $asm.puts "div#{x86Suffix(kind)} #{orderOperands(operands[0].x86Operand(kind), operands[1].x86Operand(kind))}"
+        elsif operands.size == 3
+            $asm.puts "vdiv#{x86Suffix(kind)} #{orderOperands(operands[1].x86Operand(kind), operands[0].x86Operand(kind), operands[2].x86Operand(kind))}"
+        else
+            raise "Unexpected number of operands for floating point addition: #{operands.size}"
+        end
+    end
+
     def handleX86Peek()
         sp = RegisterID.new(nil, "sp")
         opA = offsetRegister(operands[0].value * x86Bytes(:ptr), sp.x86Operand(:ptr))
@@ -825,6 +941,120 @@ class Instruction
         end
     end
 
+    def countLeadingZeros(kind)
+        target = operands[1]
+        srcIsNonZero = LocalLabel.unique("srcIsNonZero")
+        skipNonZeroCase = LocalLabel.unique("skipNonZeroCase")
+        zeroValue = Immediate.new(codeOrigin, x86Bytes(kind) * 8)
+        xorValue = Immediate.new(codeOrigin, kind == :quad ? 0x3f : 0x1f)
+        xor = kind == :quad ? "xorq" : "xori"
+
+        $asm.puts "bsr#{x86Suffix(kind)} #{x86Operands(kind, kind)}"
+
+        Sequence.new(codeOrigin, [
+            Instruction.new(codeOrigin, "bnz", [LocalLabelReference.new(codeOrigin, srcIsNonZero)]),
+            Instruction.new(codeOrigin, "move", [zeroValue, target]),
+            Instruction.new(codeOrigin, "jmp", [LocalLabelReference.new(codeOrigin, skipNonZeroCase)]),
+
+            srcIsNonZero,
+            Instruction.new(codeOrigin, xor, [xorValue, target]),
+
+            skipNonZeroCase,
+        ]).lower($activeBackend)
+    end
+
+    def countTrailingZeros(kind)
+        target = operands[1]
+        srcIsNonZero = LocalLabel.unique("srcIsNonZero")
+        zeroValue = Immediate.new(codeOrigin, x86Bytes(kind) * 8)
+
+        $asm.puts "bsf#{x86Suffix(kind)} #{x86Operands(kind, kind)}"
+
+        Sequence.new(codeOrigin, [
+            Instruction.new(codeOrigin, "bnz", [LocalLabelReference.new(codeOrigin, srcIsNonZero)]),
+            Instruction.new(codeOrigin, "move", [zeroValue, target]),
+            srcIsNonZero,
+        ]).lower($activeBackend)
+    end
+
+    def truncateFloatingPointToQuad(kind)
+        src = operands[0]
+        dst = operands[1]
+        slow = LocalLabel.unique("slow")
+        done = LocalLabel.unique("done")
+        gprScratch = X64_SCRATCH_REGISTER
+        fprScratch = FPRegisterID.forName(codeOrigin, "wfa7")
+        int64SignBit = Immediate.new(codeOrigin, 0x8000000000000000)
+        case kind
+        when :float
+            int64Min = Immediate.new(codeOrigin, 0xdf000000)
+            negInt64Min = Immediate.new(codeOrigin, 0x5f000000)
+            integerSuffix = "i"
+            floatingSuffix = "f"
+        when :double
+            int64Min = Immediate.new(codeOrigin, 0xc3e0000000000000)
+            negInt64Min = Immediate.new(codeOrigin, 0x43e0000000000000)
+            integerSuffix = "q"
+            floatingSuffix = "d"
+        else
+            raise
+        end
+
+        Sequence.new(codeOrigin, [
+            Instruction.new(codeOrigin, "move", [negInt64Min, gprScratch]),
+            Instruction.new(codeOrigin, "f#{integerSuffix}2#{floatingSuffix}", [gprScratch, fprScratch]),
+            Instruction.new(codeOrigin, "b#{floatingSuffix}gteq", [src, fprScratch, LocalLabelReference.new(codeOrigin, slow)]),
+            Instruction.new(codeOrigin, "truncate#{floatingSuffix}2qs", [src, dst]),
+            Instruction.new(codeOrigin, "jmp", [LocalLabelReference.new(codeOrigin, done)]),
+
+            slow,
+            Instruction.new(codeOrigin, "move", [int64Min, gprScratch]),
+            Instruction.new(codeOrigin, "f#{integerSuffix}2#{floatingSuffix}", [gprScratch, fprScratch]),
+            Instruction.new(codeOrigin, "add#{floatingSuffix}", [src, fprScratch]),
+            Instruction.new(codeOrigin, "truncate#{floatingSuffix}2qs", [fprScratch, dst]),
+            Instruction.new(codeOrigin, "move", [int64SignBit, gprScratch]),
+            Instruction.new(codeOrigin, "orq", [gprScratch, dst]),
+
+            done,
+        ]).lower($activeBackend)
+    end
+
+    def convertQuadToFloatingPoint(kind)
+        src = operands[0]
+        scratch1 = operands[1]
+        dst = operands[2]
+        slow = LocalLabel.unique("slow")
+        done = LocalLabel.unique("done")
+        scratch2 = X64_SCRATCH_REGISTER
+        one = Immediate.new(codeOrigin, 0x1)
+
+        case kind
+        when :float
+            floatingSuffix = "f"
+        when :double
+            floatingSuffix = "d"
+        else
+            raise
+        end
+
+        Sequence.new(codeOrigin, [
+            Instruction.new(codeOrigin, "btqs", [src, LocalLabelReference.new(codeOrigin, slow)]),
+            Instruction.new(codeOrigin, "cq2#{floatingSuffix}s", [src, dst]),
+            Instruction.new(codeOrigin, "jmp", [LocalLabelReference.new(codeOrigin, done)]),
+
+            slow,
+            Instruction.new(codeOrigin, "move", [src, scratch1]),
+            Instruction.new(codeOrigin, "move", [src, scratch2]),
+            Instruction.new(codeOrigin, "urshiftq", [one, scratch1]),
+            Instruction.new(codeOrigin, "andq", [one, scratch2]),
+            Instruction.new(codeOrigin, "orq", [scratch1, scratch2]),
+            Instruction.new(codeOrigin, "cq2#{floatingSuffix}s", [scratch2, dst]),
+            Instruction.new(codeOrigin, "add#{floatingSuffix}", [dst, dst]),
+
+            done,
+        ]).lower($activeBackend)
+    end
+
     def lowerX86
         raise unless $activeBackend == "X86"
         lowerX86Common
@@ -859,6 +1089,10 @@ class Instruction
             handleX86Op("and#{x86Suffix(:ptr)}", :ptr)
         when "andq"
             handleX86Op("and#{x86Suffix(:quad)}", :quad)
+        when "andf"
+            handleX86Op("andps", :float)
+        when "andd"
+            handleX86Op("andpd", :double)
         when "lshifti"
             handleX86Shift("sal#{x86Suffix(:int)}", :int)
         when "lshiftp"
@@ -885,6 +1119,10 @@ class Instruction
             handleX86Op("or#{x86Suffix(:ptr)}", :ptr)
         when "orq"
             handleX86Op("or#{x86Suffix(:quad)}", :quad)
+        when "orf"
+            handleX86Op("orps", :float)
+        when "ord"
+            handleX86Op("orpd", :double)
         when "rshifti"
             handleX86Shift("sar#{x86Suffix(:int)}", :int)
         when "rshiftp"
@@ -897,6 +1135,14 @@ class Instruction
             handleX86Shift("shr#{x86Suffix(:ptr)}", :ptr)
         when "urshiftq"
             handleX86Shift("shr#{x86Suffix(:quad)}", :quad)
+        when "rrotatei"
+            handleX86Shift("ror#{x86Suffix(:int)}", :int)
+        when "rrotateq"
+            handleX86Shift("ror#{x86Suffix(:quad)}", :quad)
+        when "lrotatei"
+            handleX86Shift("rol#{x86Suffix(:int)}", :int)
+        when "lrotateq"
+            handleX86Shift("rol#{x86Suffix(:quad)}", :quad)
         when "subi"
             handleX86Sub(:int)
         when "subp"
@@ -971,24 +1217,90 @@ class Instruction
             end
         when "storeb"
             $asm.puts "mov#{x86Suffix(:byte)} #{x86Operands(:byte, :byte)}"
+        when "storeh"
+            $asm.puts "mov#{x86Suffix(:half)} #{x86Operands(:half, :half)}"
+        when "loadf"
+            $asm.puts "movss #{x86Operands(:float, :float)}"
         when "loadd"
             $asm.puts "movsd #{x86Operands(:double, :double)}"
         when "moved"
             $asm.puts "movsd #{x86Operands(:double, :double)}"
+        when "storef"
+            $asm.puts "movss #{x86Operands(:float, :float)}"
         when "stored"
             $asm.puts "movsd #{x86Operands(:double, :double)}"
+        when "addf"
+            handleX86AddFP(:float)
         when "addd"
-            $asm.puts "addsd #{x86Operands(:double, :double)}"
+            handleX86AddFP(:double)
+        when "mulf"
+            handleX86MulFP(:float)
         when "muld"
-            $asm.puts "mulsd #{x86Operands(:double, :double)}"
+            handleX86MulFP(:double)
+        when "subf"
+            handleX86SubFP(:float)
         when "subd"
-            $asm.puts "subsd #{x86Operands(:double, :double)}"
+            handleX86SubFP(:double)
+        when "divf"
+            handleX86DivFP(:float)
         when "divd"
-            $asm.puts "divsd #{x86Operands(:double, :double)}"
+            handleX86DivFP(:double)
+        when "sqrtf"
+            $asm.puts "sqrtss #{operands[0].x86Operand(:float)}, #{operands[1].x86Operand(:float)}"
         when "sqrtd"
             $asm.puts "sqrtsd #{operands[0].x86Operand(:double)}, #{operands[1].x86Operand(:double)}"
+        when "roundf"
+            $asm.puts "roundss $0, #{operands[0].x86Operand(:double)}, #{operands[1].x86Operand(:double)}"
+        when "roundd"
+            $asm.puts "roundsd $0, #{operands[0].x86Operand(:double)}, #{operands[1].x86Operand(:double)}"
+        when "floorf"
+            $asm.puts "roundss $1, #{operands[0].x86Operand(:double)}, #{operands[1].x86Operand(:double)}"
+        when "floord"
+            $asm.puts "roundsd $1, #{operands[0].x86Operand(:double)}, #{operands[1].x86Operand(:double)}"
+        when "ceilf"
+            $asm.puts "roundss $2, #{operands[0].x86Operand(:double)}, #{operands[1].x86Operand(:double)}"
+        when "ceild"
+            $asm.puts "roundsd $2, #{operands[0].x86Operand(:double)}, #{operands[1].x86Operand(:double)}"
+        when "truncatef"
+            $asm.puts "roundss $3, #{operands[0].x86Operand(:double)}, #{operands[1].x86Operand(:double)}"
+        when "truncated"
+            $asm.puts "roundsd $3, #{operands[0].x86Operand(:double)}, #{operands[1].x86Operand(:double)}"
+        when "truncatef2i"
+            $asm.puts "cvttss2si #{operands[0].x86Operand(:float)}, #{operands[1].x86Operand(:quad)}"
+        when "truncated2i"
+            $asm.puts "cvttsd2si #{operands[0].x86Operand(:double)}, #{operands[1].x86Operand(:quad)}"
+        when "truncatef2q"
+            truncateFloatingPointToQuad(:float)
+        when "truncated2q"
+            truncateFloatingPointToQuad(:double)
+        when "truncatef2is"
+            $asm.puts "cvttss2si #{operands[0].x86Operand(:float)}, #{operands[1].x86Operand(:int)}"
+        when "truncatef2qs"
+            $asm.puts "cvttss2si #{operands[0].x86Operand(:float)}, #{operands[1].x86Operand(:quad)}"
+        when "truncated2is"
+            $asm.puts "cvttsd2si #{operands[0].x86Operand(:double)}, #{operands[1].x86Operand(:int)}"
+        when "truncated2qs"
+            $asm.puts "cvttsd2si #{operands[0].x86Operand(:double)}, #{operands[1].x86Operand(:quad)}"
         when "ci2d"
+            $asm.puts "cvtsi2sd #{orderOperands(operands[0].x86Operand(:quad), operands[1].x86Operand(:double))}"
+        when "ci2ds"
             $asm.puts "cvtsi2sd #{orderOperands(operands[0].x86Operand(:int), operands[1].x86Operand(:double))}"
+        when "ci2fs"
+            $asm.puts "cvtsi2ss #{orderOperands(operands[0].x86Operand(:int), operands[1].x86Operand(:float))}"
+        when "ci2f"
+            $asm.puts "cvtsi2ss #{orderOperands(operands[0].x86Operand(:quad), operands[1].x86Operand(:float))}"
+        when "cq2f"
+            convertQuadToFloatingPoint(:float)
+        when "cq2d"
+            convertQuadToFloatingPoint(:double)
+        when "cq2fs"
+            $asm.puts "cvtsi2ssq #{orderOperands(operands[0].x86Operand(:quad), operands[1].x86Operand(:float))}"
+        when "cq2ds"
+            $asm.puts "cvtsi2sdq #{orderOperands(operands[0].x86Operand(:quad), operands[1].x86Operand(:double))}"
+        when "cd2f"
+            $asm.puts "cvtsd2ss #{x86Operands(:double, :float)}"
+        when "cf2d"
+            $asm.puts "cvtss2sd #{x86Operands(:float, :double)}"
         when "bdeq"
             $asm.puts "ucomisd #{orderOperands(operands[0].x86Operand(:double), operands[1].x86Operand(:double))}"
             if operands[0] == operands[1]
@@ -996,22 +1308,22 @@ class Instruction
                 $asm.puts "jnp #{operands[2].asmLabel}"
             else
                 isUnordered = LocalLabel.unique("bdeq")
-                $asm.puts "jp #{LabelReference.new(codeOrigin, isUnordered).asmLabel}"
-                $asm.puts "je #{LabelReference.new(codeOrigin, operands[2]).asmLabel}"
-                isUnordered.lower("X86")
+                $asm.puts "jp #{LocalLabelReference.new(codeOrigin, isUnordered).asmLabel}"
+                $asm.puts "je #{LocalLabelReference.new(codeOrigin, operands[2]).asmLabel}"
+                isUnordered.lower($activeBackend)
             end
         when "bdneq"
-            handleX86DoubleBranch("jne", :normal)
+            handleX86FPBranch(:double, "jne", :normal)
         when "bdgt"
-            handleX86DoubleBranch("ja", :normal)
+            handleX86FPBranch(:double, "ja", :normal)
         when "bdgteq"
-            handleX86DoubleBranch("jae", :normal)
+            handleX86FPBranch(:double, "jae", :normal)
         when "bdlt"
-            handleX86DoubleBranch("ja", :reverse)
+            handleX86FPBranch(:double, "ja", :reverse)
         when "bdlteq"
-            handleX86DoubleBranch("jae", :reverse)
+            handleX86FPBranch(:double, "jae", :reverse)
         when "bdequn"
-            handleX86DoubleBranch("je", :normal)
+            handleX86FPBranch(:double, "je", :normal)
         when "bdnequn"
             $asm.puts "ucomisd #{orderOperands(operands[0].x86Operand(:double), operands[1].x86Operand(:double))}"
             if operands[0] == operands[1]
@@ -1020,20 +1332,45 @@ class Instruction
             else
                 isUnordered = LocalLabel.unique("bdnequn")
                 isEqual = LocalLabel.unique("bdnequn")
-                $asm.puts "jp #{LabelReference.new(codeOrigin, isUnordered).asmLabel}"
-                $asm.puts "je #{LabelReference.new(codeOrigin, isEqual).asmLabel}"
-                isUnordered.lower("X86")
+                $asm.puts "jp #{LocalLabelReference.new(codeOrigin, isUnordered).asmLabel}"
+                $asm.puts "je #{LocalLabelReference.new(codeOrigin, isEqual).asmLabel}"
+                isUnordered.lower($activeBackend)
                 $asm.puts "jmp #{operands[2].asmLabel}"
-                isEqual.lower("X86")
+                isEqual.lower($activeBackend)
             end
         when "bdgtun"
-            handleX86DoubleBranch("jb", :reverse)
+            handleX86FPBranch(:double, "jb", :reverse)
         when "bdgtequn"
-            handleX86DoubleBranch("jbe", :reverse)
+            handleX86FPBranch(:double, "jbe", :reverse)
         when "bdltun"
-            handleX86DoubleBranch("jb", :normal)
+            handleX86FPBranch(:double, "jb", :normal)
         when "bdltequn"
-            handleX86DoubleBranch("jbe", :normal)
+            handleX86FPBranch(:double, "jbe", :normal)
+        when "bfeq"
+            $asm.puts "ucomiss #{orderOperands(operands[0].x86Operand(:float), operands[1].x86Operand(:float))}"
+            if operands[0] == operands[1]
+                # This is just a jump ordered, which is a jnp.
+                $asm.puts "jnp #{operands[2].asmLabel}"
+            else
+                isUnordered = LocalLabel.unique("bfeq")
+                $asm.puts "jp #{LocalLabelReference.new(codeOrigin, isUnordered).asmLabel}"
+                $asm.puts "je #{LocalLabelReference.new(codeOrigin, operands[2]).asmLabel}"
+                isUnordered.lower($activeBackend)
+            end
+        when "bfgt"
+            handleX86FPBranch(:float, "ja", :normal)
+        when "bfgteq"
+            handleX86FPBranch(:float, "jae", :normal)
+        when "bflt"
+            handleX86FPBranch(:float, "ja", :reverse)
+        when "bfgtun"
+            handleX86FPBranch(:float, "jb", :reverse)
+        when "bfgtequn"
+            handleX86FPBranch(:float, "jbe", :reverse)
+        when "bfltun"
+            handleX86FPBranch(:float, "jb", :normal)
+        when "bfltequn"
+            handleX86FPBranch(:float, "jbe", :normal)
         when "btd2i"
             $asm.puts "cvttsd2si #{operands[0].x86Operand(:double)}, #{operands[1].x86Operand(:int)}"
             $asm.puts "cmpl $0x80000000 #{operands[1].x86Operand(:int)}"
@@ -1316,6 +1653,34 @@ class Instruction
             handleX86IntCompareSet("setle", :ptr)
         when "cqlteq"
             handleX86IntCompareSet("setle", :quad)
+        when "cfeq"
+            handleX86FPCompareSet(:float, :eq)
+        when "cdeq"
+            handleX86FPCompareSet(:double, :eq)
+        when "cfneq"
+            handleX86FPCompareSet(:float, "setne")
+        when "cdneq"
+            handleX86FPCompareSet(:double, "setne")
+        when "cfnequn"
+            handleX86FPCompareSet(:float, :nequn)
+        when "cdnequn"
+            handleX86FPCompareSet(:double, :nequn)
+        when "cfgt"
+            handleX86FPCompareSet(:float, "seta")
+        when "cdgt"
+            handleX86FPCompareSet(:double, "seta")
+        when "cfgteq"
+            handleX86FPCompareSet(:float, "setae")
+        when "cdgteq"
+            handleX86FPCompareSet(:double, "setae")
+        when "cflt"
+            handleX86FPCompareSet(:float, "seta", :reverse)
+        when "cdlt"
+            handleX86FPCompareSet(:double, "seta", :reverse)
+        when "cflteq"
+            handleX86FPCompareSet(:float, "setae", :reverse)
+        when "cdlteq"
+            handleX86FPCompareSet(:double, "setae", :reverse)
         when "tis"
             handleX86SetTest("sets", :int)
         when "tiz"
@@ -1346,8 +1711,28 @@ class Instruction
             handleX86Poke()
         when "cdqi"
             $asm.puts "cdq"
+        when "cqoq"
+            $asm.puts "cqo"
         when "idivi"
             $asm.puts "idiv#{x86Suffix(:int)} #{operands[0].x86Operand(:int)}"
+        when "udivi"
+            $asm.puts "div#{x86Suffix(:int)} #{operands[0].x86Operand(:int)}"
+        when "idivq"
+            $asm.puts "idiv#{x86Suffix(:quad)} #{operands[0].x86Operand(:quad)}"
+        when "udivq"
+            $asm.puts "div#{x86Suffix(:quad)} #{operands[0].x86Operand(:quad)}"
+        when "popcnti"
+            $asm.puts "popcnt#{x86Suffix(:int)} #{x86Operands(:int, :int)}"
+        when "popcntq"
+            $asm.puts "popcnt#{x86Suffix(:quad)} #{x86Operands(:quad, :quad)}"
+        when "tzcnti"
+            countTrailingZeros(:int)
+        when "tzcntq"
+            countTrailingZeros(:quad)
+        when "lzcnti"
+            countLeadingZeros(:int)
+        when "lzcntq"
+            countLeadingZeros(:quad)
         when "fii2d"
             $asm.puts "movd #{operands[0].x86Operand(:int)}, #{operands[2].x86Operand(:double)}"
             $asm.puts "movd #{operands[1].x86Operand(:int)}, %xmm7"
@@ -1374,6 +1759,10 @@ class Instruction
                 # Debugging shows that movd actually moves a qword when using MASM.
                 $asm.puts "movd #{operands[1].x86Operand(:quad)}, #{operands[0].x86Operand(:double)}"
             end
+        when "fi2f"
+            $asm.puts "movd #{x86Operands(:int, :float)}"
+        when "ff2i"
+            $asm.puts "movd #{x86Operands(:float, :int)}"
         when "bo"
             $asm.puts "jo #{operands[0].asmLabel}"
         when "bs"
@@ -1393,6 +1782,46 @@ class Instruction
             else
                 $asm.puts "lock; orl $0, (#{sp.x86Operand(:ptr)})"
             end
+        when "absf"
+            $asm.puts "movl #{orderOperands("$0x80000000", X64_SCRATCH_REGISTER.x86Operand(:int))}"
+            $asm.puts "movd #{orderOperands(X64_SCRATCH_REGISTER.x86Operand(:int), operands[1].x86Operand(:float))}"
+            $asm.puts "andnps #{orderOperands(operands[0].x86Operand(:float), operands[1].x86Operand(:float))}"
+        when "absd"
+            $asm.puts "movq #{orderOperands("$0x8000000000000000", X64_SCRATCH_REGISTER.x86Operand(:quad))}"
+            $asm.puts "movd #{orderOperands(X64_SCRATCH_REGISTER.x86Operand(:quad), operands[1].x86Operand(:double))}"
+            $asm.puts "andnps #{orderOperands(operands[0].x86Operand(:double), operands[1].x86Operand(:double))}"
+        when "negf"
+            $asm.puts "movl #{orderOperands("$0x80000000", X64_SCRATCH_REGISTER.x86Operand(:int))}"
+            $asm.puts "movd #{orderOperands(X64_SCRATCH_REGISTER.x86Operand(:int), operands[1].x86Operand(:float))}"
+            $asm.puts "xorps #{orderOperands(operands[0].x86Operand(:float), operands[1].x86Operand(:float))}"
+        when "negd"
+            $asm.puts "movq #{orderOperands("$0x8000000000000000", X64_SCRATCH_REGISTER.x86Operand(:quad))}"
+            $asm.puts "movd #{orderOperands(X64_SCRATCH_REGISTER.x86Operand(:quad), operands[1].x86Operand(:double))}"
+            $asm.puts "xorpd #{orderOperands(operands[0].x86Operand(:double), operands[1].x86Operand(:double))}"
+        when "tls_loadp"
+            raise "tls_loadp is only supported on x64" unless isX64
+            if operands[0].immediate?
+                mem = "%gs:#{operands[0].value * 8}"
+            else
+                mem = BaseIndex.new(codeOrigin, nil, operands[0], 8, "%gs:").x86AddressOperand(:quad)
+            end
+            $asm.puts "movq #{orderOperands(mem, operands[1].x86Operand(:quad))}"
+        when "tls_loadp"
+            raise "tls_loadp is only supported on x64" unless isX64
+            if operands[0].immediate?
+                mem = "%gs:#{operands[0].value * x86Bytes(:ptr)}"
+            else
+                mem = BaseIndex.new(codeOrigin, nil, operands[0], x86Bytes(:ptr), "%gs:").x86AddressOperand(:quad)
+            end
+            $asm.puts "mov#{x86Suffix(:ptr)} #{orderOperands(mem, operands[1].x86Operand(:quad))}"
+        when "tls_storep"
+            raise "tls_loadp is only supported on x64" unless isX64
+            if operands[1].immediate?
+                mem = "%gs:#{operands[1].value * x86Bytes(:ptr)}"
+            else
+                mem = BaseIndex.new(codeOrigin, nil, operands[1], x86Bytes(:ptr), "%gs:").x86AddressOperand(:ptr)
+            end
+            $asm.puts "mov#{x86Suffix(:ptr)} #{orderOperands(operands[0].x86Operand(:ptr), mem)}"
         else
             lowerDefault
         end
index 9c3032a..aa44866 100644 (file)
@@ -27,6 +27,7 @@
 
 #include "BytecodeIntrinsicRegistry.h"
 #include "JITCode.h"
+#include "Label.h"
 #include "ParserArena.h"
 #include "ParserModes.h"
 #include "ParserTokens.h"
@@ -45,7 +46,6 @@ namespace JSC {
     class BytecodeGenerator;
     class FunctionMetadataNode;
     class FunctionParameters;
-    class Label;
     class ModuleAnalyzer;
     class ModuleScopeData;
     class PropertyListNode;
index 9f399dc..a1bccbd 100644 (file)
@@ -134,7 +134,7 @@ public:
             m_foundStartCallFrame = true;
 
         if (m_foundStartCallFrame) {
-            if (visitor->callFrame()->codeBlock()) {
+            if (!visitor->isWasmFrame() && visitor->callFrame()->codeBlock()) {
                 m_foundCallFrame = visitor->callFrame();
                 return StackVisitor::Done;
             }
index 62192bc..0c2def3 100644 (file)
@@ -127,10 +127,8 @@ void ErrorInstance::finishCreation(JSGlobalObject* globalObject, VM& vm, const S
         BytecodeIndex bytecodeIndex;
         CallFrame* callFrame;
         getBytecodeIndex(vm, vm.topCallFrame, m_stackTrace.get(), callFrame, bytecodeIndex);
-        if (callFrame && callFrame->codeBlock()) {
-            ASSERT(!callFrame->callee().isWasm());
+        if (callFrame && callFrame->codeBlock() && !callFrame->callee().isWasm())
             appendSourceToError(globalObject, callFrame, this, bytecodeIndex);
-        }
     }
 }
 
index 5dafe81..40ad218 100644 (file)
@@ -359,6 +359,11 @@ static void overrideDefaults()
 #if !HAVE(MACH_EXCEPTIONS)
     Options::useMachForExceptions() = false;
 #endif
+
+    if (Options::useWasmLLInt()) {
+        Options::thresholdForOMGOptimizeAfterWarmUp() = 1500;
+        Options::thresholdForOMGOptimizeSoon() = 100;
+    }
 }
 
 static void correctOptions()
index 461f038..5a25684 100644 (file)
@@ -441,6 +441,7 @@ namespace JSC {
     v(Unsigned, maxNumWebAssemblyFastMemories, 4, Normal, nullptr) \
     v(Bool, useFastTLSForWasmContext, true, Normal, "If true, we will store context in fast TLS. If false, we will pin it to a register.") \
     v(Bool, wasmBBQUsesAir, true, Normal, nullptr) \
+    v(Bool, useWasmLLInt, true, Normal, nullptr) \
     v(Size, webAssemblyBBQAirModeThreshold, isIOS() ? (10 * MB) : 0, Normal, "If 0, we always use BBQ Air. If Wasm module code size hits this threshold, we compile Wasm module with B3 BBQ mode.") \
     v(Bool, useWebAssemblyStreamingApi, enableWebAssemblyStreamingApi, Normal, "Allow to run WebAssembly's Streaming API") \
     v(Bool, useEagerWebAssemblyModuleHashing, false, Normal, "Unnamed WebAssembly modules are identified in backtraces through their hash, if available.") \
index 9bcf961..c2fb081 100644 (file)
@@ -123,6 +123,8 @@ protected:
         CallSiteIndex callSiteIndex;
         CalleeBits unsafeCallee = m_callFrame->unsafeCallee();
         CodeBlock* codeBlock = m_callFrame->unsafeCodeBlock();
+        if (unsafeCallee.isWasm())
+            codeBlock = nullptr;