github
/
ruby
зеркало из https://github.com/github/ruby.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
ruby/yjit_asm.h

409 строки
18 KiB
C
Исходник Обычный вид История

Yet Another Ruby JIT! Renaming uJIT to YJIT. AKA s/ujit/yjit/g.
2021-03-07 02:46:56 +03:00
#ifndef YJIT_ASM_H
#define YJIT_ASM_H 1
Progress on porting x86 assembler for MicroJIT
2020-09-04 22:56:00 +03:00
#include <stdint.h>
#include <stddef.h>
#include <stdbool.h>
// Maximum number of labels to link
#define MAX_LABELS 32
// Maximum number of label references
#define MAX_LABEL_REFS 32
Progress on x86 assembler. Encode a few simple instructions.
2020-09-08 23:45:35 +03:00
// Reference to an ASM label
Progress on porting x86 assembler for MicroJIT
2020-09-04 22:56:00 +03:00
typedef struct LabelRef
{
Progress on x86 assembler. Encode a few simple instructions.
2020-09-08 23:45:35 +03:00
// Position in the code block where the label reference exists
Introduce version_t struct. Will be needed for code invalidation.
2021-01-12 22:56:43 +03:00
uint32_t pos;
Progress on porting x86 assembler for MicroJIT
2020-09-04 22:56:00 +03:00
// Label which this refers to
Introduce version_t struct. Will be needed for code invalidation.
2021-01-12 22:56:43 +03:00
uint32_t label_idx;
Progress on porting x86 assembler for MicroJIT
2020-09-04 22:56:00 +03:00
} labelref_t;
Progress on x86 assembler. Encode a few simple instructions.
2020-09-08 23:45:35 +03:00
// Block of executable memory into which instructions can be written
Progress on porting x86 assembler for MicroJIT
2020-09-04 22:56:00 +03:00
typedef struct CodeBlock
{
// Memory block
YJIT: Bounds check every byte in the assembler Previously, YJIT assumed that basic blocks never consume more than 1 KiB of memory. This assumption does not hold for long Ruby methods such as the one in the following: ```ruby eval(<<RUBY) def set_local_a_lot #{'_=0;'*0x40000} end RUBY set_local_a_lot ``` For low `--yjit-exec-mem-size` values, one basic block could exhaust the entire buffer. Introduce a new field `codeblock_t::dropped_bytes` that the assembler sets whenever it runs out of space. Check this field in gen_single_block() to respond to out of memory situations and other error conditions. This design avoids making the control flow graph of existing code generation functions more complex. Use POSIX shell in misc/test_yjit_asm.sh since bash is expanding `0%/*/*` differently. Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
2021-12-01 22:15:23 +03:00
// Users are advised to not use this directly.
uint8_t *mem_block_;
Progress on porting x86 assembler for MicroJIT
2020-09-04 22:56:00 +03:00
// Memory block size
Introduce version_t struct. Will be needed for code invalidation.
2021-01-12 22:56:43 +03:00
uint32_t mem_size;
Progress on porting x86 assembler for MicroJIT
2020-09-04 22:56:00 +03:00
YJIT: Bounds check every byte in the assembler Previously, YJIT assumed that basic blocks never consume more than 1 KiB of memory. This assumption does not hold for long Ruby methods such as the one in the following: ```ruby eval(<<RUBY) def set_local_a_lot #{'_=0;'*0x40000} end RUBY set_local_a_lot ``` For low `--yjit-exec-mem-size` values, one basic block could exhaust the entire buffer. Introduce a new field `codeblock_t::dropped_bytes` that the assembler sets whenever it runs out of space. Check this field in gen_single_block() to respond to out of memory situations and other error conditions. This design avoids making the control flow graph of existing code generation functions more complex. Use POSIX shell in misc/test_yjit_asm.sh since bash is expanding `0%/*/*` differently. Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
2021-12-01 22:15:23 +03:00
// Current writing position
Introduce version_t struct. Will be needed for code invalidation.
2021-01-12 22:56:43 +03:00
uint32_t write_pos;
Progress on porting x86 assembler for MicroJIT
2020-09-04 22:56:00 +03:00
// Table of registered label addresses
Introduce version_t struct. Will be needed for code invalidation.
2021-01-12 22:56:43 +03:00
uint32_t label_addrs[MAX_LABELS];
Progress on porting x86 assembler for MicroJIT
2020-09-04 22:56:00 +03:00
Ported label linking and conditional jumps
2020-09-11 22:10:16 +03:00
// Table of registered label names
// Note that these should be constant strings only
style: align pointer "*" to the right
2021-09-29 21:58:01 +03:00
const char *label_names[MAX_LABELS];
Ported label linking and conditional jumps
2020-09-11 22:10:16 +03:00
Progress on porting x86 assembler for MicroJIT
2020-09-04 22:56:00 +03:00
// References to labels
labelref_t label_refs[MAX_LABEL_REFS];
// Number of labels registeered
Introduce version_t struct. Will be needed for code invalidation.
2021-01-12 22:56:43 +03:00
uint32_t num_labels;
Progress on porting x86 assembler for MicroJIT
2020-09-04 22:56:00 +03:00
// Number of references to labels
Introduce version_t struct. Will be needed for code invalidation.
2021-01-12 22:56:43 +03:00
uint32_t num_refs;
Progress on porting x86 assembler for MicroJIT
2020-09-04 22:56:00 +03:00
Mark JIT code as writeable / executable depending on the situation Some platforms don't want memory to be marked as writeable and executable at the same time. When we write to the code block, we calculate the OS page that the buffer position maps to. Then we call `mprotect` to allow writes on that particular page. As an optimization, we cache the "last written" aligned page which allows us to amortize the cost of the `mprotect` call. In other words, sequential writes to the same page will only call `mprotect` on the page once. When we're done writing, we call `mprotect` on the entire JIT buffer. This means we don't need to keep track of which pages were marked as writeable, we let the OS take care of that. Co-authored-by: John Hawthorn <john@hawthorn.email>
2021-10-27 02:57:30 +03:00
// Keep track of the current aligned write position.
// Used for changing protection when writing to the JIT buffer
uint32_t current_aligned_write_pos;
YJIT: Bounds check every byte in the assembler Previously, YJIT assumed that basic blocks never consume more than 1 KiB of memory. This assumption does not hold for long Ruby methods such as the one in the following: ```ruby eval(<<RUBY) def set_local_a_lot #{'_=0;'*0x40000} end RUBY set_local_a_lot ``` For low `--yjit-exec-mem-size` values, one basic block could exhaust the entire buffer. Introduce a new field `codeblock_t::dropped_bytes` that the assembler sets whenever it runs out of space. Check this field in gen_single_block() to respond to out of memory situations and other error conditions. This design avoids making the control flow graph of existing code generation functions more complex. Use POSIX shell in misc/test_yjit_asm.sh since bash is expanding `0%/*/*` differently. Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
2021-12-01 22:15:23 +03:00
// Set if the assembler is unable to output some instructions,
// for example, when there is not enough space or when a jump
// target is too far away.
bool dropped_bytes;
// Flag to enable or disable comments
bool has_asm;
Progress on porting x86 assembler for MicroJIT
2020-09-04 22:56:00 +03:00
} codeblock_t;
Mark JIT code as writeable / executable depending on the situation Some platforms don't want memory to be marked as writeable and executable at the same time. When we write to the code block, we calculate the OS page that the buffer position maps to. Then we call `mprotect` to allow writes on that particular page. As an optimization, we cache the "last written" aligned page which allows us to amortize the cost of the `mprotect` call. In other words, sequential writes to the same page will only call `mprotect` on the page once. When we're done writing, we call `mprotect` on the entire JIT buffer. This means we don't need to keep track of which pages were marked as writeable, we let the OS take care of that. Co-authored-by: John Hawthorn <john@hawthorn.email>
2021-10-27 02:57:30 +03:00
// 1 is not aligned so this won't match any pages
#define ALIGNED_WRITE_POSITION_NONE 1
Progress on x86 assembler. Encode a few simple instructions.
2020-09-08 23:45:35 +03:00
enum OpndType
{
OPND_NONE,
OPND_REG,
OPND_IMM,
Compiling getlocal_WC_0
2020-09-15 22:12:31 +03:00
OPND_MEM
Progress on x86 assembler. Encode a few simple instructions.
2020-09-08 23:45:35 +03:00
};
enum RegType
{
REG_GP,
REG_FP,
REG_XMM,
REG_IP
};
typedef struct X86Reg
{
// Register type
uint8_t reg_type;
// Register index number
uint8_t reg_no;
} x86reg_t;
typedef struct X86Mem
{
/// Base register number
uint8_t base_reg_no;
/// Index register number
uint8_t idx_reg_no;
/// SIB scale exponent value (power of two, two bits)
uint8_t scale_exp;
/// Has index register flag
bool has_idx;
add and mov instruction encoding ported and tested
2020-09-09 23:45:28 +03:00
// TODO: should this be here, or should we have an extra operand type?
Progress on x86 assembler. Encode a few simple instructions.
2020-09-08 23:45:35 +03:00
/// IP-relative addressing flag
bool is_iprel;
/// Constant displacement from the base, not scaled
int32_t disp;
} x86mem_t;
Started porting instruction encoding
2020-09-05 00:10:11 +03:00
typedef struct X86Opnd
{
Progress on x86 assembler. Encode a few simple instructions.
2020-09-08 23:45:35 +03:00
// Operand type
uint8_t type;
Started porting instruction encoding
2020-09-05 00:10:11 +03:00
Progress on x86 assembler. Encode a few simple instructions.
2020-09-08 23:45:35 +03:00
// Size in bits
uint16_t num_bits;
Started porting instruction encoding
2020-09-05 00:10:11 +03:00
Progress on x86 assembler. Encode a few simple instructions.
2020-09-08 23:45:35 +03:00
union
{
// Register operand
x86reg_t reg;
Started porting instruction encoding
2020-09-05 00:10:11 +03:00
Progress on x86 assembler. Encode a few simple instructions.
2020-09-08 23:45:35 +03:00
// Memory operand
x86mem_t mem;
// Signed immediate value
int64_t imm;
// Unsigned immediate value
Remove PC argument from ujit instructions
2020-09-14 21:36:39 +03:00
uint64_t unsig_imm;
Anonymous union is a C11 feature unavailable to us
2020-10-06 00:11:50 +03:00
} as;
Started porting instruction encoding
2020-09-05 00:10:11 +03:00
} x86opnd_t;
add and mov instruction encoding ported and tested
2020-09-09 23:45:28 +03:00
// Dummy none/null operand
Anonymous union is a C11 feature unavailable to us
2020-10-06 00:11:50 +03:00
static const x86opnd_t NO_OPND = { OPND_NONE, 0, .as.imm = 0 };
add and mov instruction encoding ported and tested
2020-09-09 23:45:28 +03:00
Compiling getlocal_WC_0
2020-09-15 22:12:31 +03:00
// Instruction pointer
Anonymous union is a C11 feature unavailable to us
2020-10-06 00:11:50 +03:00
static const x86opnd_t RIP = { OPND_REG, 64, .as.reg = { REG_IP, 5 }};
Compiling getlocal_WC_0
2020-09-15 22:12:31 +03:00
Progress on x86 assembler. Encode a few simple instructions.
2020-09-08 23:45:35 +03:00
// 64-bit GP registers
Anonymous union is a C11 feature unavailable to us
2020-10-06 00:11:50 +03:00
static const x86opnd_t RAX = { OPND_REG, 64, .as.reg = { REG_GP, 0 }};
static const x86opnd_t RCX = { OPND_REG, 64, .as.reg = { REG_GP, 1 }};
static const x86opnd_t RDX = { OPND_REG, 64, .as.reg = { REG_GP, 2 }};
static const x86opnd_t RBX = { OPND_REG, 64, .as.reg = { REG_GP, 3 }};
static const x86opnd_t RSP = { OPND_REG, 64, .as.reg = { REG_GP, 4 }};
static const x86opnd_t RBP = { OPND_REG, 64, .as.reg = { REG_GP, 5 }};
static const x86opnd_t RSI = { OPND_REG, 64, .as.reg = { REG_GP, 6 }};
static const x86opnd_t RDI = { OPND_REG, 64, .as.reg = { REG_GP, 7 }};
static const x86opnd_t R8 = { OPND_REG, 64, .as.reg = { REG_GP, 8 }};
static const x86opnd_t R9 = { OPND_REG, 64, .as.reg = { REG_GP, 9 }};
static const x86opnd_t R10 = { OPND_REG, 64, .as.reg = { REG_GP, 10 }};
static const x86opnd_t R11 = { OPND_REG, 64, .as.reg = { REG_GP, 11 }};
static const x86opnd_t R12 = { OPND_REG, 64, .as.reg = { REG_GP, 12 }};
static const x86opnd_t R13 = { OPND_REG, 64, .as.reg = { REG_GP, 13 }};
static const x86opnd_t R14 = { OPND_REG, 64, .as.reg = { REG_GP, 14 }};
static const x86opnd_t R15 = { OPND_REG, 64, .as.reg = { REG_GP, 15 }};
Progress on x86 assembler. Encode a few simple instructions.
2020-09-08 23:45:35 +03:00
Added sub instruction, 32-bit registers, more tests
2020-09-10 17:57:29 +03:00
// 32-bit GP registers
Anonymous union is a C11 feature unavailable to us
2020-10-06 00:11:50 +03:00
static const x86opnd_t EAX = { OPND_REG, 32, .as.reg = { REG_GP, 0 }};
static const x86opnd_t ECX = { OPND_REG, 32, .as.reg = { REG_GP, 1 }};
static const x86opnd_t EDX = { OPND_REG, 32, .as.reg = { REG_GP, 2 }};
static const x86opnd_t EBX = { OPND_REG, 32, .as.reg = { REG_GP, 3 }};
static const x86opnd_t ESP = { OPND_REG, 32, .as.reg = { REG_GP, 4 }};
static const x86opnd_t EBP = { OPND_REG, 32, .as.reg = { REG_GP, 5 }};
static const x86opnd_t ESI = { OPND_REG, 32, .as.reg = { REG_GP, 6 }};
static const x86opnd_t EDI = { OPND_REG, 32, .as.reg = { REG_GP, 7 }};
static const x86opnd_t R8D = { OPND_REG, 32, .as.reg = { REG_GP, 8 }};
static const x86opnd_t R9D = { OPND_REG, 32, .as.reg = { REG_GP, 9 }};
static const x86opnd_t R10D = { OPND_REG, 32, .as.reg = { REG_GP, 10 }};
static const x86opnd_t R11D = { OPND_REG, 32, .as.reg = { REG_GP, 11 }};
static const x86opnd_t R12D = { OPND_REG, 32, .as.reg = { REG_GP, 12 }};
static const x86opnd_t R13D = { OPND_REG, 32, .as.reg = { REG_GP, 13 }};
static const x86opnd_t R14D = { OPND_REG, 32, .as.reg = { REG_GP, 14 }};
static const x86opnd_t R15D = { OPND_REG, 32, .as.reg = { REG_GP, 15 }};
Added sub instruction, 32-bit registers, more tests
2020-09-10 17:57:29 +03:00
Added 8-bit and 16-bit general-purpose registers, more tests.
2020-09-22 18:39:04 +03:00
// 16-bit GP registers
Anonymous union is a C11 feature unavailable to us
2020-10-06 00:11:50 +03:00
static const x86opnd_t AX = { OPND_REG, 16, .as.reg = { REG_GP, 0 }};
static const x86opnd_t CX = { OPND_REG, 16, .as.reg = { REG_GP, 1 }};
static const x86opnd_t DX = { OPND_REG, 16, .as.reg = { REG_GP, 2 }};
static const x86opnd_t BX = { OPND_REG, 16, .as.reg = { REG_GP, 3 }};
static const x86opnd_t SP = { OPND_REG, 16, .as.reg = { REG_GP, 4 }};
static const x86opnd_t BP = { OPND_REG, 16, .as.reg = { REG_GP, 5 }};
static const x86opnd_t SI = { OPND_REG, 16, .as.reg = { REG_GP, 6 }};
static const x86opnd_t DI = { OPND_REG, 16, .as.reg = { REG_GP, 7 }};
static const x86opnd_t R8W = { OPND_REG, 16, .as.reg = { REG_GP, 8 }};
static const x86opnd_t R9W = { OPND_REG, 16, .as.reg = { REG_GP, 9 }};
static const x86opnd_t R10W = { OPND_REG, 16, .as.reg = { REG_GP, 10 }};
static const x86opnd_t R11W = { OPND_REG, 16, .as.reg = { REG_GP, 11 }};
static const x86opnd_t R12W = { OPND_REG, 16, .as.reg = { REG_GP, 12 }};
static const x86opnd_t R13W = { OPND_REG, 16, .as.reg = { REG_GP, 13 }};
static const x86opnd_t R14W = { OPND_REG, 16, .as.reg = { REG_GP, 14 }};
static const x86opnd_t R15W = { OPND_REG, 16, .as.reg = { REG_GP, 15 }};
Added 8-bit and 16-bit general-purpose registers, more tests.
2020-09-22 18:39:04 +03:00
// 8-bit GP registers
Anonymous union is a C11 feature unavailable to us
2020-10-06 00:11:50 +03:00
static const x86opnd_t AL = { OPND_REG, 8, .as.reg = { REG_GP, 0 }};
static const x86opnd_t CL = { OPND_REG, 8, .as.reg = { REG_GP, 1 }};
static const x86opnd_t DL = { OPND_REG, 8, .as.reg = { REG_GP, 2 }};
static const x86opnd_t BL = { OPND_REG, 8, .as.reg = { REG_GP, 3 }};
static const x86opnd_t SPL = { OPND_REG, 8, .as.reg = { REG_GP, 4 }};
static const x86opnd_t BPL = { OPND_REG, 8, .as.reg = { REG_GP, 5 }};
static const x86opnd_t SIL = { OPND_REG, 8, .as.reg = { REG_GP, 6 }};
static const x86opnd_t DIL = { OPND_REG, 8, .as.reg = { REG_GP, 7 }};
static const x86opnd_t R8B = { OPND_REG, 8, .as.reg = { REG_GP, 8 }};
static const x86opnd_t R9B = { OPND_REG, 8, .as.reg = { REG_GP, 9 }};
static const x86opnd_t R10B = { OPND_REG, 8, .as.reg = { REG_GP, 10 }};
static const x86opnd_t R11B = { OPND_REG, 8, .as.reg = { REG_GP, 11 }};
static const x86opnd_t R12B = { OPND_REG, 8, .as.reg = { REG_GP, 12 }};
static const x86opnd_t R13B = { OPND_REG, 8, .as.reg = { REG_GP, 13 }};
static const x86opnd_t R14B = { OPND_REG, 8, .as.reg = { REG_GP, 14 }};
static const x86opnd_t R15B = { OPND_REG, 8, .as.reg = { REG_GP, 15 }};
Added 8-bit and 16-bit general-purpose registers, more tests.
2020-09-22 18:39:04 +03:00
First pass at CFUNC calls complete. Not yet functional.
2020-10-20 22:03:06 +03:00
// C argument registers
#define NUM_C_ARG_REGS 6
#define C_ARG_REGS ( (x86opnd_t[]){ RDI, RSI, RDX, RCX, R8, R9 } )
Fix changes from rebase
2021-10-01 14:17:09 +03:00
// Compute the number of bits needed to store a signed or unsigned value
Put YJIT into a single compilation unit For upstreaming, we want functions we export either prefixed with "rb_" or made static. Historically we haven't been following this rule, so we were "leaking" a lot of symbols as `make leak-globals` would tell us. This change unifies everything YJIT into a single compilation unit, yjit.o, and makes everything unprefixed static to pass `make leak-globals`. This manual "unified build" setup is similar to that of vm.o. Having everything in one compilation unit allows static functions to be visible across YJIT files and removes the need for declarations in headers in some cases. Unnecessary declarations were removed. Other changes of note: - switched to MJIT_SYMBOL_EXPORT_BEGIN which indicates stuff as being off limits for native extensions - the first include of each YJIT file is change to be "internal.h" - undefined MAP_STACK before explicitly redefining it since it collide's with a definition in system headers. Consider renaming?
2021-10-02 01:38:39 +03:00
static inline uint32_t sig_imm_size(int64_t imm);
static inline uint32_t unsig_imm_size(uint64_t imm);
Fix changes from rebase
2021-10-01 14:17:09 +03:00
add and mov instruction encoding ported and tested
2020-09-09 23:45:28 +03:00
// Memory operand with base register and displacement/offset
Put YJIT into a single compilation unit For upstreaming, we want functions we export either prefixed with "rb_" or made static. Historically we haven't been following this rule, so we were "leaking" a lot of symbols as `make leak-globals` would tell us. This change unifies everything YJIT into a single compilation unit, yjit.o, and makes everything unprefixed static to pass `make leak-globals`. This manual "unified build" setup is similar to that of vm.o. Having everything in one compilation unit allows static functions to be visible across YJIT files and removes the need for declarations in headers in some cases. Unnecessary declarations were removed. Other changes of note: - switched to MJIT_SYMBOL_EXPORT_BEGIN which indicates stuff as being off limits for native extensions - the first include of each YJIT file is change to be "internal.h" - undefined MAP_STACK before explicitly redefining it since it collide's with a definition in system headers. Consider renaming?
2021-10-02 01:38:39 +03:00
static inline x86opnd_t mem_opnd(uint32_t num_bits, x86opnd_t base_reg, int32_t disp);
add and mov instruction encoding ported and tested
2020-09-09 23:45:28 +03:00
Tally instructions when taking side exists for --ujit-stats shopify/ruby#29 Co-authored-by: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com>
2021-01-28 00:13:27 +03:00
// Scale-index-base memory operand
Put YJIT into a single compilation unit For upstreaming, we want functions we export either prefixed with "rb_" or made static. Historically we haven't been following this rule, so we were "leaking" a lot of symbols as `make leak-globals` would tell us. This change unifies everything YJIT into a single compilation unit, yjit.o, and makes everything unprefixed static to pass `make leak-globals`. This manual "unified build" setup is similar to that of vm.o. Having everything in one compilation unit allows static functions to be visible across YJIT files and removes the need for declarations in headers in some cases. Unnecessary declarations were removed. Other changes of note: - switched to MJIT_SYMBOL_EXPORT_BEGIN which indicates stuff as being off limits for native extensions - the first include of each YJIT file is change to be "internal.h" - undefined MAP_STACK before explicitly redefining it since it collide's with a definition in system headers. Consider renaming?
2021-10-02 01:38:39 +03:00
static inline x86opnd_t mem_opnd_sib(uint32_t num_bits, x86opnd_t base_reg, x86opnd_t index_reg, int32_t scale, int32_t disp);
Tally instructions when taking side exists for --ujit-stats shopify/ruby#29 Co-authored-by: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com>
2021-01-28 00:13:27 +03:00
add and mov instruction encoding ported and tested
2020-09-09 23:45:28 +03:00
// Immediate number operand
Put YJIT into a single compilation unit For upstreaming, we want functions we export either prefixed with "rb_" or made static. Historically we haven't been following this rule, so we were "leaking" a lot of symbols as `make leak-globals` would tell us. This change unifies everything YJIT into a single compilation unit, yjit.o, and makes everything unprefixed static to pass `make leak-globals`. This manual "unified build" setup is similar to that of vm.o. Having everything in one compilation unit allows static functions to be visible across YJIT files and removes the need for declarations in headers in some cases. Unnecessary declarations were removed. Other changes of note: - switched to MJIT_SYMBOL_EXPORT_BEGIN which indicates stuff as being off limits for native extensions - the first include of each YJIT file is change to be "internal.h" - undefined MAP_STACK before explicitly redefining it since it collide's with a definition in system headers. Consider renaming?
2021-10-02 01:38:39 +03:00
static inline x86opnd_t imm_opnd(int64_t val);
add and mov instruction encoding ported and tested
2020-09-09 23:45:28 +03:00
Remove PC argument from ujit instructions
2020-09-14 21:36:39 +03:00
// Constant pointer operand
Put YJIT into a single compilation unit For upstreaming, we want functions we export either prefixed with "rb_" or made static. Historically we haven't been following this rule, so we were "leaking" a lot of symbols as `make leak-globals` would tell us. This change unifies everything YJIT into a single compilation unit, yjit.o, and makes everything unprefixed static to pass `make leak-globals`. This manual "unified build" setup is similar to that of vm.o. Having everything in one compilation unit allows static functions to be visible across YJIT files and removes the need for declarations in headers in some cases. Unnecessary declarations were removed. Other changes of note: - switched to MJIT_SYMBOL_EXPORT_BEGIN which indicates stuff as being off limits for native extensions - the first include of each YJIT file is change to be "internal.h" - undefined MAP_STACK before explicitly redefining it since it collide's with a definition in system headers. Consider renaming?
2021-10-02 01:38:39 +03:00
static inline x86opnd_t const_ptr_opnd(const void *ptr);
Remove PC argument from ujit instructions
2020-09-14 21:36:39 +03:00
Added member_opnd() convenience macro
2020-09-30 23:57:54 +03:00
// Struct member operand
#define member_opnd(base_reg, struct_type, member_name) mem_opnd( \
8 * sizeof(((struct_type*)0)->member_name), \
base_reg, \
offsetof(struct_type, member_name) \
)
Implemented opt_minus in MicroJIT
2020-10-02 19:12:28 +03:00
// Struct member operand with an array index
#define member_opnd_idx(base_reg, struct_type, member_name, idx) mem_opnd( \
Respect redefinition in MicroJIT's opt_minus
2020-10-05 18:19:11 +03:00
8 * sizeof(((struct_type*)0)->member_name[0]), \
Implemented opt_minus in MicroJIT
2020-10-02 19:12:28 +03:00
base_reg, \
(offsetof(struct_type, member_name) + \
Respect redefinition in MicroJIT's opt_minus
2020-10-05 18:19:11 +03:00
sizeof(((struct_type*)0)->member_name[0]) * idx) \
Implemented opt_minus in MicroJIT
2020-10-02 19:12:28 +03:00
)
YJIT code pages refactoring for code GC (#5073) * New code page allocation logic * Fix leaked globals * Fix leaked symbols, yjit asm tests * Make COUNTED_EXIT take a jit argument, so we can eliminate global ocb * Remove extra whitespace * Change block start_pos/end_pos to be pointers instead of uint32_t * Change branch end_pos and start_pos to end_addr, start_addr
2021-11-04 23:05:41 +03:00
// Allocate executable memory
Put YJIT into a single compilation unit For upstreaming, we want functions we export either prefixed with "rb_" or made static. Historically we haven't been following this rule, so we were "leaking" a lot of symbols as `make leak-globals` would tell us. This change unifies everything YJIT into a single compilation unit, yjit.o, and makes everything unprefixed static to pass `make leak-globals`. This manual "unified build" setup is similar to that of vm.o. Having everything in one compilation unit allows static functions to be visible across YJIT files and removes the need for declarations in headers in some cases. Unnecessary declarations were removed. Other changes of note: - switched to MJIT_SYMBOL_EXPORT_BEGIN which indicates stuff as being off limits for native extensions - the first include of each YJIT file is change to be "internal.h" - undefined MAP_STACK before explicitly redefining it since it collide's with a definition in system headers. Consider renaming?
2021-10-02 01:38:39 +03:00
static uint8_t *alloc_exec_mem(uint32_t mem_size);
YJIT code pages refactoring for code GC (#5073) * New code page allocation logic * Fix leaked globals * Fix leaked symbols, yjit asm tests * Make COUNTED_EXIT take a jit argument, so we can eliminate global ocb * Remove extra whitespace * Change block start_pos/end_pos to be pointers instead of uint32_t * Change branch end_pos and start_pos to end_addr, start_addr
2021-11-04 23:05:41 +03:00
// Code block functions
Put YJIT into a single compilation unit For upstreaming, we want functions we export either prefixed with "rb_" or made static. Historically we haven't been following this rule, so we were "leaking" a lot of symbols as `make leak-globals` would tell us. This change unifies everything YJIT into a single compilation unit, yjit.o, and makes everything unprefixed static to pass `make leak-globals`. This manual "unified build" setup is similar to that of vm.o. Having everything in one compilation unit allows static functions to be visible across YJIT files and removes the need for declarations in headers in some cases. Unnecessary declarations were removed. Other changes of note: - switched to MJIT_SYMBOL_EXPORT_BEGIN which indicates stuff as being off limits for native extensions - the first include of each YJIT file is change to be "internal.h" - undefined MAP_STACK before explicitly redefining it since it collide's with a definition in system headers. Consider renaming?
2021-10-02 01:38:39 +03:00
static inline void cb_init(codeblock_t *cb, uint8_t *mem_block, uint32_t mem_size);
static inline void cb_align_pos(codeblock_t *cb, uint32_t multiple);
static inline void cb_set_pos(codeblock_t *cb, uint32_t pos);
static inline void cb_set_write_ptr(codeblock_t *cb, uint8_t *code_ptr);
YJIT: Bounds check every byte in the assembler Previously, YJIT assumed that basic blocks never consume more than 1 KiB of memory. This assumption does not hold for long Ruby methods such as the one in the following: ```ruby eval(<<RUBY) def set_local_a_lot #{'_=0;'*0x40000} end RUBY set_local_a_lot ``` For low `--yjit-exec-mem-size` values, one basic block could exhaust the entire buffer. Introduce a new field `codeblock_t::dropped_bytes` that the assembler sets whenever it runs out of space. Check this field in gen_single_block() to respond to out of memory situations and other error conditions. This design avoids making the control flow graph of existing code generation functions more complex. Use POSIX shell in misc/test_yjit_asm.sh since bash is expanding `0%/*/*` differently. Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
2021-12-01 22:15:23 +03:00
static inline uint8_t *cb_get_ptr(const codeblock_t *cb, uint32_t index);
static inline uint8_t *cb_get_write_ptr(const codeblock_t *cb);
Put YJIT into a single compilation unit For upstreaming, we want functions we export either prefixed with "rb_" or made static. Historically we haven't been following this rule, so we were "leaking" a lot of symbols as `make leak-globals` would tell us. This change unifies everything YJIT into a single compilation unit, yjit.o, and makes everything unprefixed static to pass `make leak-globals`. This manual "unified build" setup is similar to that of vm.o. Having everything in one compilation unit allows static functions to be visible across YJIT files and removes the need for declarations in headers in some cases. Unnecessary declarations were removed. Other changes of note: - switched to MJIT_SYMBOL_EXPORT_BEGIN which indicates stuff as being off limits for native extensions - the first include of each YJIT file is change to be "internal.h" - undefined MAP_STACK before explicitly redefining it since it collide's with a definition in system headers. Consider renaming?
2021-10-02 01:38:39 +03:00
static inline void cb_write_byte(codeblock_t *cb, uint8_t byte);
static inline void cb_write_bytes(codeblock_t *cb, uint32_t num_bytes, ...);
static inline void cb_write_int(codeblock_t *cb, uint64_t val, uint32_t num_bits);
static inline uint32_t cb_new_label(codeblock_t *cb, const char *name);
static inline void cb_write_label(codeblock_t *cb, uint32_t label_idx);
static inline void cb_label_ref(codeblock_t *cb, uint32_t label_idx);
static inline void cb_link_labels(codeblock_t *cb);
Mark JIT code as writeable / executable depending on the situation Some platforms don't want memory to be marked as writeable and executable at the same time. When we write to the code block, we calculate the OS page that the buffer position maps to. Then we call `mprotect` to allow writes on that particular page. As an optimization, we cache the "last written" aligned page which allows us to amortize the cost of the `mprotect` call. In other words, sequential writes to the same page will only call `mprotect` on the page once. When we're done writing, we call `mprotect` on the entire JIT buffer. This means we don't need to keep track of which pages were marked as writeable, we let the OS take care of that. Co-authored-by: John Hawthorn <john@hawthorn.email>
2021-10-27 02:57:30 +03:00
static inline void cb_mark_all_writeable(codeblock_t *cb);
static inline void cb_mark_position_writeable(codeblock_t *cb, uint32_t write_pos);
static inline void cb_mark_all_executable(codeblock_t *cb);
Progress on porting x86 assembler for MicroJIT
2020-09-04 22:56:00 +03:00
Progress on x86 assembler. Encode a few simple instructions.
2020-09-08 23:45:35 +03:00
// Encode individual instructions into a code block
Put YJIT into a single compilation unit For upstreaming, we want functions we export either prefixed with "rb_" or made static. Historically we haven't been following this rule, so we were "leaking" a lot of symbols as `make leak-globals` would tell us. This change unifies everything YJIT into a single compilation unit, yjit.o, and makes everything unprefixed static to pass `make leak-globals`. This manual "unified build" setup is similar to that of vm.o. Having everything in one compilation unit allows static functions to be visible across YJIT files and removes the need for declarations in headers in some cases. Unnecessary declarations were removed. Other changes of note: - switched to MJIT_SYMBOL_EXPORT_BEGIN which indicates stuff as being off limits for native extensions - the first include of each YJIT file is change to be "internal.h" - undefined MAP_STACK before explicitly redefining it since it collide's with a definition in system headers. Consider renaming?
2021-10-02 01:38:39 +03:00
static inline void add(codeblock_t *cb, x86opnd_t opnd0, x86opnd_t opnd1);
static inline void and(codeblock_t *cb, x86opnd_t opnd0, x86opnd_t opnd1);
static inline void call_ptr(codeblock_t *cb, x86opnd_t scratch_reg, uint8_t *dst_ptr);
static inline void call_label(codeblock_t *cb, uint32_t label_idx);
static inline void call(codeblock_t *cb, x86opnd_t opnd);
static inline void cmova(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovae(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovb(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovbe(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovc(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmove(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovg(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovge(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovl(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovle(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovna(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovnae(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovnb(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovnbe(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovnc(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovne(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovng(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovnge(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovnl(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovnle(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovno(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovnp(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovns(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovnz(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovo(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovp(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovpe(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovpo(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovs(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmovz(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void cmp(codeblock_t *cb, x86opnd_t opnd0, x86opnd_t opnd1);
static inline void cdq(codeblock_t *cb);
static inline void cqo(codeblock_t *cb);
static inline void int3(codeblock_t *cb);
static inline void ja_label(codeblock_t *cb, uint32_t label_idx);
static inline void jae_label(codeblock_t *cb, uint32_t label_idx);
static inline void jb_label(codeblock_t *cb, uint32_t label_idx);
static inline void jbe_label(codeblock_t *cb, uint32_t label_idx);
static inline void jc_label(codeblock_t *cb, uint32_t label_idx);
static inline void je_label(codeblock_t *cb, uint32_t label_idx);
static inline void jg_label(codeblock_t *cb, uint32_t label_idx);
static inline void jge_label(codeblock_t *cb, uint32_t label_idx);
static inline void jl_label(codeblock_t *cb, uint32_t label_idx);
static inline void jle_label(codeblock_t *cb, uint32_t label_idx);
static inline void jna_label(codeblock_t *cb, uint32_t label_idx);
static inline void jnae_label(codeblock_t *cb, uint32_t label_idx);
static inline void jnb_label(codeblock_t *cb, uint32_t label_idx);
static inline void jnbe_label(codeblock_t *cb, uint32_t label_idx);
static inline void jnc_label(codeblock_t *cb, uint32_t label_idx);
static inline void jne_label(codeblock_t *cb, uint32_t label_idx);
static inline void jng_label(codeblock_t *cb, uint32_t label_idx);
static inline void jnge_label(codeblock_t *cb, uint32_t label_idx);
static inline void jnl_label(codeblock_t *cb, uint32_t label_idx);
static inline void jnle_label(codeblock_t *cb, uint32_t label_idx);
static inline void jno_label(codeblock_t *cb, uint32_t label_idx);
static inline void jnp_label(codeblock_t *cb, uint32_t label_idx);
static inline void jns_label(codeblock_t *cb, uint32_t label_idx);
static inline void jnz_label(codeblock_t *cb, uint32_t label_idx);
static inline void jo_label(codeblock_t *cb, uint32_t label_idx);
static inline void jp_label(codeblock_t *cb, uint32_t label_idx);
static inline void jpe_label(codeblock_t *cb, uint32_t label_idx);
static inline void jpo_label(codeblock_t *cb, uint32_t label_idx);
static inline void js_label(codeblock_t *cb, uint32_t label_idx);
static inline void jz_label(codeblock_t *cb, uint32_t label_idx);
static inline void ja_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jae_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jb_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jbe_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jc_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void je_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jg_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jge_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jl_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jle_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jna_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jnae_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jnb_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jnbe_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jnc_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jne_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jng_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jnge_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jnl_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jnle_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jno_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jnp_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jns_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jnz_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jo_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jp_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jpe_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jpo_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void js_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jz_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jmp_label(codeblock_t *cb, uint32_t label_idx);
static inline void jmp_ptr(codeblock_t *cb, uint8_t *ptr);
static inline void jmp_rm(codeblock_t *cb, x86opnd_t opnd);
static inline void jmp32(codeblock_t *cb, int32_t offset);
static inline void lea(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void mov(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void movsx(codeblock_t *cb, x86opnd_t dst, x86opnd_t src);
static inline void neg(codeblock_t *cb, x86opnd_t opnd);
static inline void nop(codeblock_t *cb, uint32_t length);
static inline void not(codeblock_t *cb, x86opnd_t opnd);
static inline void or(codeblock_t *cb, x86opnd_t opnd0, x86opnd_t opnd1);
static inline void pop(codeblock_t *cb, x86opnd_t reg);
static inline void popfq(codeblock_t *cb);
static inline void push(codeblock_t *cb, x86opnd_t opnd);
static inline void pushfq(codeblock_t *cb);
static inline void ret(codeblock_t *cb);
static inline void sal(codeblock_t *cb, x86opnd_t opnd0, x86opnd_t opnd1);
static inline void sar(codeblock_t *cb, x86opnd_t opnd0, x86opnd_t opnd1);
static inline void shl(codeblock_t *cb, x86opnd_t opnd0, x86opnd_t opnd1);
static inline void shr(codeblock_t *cb, x86opnd_t opnd0, x86opnd_t opnd1);
static inline void sub(codeblock_t *cb, x86opnd_t opnd0, x86opnd_t opnd1);
static inline void test(codeblock_t *cb, x86opnd_t rm_opnd, x86opnd_t test_opnd);
static inline void ud2(codeblock_t *cb);
static inline void xchg(codeblock_t *cb, x86opnd_t rm_opnd, x86opnd_t r_opnd);
static inline void xor(codeblock_t *cb, x86opnd_t opnd0, x86opnd_t opnd1);
static inline void cb_write_lock_prefix(codeblock_t *cb);
Runtime counters for bailing from opt_send_without_block This allows counting things in generated code such as side exits or different reasons for failing to compile. Runtime counters are exposed to Ruby as `UJIT.runtime_stats` for ease of reporting and analysis.
2021-03-03 02:27:50 +03:00
Progress on porting x86 assembler for MicroJIT
2020-09-04 22:56:00 +03:00
#endif