microsoft
/
WSL2-Linux-Kernel
зеркало из https://github.com/microsoft/WSL2-Linux-Kernel.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
WSL2-Linux-Kernel/arch/riscv/kernel/traps.c

236 строки
6.1 KiB
C
Исходник Обычный вид История

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 286 Based on 1 normalized pattern(s): this program is free software you can redistribute it and or modify it under the terms of the gnu general public license as published by the free software foundation version 2 this program is distributed in the hope that it will be useful but without any warranty without even the implied warranty of merchantability or fitness for a particular purpose see the gnu general public license for more details extracted by the scancode license scanner the SPDX license identifier GPL-2.0-only has been chosen to replace the boilerplate/reference in 97 file(s). Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Allison Randal <allison@lohutok.net> Reviewed-by: Alexios Zavras <alexios.zavras@intel.com> Cc: linux-spdx@vger.kernel.org Link: https://lkml.kernel.org/r/20190529141901.025053186@linutronix.de Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-05-29 17:18:00 +03:00
// SPDX-License-Identifier: GPL-2.0-only
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 04:00:26 +03:00
/*
* Copyright (C) 2012 Regents of the University of California
*/
riscv: add missing header file includes sparse identifies several missing prototypes caused by missing preprocessor include directives: arch/riscv/kernel/cpufeature.c:16:6: warning: symbol 'has_fpu' was not declared. Should it be static? arch/riscv/kernel/process.c:26:6: warning: symbol 'arch_cpu_idle' was not declared. Should it be static? arch/riscv/kernel/reset.c:15:6: warning: symbol 'pm_power_off' was not declared. Should it be static? arch/riscv/kernel/syscall_table.c:15:6: warning: symbol 'sys_call_table' was not declared. Should it be static? arch/riscv/kernel/traps.c:149:13: warning: symbol 'trap_init' was not declared. Should it be static? arch/riscv/kernel/vdso.c:54:5: warning: symbol 'arch_setup_additional_pages' was not declared. Should it be static? arch/riscv/kernel/smp.c:64:6: warning: symbol 'arch_match_cpu_phys_id' was not declared. Should it be static? arch/riscv/kernel/module-sections.c:89:5: warning: symbol 'module_frob_arch_sections' was not declared. Should it be static? arch/riscv/mm/context.c:42:6: warning: symbol 'switch_mm' was not declared. Should it be static? Fix by including the appropriate header files in the appropriate source files. This patch should have no functional impact. Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com> Reviewed-by: Christoph Hellwig <hch@lst.de>
2019-10-18 01:21:28 +03:00
#include <linux/cpu.h>
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 04:00:26 +03:00
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/sched.h>
#include <linux/sched/debug.h>
#include <linux/sched/signal.h>
#include <linux/signal.h>
#include <linux/kdebug.h>
#include <linux/uaccess.h>
riscv: Add kprobes supported This patch enables "kprobe & kretprobe" to work with ftrace interface. It utilized software breakpoint as single-step mechanism. Some instructions which can't be single-step executed must be simulated in kernel execution slot, such as: branch, jal, auipc, la ... Some instructions should be rejected for probing and we use a blacklist to filter, such as: ecall, ebreak, ... We use ebreak & c.ebreak to replace origin instruction and the kprobe handler prepares an executable memory slot for out-of-line execution with a copy of the original instruction being probed. In execution slot we add ebreak behind original instruction to simulate a single-setp mechanism. The patch is based on packi's work [1] and csky's work [2]. - The kprobes_trampoline.S is all from packi's patch - The single-step mechanism is new designed for riscv without hw single-step trap - The simulation codes are from csky - Frankly, all codes refer to other archs' implementation [1] https://lore.kernel.org/linux-riscv/20181113195804.22825-1-me@packi.ch/ [2] https://lore.kernel.org/linux-csky/20200403044150.20562-9-guoren@kernel.org/ Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Co-developed-by: Patrick Stählin <me@packi.ch> Signed-off-by: Patrick Stählin <me@packi.ch> Acked-by: Masami Hiramatsu <mhiramat@kernel.org> Tested-by: Zong Li <zong.li@sifive.com> Reviewed-by: Pekka Enberg <penberg@kernel.org> Cc: Patrick Stählin <me@packi.ch> Cc: Palmer Dabbelt <palmerdabbelt@google.com> Cc: Björn Töpel <bjorn.topel@gmail.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-12-17 19:01:42 +03:00
#include <linux/kprobes.h>
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 04:00:26 +03:00
#include <linux/mm.h>
#include <linux/module.h>
#include <linux/irq.h>
riscv: traps: Fix no prototype warnings Fix all W=1 compilation warnings:'no previous prototype for' in arch/riscv/kernel/traps.c: arch/riscv/kernel/traps.c:96:15: warning: no previous prototype for ‘do_trap_unknown’ [-Wmissing-prototypes] 96 | DO_ERROR_INFO(do_trap_unknown, | ^~~~~~~~~~~~~~~ arch/riscv/kernel/traps.c:91:27: note: in definition of macro ‘DO_ERROR_INFO’ 91 | asmlinkage __visible void name(struct pt_regs *regs) \ | ^~~~ arch/riscv/kernel/traps.c:98:15: warning: no previous prototype for ‘do_trap_insn_misaligned’ [-Wmissing-prototypes] 98 | DO_ERROR_INFO(do_trap_insn_misaligned, | ^~~~~~~~~~~~~~~~~~~~~~~ arch/riscv/kernel/traps.c:91:27: note: in definition of macro ‘DO_ERROR_INFO’ 91 | asmlinkage __visible void name(struct pt_regs *regs) \ | ^~~~ arch/riscv/kernel/traps.c:100:15: warning: no previous prototype for ‘do_trap_insn_fault’ [-Wmissing-prototypes] ... Reported-by: Hulk Robot <hulkci@huawei.com> Signed-off-by: Nanyong Sun <sunnanyong@huawei.com> [Palmer: fix checkpatch warnings] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-03-05 14:33:24 +03:00
#include <asm/asm-prototypes.h>
riscv: Add dump stack in show_regs Like commit 1149aad10b1e ("arm64: Add dump_backtrace() in show_regs"), dump the stack in riscv show_regs as common code expects. Reviewed-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-01-11 15:40:12 +03:00
#include <asm/bug.h>
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 04:00:26 +03:00
#include <asm/processor.h>
#include <asm/ptrace.h>
#include <asm/csr.h>
int show_unhandled_signals = 1;
static DEFINE_SPINLOCK(die_lock);
void die(struct pt_regs *regs, const char *str)
{
static int die_counter;
int ret;
oops_enter();
spin_lock_irq(&die_lock);
console_verbose();
bust_spinlocks(1);
pr_emerg("%s [#%d]\n", str, ++die_counter);
print_modules();
show_regs(regs);
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 15:10:32 +03:00
ret = notify_die(DIE_OOPS, str, regs, 0, regs->cause, SIGSEGV);
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 04:00:26 +03:00
bust_spinlocks(0);
add_taint(TAINT_DIE, LOCKDEP_NOW_UNRELIABLE);
spin_unlock_irq(&die_lock);
oops_exit();
if (in_interrupt())
panic("Fatal exception in interrupt");
if (panic_on_oops)
panic("Fatal exception");
if (ret != NOTIFY_STOP)
exit: Add and use make_task_dead. There are two big uses of do_exit. The first is it's design use to be the guts of the exit(2) system call. The second use is to terminate a task after something catastrophic has happened like a NULL pointer in kernel code. Add a function make_task_dead that is initialy exactly the same as do_exit to cover the cases where do_exit is called to handle catastrophic failure. In time this can probably be reduced to just a light wrapper around do_task_dead. For now keep it exactly the same so that there will be no behavioral differences introducing this new concept. Replace all of the uses of do_exit that use it for catastraphic task cleanup with make_task_dead to make it clear what the code is doing. As part of this rename rewind_stack_do_exit rewind_stack_and_make_dead. Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
2021-06-28 22:52:01 +03:00
make_task_dead(SIGSEGV);
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 04:00:26 +03:00
}
signal/riscv: Remove tsk parameter from do_trap The do_trap function is always called with tsk == current. Make that obvious by removing the tsk parameter. This also makes it clear that do_trap calls force_sig_fault on the current task. Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
2019-02-06 04:10:48 +03:00
void do_trap(struct pt_regs *regs, int signo, int code, unsigned long addr)
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 04:00:26 +03:00
{
signal/riscv: Remove tsk parameter from do_trap The do_trap function is always called with tsk == current. Make that obvious by removing the tsk parameter. This also makes it clear that do_trap calls force_sig_fault on the current task. Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
2019-02-06 04:10:48 +03:00
struct task_struct *tsk = current;
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 04:00:26 +03:00
if (show_unhandled_signals && unhandled_signal(tsk, signo)
&& printk_ratelimit()) {
pr_info("%s[%d]: unhandled signal %d code 0x%x at 0x" REG_FMT,
tsk->comm, task_pid_nr(tsk), signo, code, addr);
riscv: remove duplicate macros from ptrace.h No need to have two names for the same thing. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2019-04-15 12:14:40 +03:00
print_vma_addr(KERN_CONT " in ", instruction_pointer(regs));
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 04:00:26 +03:00
pr_cont("\n");
riscv: Add dump stack in show_regs Like commit 1149aad10b1e ("arm64: Add dump_backtrace() in show_regs"), dump the stack in riscv show_regs as common code expects. Reviewed-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-01-11 15:40:12 +03:00
__show_regs(regs);
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 04:00:26 +03:00
}
signal: Remove the task parameter from force_sig_fault As synchronous exceptions really only make sense against the current task (otherwise how are you synchronous) remove the task parameter from from force_sig_fault to make it explicit that is what is going on. The two known exceptions that deliver a synchronous exception to a stopped ptraced task have already been changed to force_sig_fault_to_task. The callers have been changed with the following emacs regular expression (with obvious variations on the architectures that take more arguments) to avoid typos: force_sig_fault[(]\([^,]+\)[,]\([^,]+\)[,]\([^,]+\)[,]\W+current[)] -> force_sig_fault(\1,\2,\3) Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
2019-05-23 19:04:24 +03:00
force_sig_fault(signo, code, (void __user *)addr);
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 04:00:26 +03:00
}
static void do_trap_error(struct pt_regs *regs, int signo, int code,
unsigned long addr, const char *str)
{
riscv: Add uprobes supported This patch adds support for uprobes on riscv architecture. Just like kprobe, it support single-step and simulate instructions. Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Reviewed-by: Pekka Enberg <penberg@kernel.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-12-17 19:01:44 +03:00
current->thread.bad_cause = regs->cause;
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 04:00:26 +03:00
if (user_mode(regs)) {
signal/riscv: Remove tsk parameter from do_trap The do_trap function is always called with tsk == current. Make that obvious by removing the tsk parameter. This also makes it clear that do_trap calls force_sig_fault on the current task. Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
2019-02-06 04:10:48 +03:00
do_trap(regs, signo, code, addr);
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 04:00:26 +03:00
} else {
if (!fixup_exception(regs))
die(regs, str);
}
}
riscv: xip: support runtime trap patching RISCV_ERRATA_ALTERNATIVE patches text at runtime which is currently not possible when the kernel is executed from the flash in XIP mode. Since runtime patching concerns only traps at the moment, let's just have all the traps reside in RAM anyway if RISCV_ERRATA_ALTERNATIVE is set. Thus, these functions will be patch-able even when the .text section is in flash. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-05-31 11:53:42 +03:00
#if defined (CONFIG_XIP_KERNEL) && defined (CONFIG_RISCV_ERRATA_ALTERNATIVE)
#define __trap_section __section(".xip.traps")
#else
#define __trap_section
#endif
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 04:00:26 +03:00
#define DO_ERROR_INFO(name, signo, code, str) \
riscv: xip: support runtime trap patching RISCV_ERRATA_ALTERNATIVE patches text at runtime which is currently not possible when the kernel is executed from the flash in XIP mode. Since runtime patching concerns only traps at the moment, let's just have all the traps reside in RAM anyway if RISCV_ERRATA_ALTERNATIVE is set. Thus, these functions will be patch-able even when the .text section is in flash. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-05-31 11:53:42 +03:00
asmlinkage __visible __trap_section void name(struct pt_regs *regs) \
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 04:00:26 +03:00
{ \
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 15:10:32 +03:00
do_trap_error(regs, signo, code, regs->epc, "Oops - " str); \
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 04:00:26 +03:00
}
DO_ERROR_INFO(do_trap_unknown,
SIGILL, ILL_ILLTRP, "unknown exception");
DO_ERROR_INFO(do_trap_insn_misaligned,
SIGBUS, BUS_ADRALN, "instruction address misaligned");
DO_ERROR_INFO(do_trap_insn_fault,
SIGSEGV, SEGV_ACCERR, "instruction access fault");
DO_ERROR_INFO(do_trap_insn_illegal,
SIGILL, ILL_ILLOPC, "illegal instruction");
DO_ERROR_INFO(do_trap_load_fault,
SIGSEGV, SEGV_ACCERR, "load access fault");
riscv: Unaligned load/store handling for M_MODE Add handlers for unaligned load and store traps that may be generated by applications. Code heavily inspired from the OpenSBI project. Handling of the unaligned access traps is suitable for applications compiled with or without compressed instructions and is independent of the kernel CONFIG_RISCV_ISA_C option value. Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Signed-off-by: Anup Patel <anup.patel@wdc.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-16 03:47:36 +03:00
#ifndef CONFIG_RISCV_M_MODE
DO_ERROR_INFO(do_trap_load_misaligned,
SIGBUS, BUS_ADRALN, "Oops - load address misaligned");
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 04:00:26 +03:00
DO_ERROR_INFO(do_trap_store_misaligned,
riscv: Unaligned load/store handling for M_MODE Add handlers for unaligned load and store traps that may be generated by applications. Code heavily inspired from the OpenSBI project. Handling of the unaligned access traps is suitable for applications compiled with or without compressed instructions and is independent of the kernel CONFIG_RISCV_ISA_C option value. Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Signed-off-by: Anup Patel <anup.patel@wdc.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-16 03:47:36 +03:00
SIGBUS, BUS_ADRALN, "Oops - store (or AMO) address misaligned");
#else
int handle_misaligned_load(struct pt_regs *regs);
int handle_misaligned_store(struct pt_regs *regs);
riscv: xip: support runtime trap patching RISCV_ERRATA_ALTERNATIVE patches text at runtime which is currently not possible when the kernel is executed from the flash in XIP mode. Since runtime patching concerns only traps at the moment, let's just have all the traps reside in RAM anyway if RISCV_ERRATA_ALTERNATIVE is set. Thus, these functions will be patch-able even when the .text section is in flash. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-05-31 11:53:42 +03:00
asmlinkage void __trap_section do_trap_load_misaligned(struct pt_regs *regs)
riscv: Unaligned load/store handling for M_MODE Add handlers for unaligned load and store traps that may be generated by applications. Code heavily inspired from the OpenSBI project. Handling of the unaligned access traps is suitable for applications compiled with or without compressed instructions and is independent of the kernel CONFIG_RISCV_ISA_C option value. Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Signed-off-by: Anup Patel <anup.patel@wdc.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-16 03:47:36 +03:00
{
if (!handle_misaligned_load(regs))
return;
do_trap_error(regs, SIGBUS, BUS_ADRALN, regs->epc,
"Oops - load address misaligned");
}
riscv: xip: support runtime trap patching RISCV_ERRATA_ALTERNATIVE patches text at runtime which is currently not possible when the kernel is executed from the flash in XIP mode. Since runtime patching concerns only traps at the moment, let's just have all the traps reside in RAM anyway if RISCV_ERRATA_ALTERNATIVE is set. Thus, these functions will be patch-able even when the .text section is in flash. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-05-31 11:53:42 +03:00
asmlinkage void __trap_section do_trap_store_misaligned(struct pt_regs *regs)
riscv: Unaligned load/store handling for M_MODE Add handlers for unaligned load and store traps that may be generated by applications. Code heavily inspired from the OpenSBI project. Handling of the unaligned access traps is suitable for applications compiled with or without compressed instructions and is independent of the kernel CONFIG_RISCV_ISA_C option value. Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Signed-off-by: Anup Patel <anup.patel@wdc.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-16 03:47:36 +03:00
{
if (!handle_misaligned_store(regs))
return;
do_trap_error(regs, SIGBUS, BUS_ADRALN, regs->epc,
"Oops - store (or AMO) address misaligned");
}
#endif
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 04:00:26 +03:00
DO_ERROR_INFO(do_trap_store_fault,
SIGSEGV, SEGV_ACCERR, "store (or AMO) access fault");
DO_ERROR_INFO(do_trap_ecall_u,
SIGILL, ILL_ILLTRP, "environment call from U-mode");
DO_ERROR_INFO(do_trap_ecall_s,
SIGILL, ILL_ILLTRP, "environment call from S-mode");
DO_ERROR_INFO(do_trap_ecall_m,
SIGILL, ILL_ILLTRP, "environment call from M-mode");
riscv: Add the support for c.ebreak check in is_valid_bugaddr() The macro __BUG_INSN currently is defined as the "ebreak" opcode. The is_valid_bugaddr() function compares the instruction pointed to by $sepc with macro __BUG_INSN to check whether the current trap exception is caused by an "ebreak" instruction. However, this check flow is possibly erroneous because if C extension is supported, the expected trap instruction "ebreak" is possibly translated to "c.ebreak" by the assembler. Therefore, it requires a mechanism to distinguish the length of the instruction in $spec and compare it to the correct trap instruction. Signed-off-by: Vincent Chen <vincentc@andestech.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2019-03-05 06:23:34 +03:00
static inline unsigned long get_break_insn_length(unsigned long pc)
{
bug_insn_t insn;
maccess: rename probe_kernel_address to get_kernel_nofault Better describe what this helper does, and match the naming of copy_from_kernel_nofault. Also switch the argument order around, so that it acts and looks like get_user(). Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2020-06-17 10:37:55 +03:00
if (get_kernel_nofault(insn, (bug_insn_t *)pc))
riscv: Add the support for c.ebreak check in is_valid_bugaddr() The macro __BUG_INSN currently is defined as the "ebreak" opcode. The is_valid_bugaddr() function compares the instruction pointed to by $sepc with macro __BUG_INSN to check whether the current trap exception is caused by an "ebreak" instruction. However, this check flow is possibly erroneous because if C extension is supported, the expected trap instruction "ebreak" is possibly translated to "c.ebreak" by the assembler. Therefore, it requires a mechanism to distinguish the length of the instruction in $spec and compare it to the correct trap instruction. Signed-off-by: Vincent Chen <vincentc@andestech.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2019-03-05 06:23:34 +03:00
return 0;
riscv: add macro to get instruction length Extract the calculation of instruction length for common use. Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-09 19:55:42 +03:00
return GET_INSN_LENGTH(insn);
riscv: Add the support for c.ebreak check in is_valid_bugaddr() The macro __BUG_INSN currently is defined as the "ebreak" opcode. The is_valid_bugaddr() function compares the instruction pointed to by $sepc with macro __BUG_INSN to check whether the current trap exception is caused by an "ebreak" instruction. However, this check flow is possibly erroneous because if C extension is supported, the expected trap instruction "ebreak" is possibly translated to "c.ebreak" by the assembler. Therefore, it requires a mechanism to distinguish the length of the instruction in $spec and compare it to the correct trap instruction. Signed-off-by: Vincent Chen <vincentc@andestech.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2019-03-05 06:23:34 +03:00
}
riscv: xip: support runtime trap patching RISCV_ERRATA_ALTERNATIVE patches text at runtime which is currently not possible when the kernel is executed from the flash in XIP mode. Since runtime patching concerns only traps at the moment, let's just have all the traps reside in RAM anyway if RISCV_ERRATA_ALTERNATIVE is set. Thus, these functions will be patch-able even when the .text section is in flash. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-05-31 11:53:42 +03:00
asmlinkage __visible __trap_section void do_trap_break(struct pt_regs *regs)
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 04:00:26 +03:00
{
riscv: Add kprobes supported This patch enables "kprobe & kretprobe" to work with ftrace interface. It utilized software breakpoint as single-step mechanism. Some instructions which can't be single-step executed must be simulated in kernel execution slot, such as: branch, jal, auipc, la ... Some instructions should be rejected for probing and we use a blacklist to filter, such as: ecall, ebreak, ... We use ebreak & c.ebreak to replace origin instruction and the kprobe handler prepares an executable memory slot for out-of-line execution with a copy of the original instruction being probed. In execution slot we add ebreak behind original instruction to simulate a single-setp mechanism. The patch is based on packi's work [1] and csky's work [2]. - The kprobes_trampoline.S is all from packi's patch - The single-step mechanism is new designed for riscv without hw single-step trap - The simulation codes are from csky - Frankly, all codes refer to other archs' implementation [1] https://lore.kernel.org/linux-riscv/20181113195804.22825-1-me@packi.ch/ [2] https://lore.kernel.org/linux-csky/20200403044150.20562-9-guoren@kernel.org/ Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Co-developed-by: Patrick Stählin <me@packi.ch> Signed-off-by: Patrick Stählin <me@packi.ch> Acked-by: Masami Hiramatsu <mhiramat@kernel.org> Tested-by: Zong Li <zong.li@sifive.com> Reviewed-by: Pekka Enberg <penberg@kernel.org> Cc: Patrick Stählin <me@packi.ch> Cc: Palmer Dabbelt <palmerdabbelt@google.com> Cc: Björn Töpel <bjorn.topel@gmail.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-12-17 19:01:42 +03:00
#ifdef CONFIG_KPROBES
if (kprobe_single_step_handler(regs))
return;
if (kprobe_breakpoint_handler(regs))
return;
#endif
riscv: Add uprobes supported This patch adds support for uprobes on riscv architecture. Just like kprobe, it support single-step and simulate instructions. Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Reviewed-by: Pekka Enberg <penberg@kernel.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-12-17 19:01:44 +03:00
#ifdef CONFIG_UPROBES
if (uprobe_single_step_handler(regs))
return;
if (uprobe_breakpoint_handler(regs))
return;
#endif
current->thread.bad_cause = regs->cause;
riscv: Add kprobes supported This patch enables "kprobe & kretprobe" to work with ftrace interface. It utilized software breakpoint as single-step mechanism. Some instructions which can't be single-step executed must be simulated in kernel execution slot, such as: branch, jal, auipc, la ... Some instructions should be rejected for probing and we use a blacklist to filter, such as: ecall, ebreak, ... We use ebreak & c.ebreak to replace origin instruction and the kprobe handler prepares an executable memory slot for out-of-line execution with a copy of the original instruction being probed. In execution slot we add ebreak behind original instruction to simulate a single-setp mechanism. The patch is based on packi's work [1] and csky's work [2]. - The kprobes_trampoline.S is all from packi's patch - The single-step mechanism is new designed for riscv without hw single-step trap - The simulation codes are from csky - Frankly, all codes refer to other archs' implementation [1] https://lore.kernel.org/linux-riscv/20181113195804.22825-1-me@packi.ch/ [2] https://lore.kernel.org/linux-csky/20200403044150.20562-9-guoren@kernel.org/ Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Co-developed-by: Patrick Stählin <me@packi.ch> Signed-off-by: Patrick Stählin <me@packi.ch> Acked-by: Masami Hiramatsu <mhiramat@kernel.org> Tested-by: Zong Li <zong.li@sifive.com> Reviewed-by: Pekka Enberg <penberg@kernel.org> Cc: Patrick Stählin <me@packi.ch> Cc: Palmer Dabbelt <palmerdabbelt@google.com> Cc: Björn Töpel <bjorn.topel@gmail.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-12-17 19:01:42 +03:00
riscv: cleanup do_trap_break If we always compile the get_break_insn_length inline function we can remove the ifdefs and let dead code elimination take care of the warn branch that is now unreadable because the report_bug stub always returns BUG_TRAP_TYPE_BUG. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-17 20:37:30 +03:00
if (user_mode(regs))
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 15:10:32 +03:00
force_sig_fault(SIGTRAP, TRAP_BRKPT, (void __user *)regs->epc);
riscv: Add KGDB support The skeleton of RISC-V KGDB port. Signed-off-by: Vincent Chen <vincent.chen@sifive.com> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-16 05:38:05 +03:00
#ifdef CONFIG_KGDB
else if (notify_die(DIE_TRAP, "EBREAK", regs, 0, regs->cause, SIGTRAP)
== NOTIFY_STOP)
return;
#endif
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 15:10:32 +03:00
else if (report_bug(regs->epc, regs) == BUG_TRAP_TYPE_WARN)
regs->epc += get_break_insn_length(regs->epc);
riscv: cleanup do_trap_break If we always compile the get_break_insn_length inline function we can remove the ifdefs and let dead code elimination take care of the warn branch that is now unreadable because the report_bug stub always returns BUG_TRAP_TYPE_BUG. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-17 20:37:30 +03:00
else
die(regs, "Kernel BUG");
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 04:00:26 +03:00
}
riscv: add do_page_fault and do_trap_break into the kprobes blacklist These two functions are used to implement the kprobes feature so they can't be kprobed. Fixes: c22b0bcb1dd0 ("riscv: Add kprobes supported") Cc: stable@vger.kernel.org Signed-off-by: Jisheng Zhang <jszhang@kernel.org> Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-03-29 21:12:26 +03:00
NOKPROBE_SYMBOL(do_trap_break);
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 04:00:26 +03:00
#ifdef CONFIG_GENERIC_BUG
int is_valid_bugaddr(unsigned long pc)
{
bug_insn_t insn;
riscv: Support BUG() in kernel module The kernel module is loaded into vmalloc region which is located below to the PAGE_OFFSET. Hence the condition, pc < PAGE_OFFSET, in the is_valid_bugaddr() will filter out all trap exceptions triggered by kernel module. To support BUG() in kernel module, the condition is changed to pc < VMALLOC_START. Signed-off-by: Vincent Chen <vincentc@andestech.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2019-03-05 06:23:35 +03:00
if (pc < VMALLOC_START)
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 04:00:26 +03:00
return 0;
maccess: rename probe_kernel_address to get_kernel_nofault Better describe what this helper does, and match the naming of copy_from_kernel_nofault. Also switch the argument order around, so that it acts and looks like get_user(). Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2020-06-17 10:37:55 +03:00
if (get_kernel_nofault(insn, (bug_insn_t *)pc))
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 04:00:26 +03:00
return 0;
riscv: Add the support for c.ebreak check in is_valid_bugaddr() The macro __BUG_INSN currently is defined as the "ebreak" opcode. The is_valid_bugaddr() function compares the instruction pointed to by $sepc with macro __BUG_INSN to check whether the current trap exception is caused by an "ebreak" instruction. However, this check flow is possibly erroneous because if C extension is supported, the expected trap instruction "ebreak" is possibly translated to "c.ebreak" by the assembler. Therefore, it requires a mechanism to distinguish the length of the instruction in $spec and compare it to the correct trap instruction. Signed-off-by: Vincent Chen <vincentc@andestech.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2019-03-05 06:23:34 +03:00
if ((insn & __INSN_LENGTH_MASK) == __INSN_LENGTH_32)
return (insn == __BUG_INSN_32);
else
return ((insn & __COMPRESSED_INSN_MASK) == __BUG_INSN_16);
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-11 04:00:26 +03:00
}
#endif /* CONFIG_GENERIC_BUG */
riscv: add VMAP_STACK overflow detection This patch adds stack overflow detection to riscv, usable when CONFIG_VMAP_STACK=y. Overflow is detected in kernel exception entry(kernel/entry.S), if the kernel stack is overflow and been detected, the overflow handler is invoked on a per-cpu overflow stack. This approach preserves GPRs and the original exception information. The overflow detect is performed before any attempt is made to access the stack and the principle of stack overflow detection: kernel stacks are aligned to double their size, enabling overflow to be detected with a single bit test. For example, a 16K stack is aligned to 32K, ensuring that bit 14 of the SP must be zero. On an overflow (or underflow), this bit is flipped. Thus, overflow (of less than the size of the stack) can be detected by testing whether this bit is set. This gives us a useful error message on stack overflow, as can be trigger with the LKDTM overflow test: [ 388.053267] lkdtm: Performing direct entry EXHAUST_STACK [ 388.053663] lkdtm: Calling function with 1024 frame size to depth 32 ... [ 388.054016] lkdtm: loop 32/32 ... [ 388.054186] lkdtm: loop 31/32 ... [ 388.054491] lkdtm: loop 30/32 ... [ 388.054672] lkdtm: loop 29/32 ... [ 388.054859] lkdtm: loop 28/32 ... [ 388.055010] lkdtm: loop 27/32 ... [ 388.055163] lkdtm: loop 26/32 ... [ 388.055309] lkdtm: loop 25/32 ... [ 388.055481] lkdtm: loop 24/32 ... [ 388.055653] lkdtm: loop 23/32 ... [ 388.055837] lkdtm: loop 22/32 ... [ 388.056015] lkdtm: loop 21/32 ... [ 388.056188] lkdtm: loop 20/32 ... [ 388.058145] Insufficient stack space to handle exception! [ 388.058153] Task stack: [0xffffffd014260000..0xffffffd014264000] [ 388.058160] Overflow stack: [0xffffffe1f8d2c220..0xffffffe1f8d2d220] [ 388.058168] CPU: 0 PID: 89 Comm: bash Not tainted 5.12.0-rc8-dirty #90 [ 388.058175] Hardware name: riscv-virtio,qemu (DT) [ 388.058187] epc : number+0x32/0x2c0 [ 388.058247] ra : vsnprintf+0x2ae/0x3f0 [ 388.058255] epc : ffffffe0002d38f6 ra : ffffffe0002d814e sp : ffffffd01425ffc0 [ 388.058263] gp : ffffffe0012e4010 tp : ffffffe08014da00 t0 : ffffffd0142606e8 [ 388.058271] t1 : 0000000000000000 t2 : 0000000000000000 s0 : ffffffd014260070 [ 388.058303] s1 : ffffffd014260158 a0 : ffffffd01426015e a1 : ffffffd014260158 [ 388.058311] a2 : 0000000000000013 a3 : ffff0a01ffffff10 a4 : ffffffe000c398e0 [ 388.058319] a5 : 511b02ec65f3e300 a6 : 0000000000a1749a a7 : 0000000000000000 [ 388.058327] s2 : ffffffff000000ff s3 : 00000000ffff0a01 s4 : ffffffe0012e50a8 [ 388.058335] s5 : 0000000000ffff0a s6 : ffffffe0012e50a8 s7 : ffffffe000da1cc0 [ 388.058343] s8 : ffffffffffffffff s9 : ffffffd0142602b0 s10: ffffffd0142602a8 [ 388.058351] s11: ffffffd01426015e t3 : 00000000000f0000 t4 : ffffffffffffffff [ 388.058359] t5 : 000000000000002f t6 : ffffffd014260158 [ 388.058366] status: 0000000000000100 badaddr: ffffffd01425fff8 cause: 000000000000000f [ 388.058374] Kernel panic - not syncing: Kernel stack overflow [ 388.058381] CPU: 0 PID: 89 Comm: bash Not tainted 5.12.0-rc8-dirty #90 [ 388.058387] Hardware name: riscv-virtio,qemu (DT) [ 388.058393] Call Trace: [ 388.058400] [<ffffffe000004944>] walk_stackframe+0x0/0xce [ 388.058406] [<ffffffe0006f0b28>] dump_backtrace+0x38/0x46 [ 388.058412] [<ffffffe0006f0b46>] show_stack+0x10/0x18 [ 388.058418] [<ffffffe0006f3690>] dump_stack+0x74/0x8e [ 388.058424] [<ffffffe0006f0d52>] panic+0xfc/0x2b2 [ 388.058430] [<ffffffe0006f0acc>] print_trace_address+0x0/0x24 [ 388.058436] [<ffffffe0002d814e>] vsnprintf+0x2ae/0x3f0 [ 388.058956] SMP: stopping secondary CPUs Signed-off-by: Tong Tiangen <tongtiangen@huawei.com> Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-06-21 06:28:55 +03:00
#ifdef CONFIG_VMAP_STACK
static DEFINE_PER_CPU(unsigned long [OVERFLOW_STACK_SIZE/sizeof(long)],
overflow_stack)__aligned(16);
/*
* shadow stack, handled_ kernel_ stack_ overflow(in kernel/entry.S) is used
* to get per-cpu overflow stack(get_overflow_stack).
*/
long shadow_stack[SHADOW_OVERFLOW_STACK_SIZE/sizeof(long)];
asmlinkage unsigned long get_overflow_stack(void)
{
return (unsigned long)this_cpu_ptr(overflow_stack) +
OVERFLOW_STACK_SIZE;
}
asmlinkage void handle_bad_stack(struct pt_regs *regs)
{
unsigned long tsk_stk = (unsigned long)current->stack;
unsigned long ovf_stk = (unsigned long)this_cpu_ptr(overflow_stack);
console_verbose();
pr_emerg("Insufficient stack space to handle exception!\n");
pr_emerg("Task stack: [0x%016lx..0x%016lx]\n",
tsk_stk, tsk_stk + THREAD_SIZE);
pr_emerg("Overflow stack: [0x%016lx..0x%016lx]\n",
ovf_stk, ovf_stk + OVERFLOW_STACK_SIZE);
__show_regs(regs);
panic("Kernel stack overflow");
for (;;)
wait_for_interrupt();
}
#endif