Nicholas Piggin 9d08fce64d workqueue: Improve scalability of workqueue watchdog touch ... [ Upstream commit 98f887f820c993e05a12e8aa816c80b8661d4c87 ] On a ~2000 CPU powerpc system, hard lockups have been observed in the workqueue code when stop_machine runs (in this case due to CPU hotplug). This is due to lots of CPUs spinning in multi_cpu_stop, calling touch_nmi_watchdog() which ends up calling wq_watchdog_touch(). wq_watchdog_touch() writes to the global variable wq_watchdog_touched, and that can find itself in the same cacheline as other important workqueue data, which slows down operations to the point of lockups. In the case of the following abridged trace, worker_pool_idr was in the hot line, causing the lockups to always appear at idr_find. watchdog: CPU 1125 self-detected hard LOCKUP @ idr_find Call Trace: get_work_pool __queue_work call_timer_fn run_timer_softirq __do_softirq do_softirq_own_stack irq_exit timer_interrupt decrementer_common_virt * interrupt: 900 (timer) at multi_cpu_stop multi_cpu_stop cpu_stopper_thread smpboot_thread_fn kthread Fix this by having wq_watchdog_touch() only write to the line if the last time a touch was recorded exceeds 1/4 of the watchdog threshold. Reported-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com> Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Reviewed-by: Paul E. McKenney <paulmck@kernel.org> Signed-off-by: Tejun Heo <tj@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org>		2024-09-12 11:07:52 +02:00
..
bpf	bpf: Eliminate remaining "make W=1" warnings in kernel/bpf/btf.o	2024-08-19 05:44:59 +02:00
cgroup	cgroup: Protect css->cgroup write under css_set_lock	2024-09-12 11:07:48 +02:00
configs	…
debug	kdb: Use the passed prompt in kdb_position_cursor()	2024-08-19 05:45:22 +02:00
dma	dma-mapping: benchmark: Don't starve others when doing the test	2024-09-12 11:07:48 +02:00
entry	entry: Respect changes to system call number by trace_sys_enter()	2024-04-10 16:18:46 +02:00
events	perf/aux: Fix AUX buffer serialization	2024-09-12 11:07:51 +02:00
futex	futex: Don't include process MM in futex key on no-MMU	2023-11-20 11:08:13 +01:00
gcov	gcov: add support for GCC 14	2024-07-05 09:14:34 +02:00
irq	genirq/irqdesc: Honor caller provided affinity in alloc_desc()	2024-08-19 05:45:46 +02:00
kcsan	kcsan: Don't expect 64 bits atomic builtins from 32 bits architectures	2023-07-23 13:47:12 +02:00
livepatch	livepatch: Fix missing newline character in klp_resolve_symbols()	2023-11-20 11:08:25 +01:00
locking	rtmutex: Drop rt_mutex::wait_lock before scheduling	2024-09-12 11:07:43 +02:00
power	PM: suspend: Set mem_sleep_current during kernel command line setup	2024-04-10 16:18:36 +02:00
printk	printk: Update @console_may_schedule in console_trylock_spinning()	2024-04-10 16:18:47 +02:00
rcu	rcu-tasks: Fix show_rcu_tasks_trace_gp_kthread buffer overflow	2024-09-12 11:07:42 +02:00
sched	sched/smt: Fix unbalance sched_smt_present dec/inc	2024-08-19 05:45:47 +02:00
time	hrtimer: Prevent queuing of hrtimer without a function callback	2024-09-04 13:23:28 +02:00
trace	tracing: Avoid possible softlockup in tracing_iter_reset()	2024-09-12 11:07:44 +02:00
.gitignore	…
Kconfig.freezer	…
Kconfig.hz	…
Kconfig.locks	locking/rwlock: Provide RT variant	2021-08-17 17:50:51 +02:00
Kconfig.preempt	…
Makefile	futex: Move to kernel/futex/	2022-12-31 13:14:04 +01:00
acct.c	acct: fix potential integer overflow in encode_comp_t()	2022-12-31 13:14:40 +01:00
async.c	async: Introduce async_schedule_dev_nocall()	2024-02-23 08:54:25 +01:00
audit.c	audit: Send netlink ACK before setting connection in auditd_set	2024-02-23 08:54:37 +01:00
audit.h	audit: log AUDIT_TIME_* records only from rules	2022-04-08 14:23:06 +02:00
audit_fsnotify.c	fsnotify: make allow_dups a property of the group	2024-04-10 16:19:02 +02:00
audit_tree.c	fsnotify: pass flags argument to fsnotify_alloc_group()	2024-04-10 16:19:02 +02:00
audit_watch.c	fsnotify: pass flags argument to fsnotify_alloc_group()	2024-04-10 16:19:02 +02:00
auditfilter.c	ima: Avoid blocking in RCU read-side critical section	2024-07-18 13:07:34 +02:00
auditsc.c	audit: fix possible soft lockup in __audit_inode_child()	2023-09-19 12:22:39 +02:00
backtracetest.c	…
bounds.c	bounds: Use the right number of bits for power-of-two CONFIG_NR_CPUS	2024-05-02 16:24:50 +02:00
capability.c	…
cfi.c	cfi: Fix __cfi_slowpath_diag RCU usage with cpuidle	2022-06-22 14:22:04 +02:00
compat.c	sched_getaffinity: don't assume 'cpumask_size()' is fully initialized	2023-04-05 11:24:53 +02:00
configs.c	…
context_tracking.c	…
cpu.c	cpu/hotplug: Fix dynstate assignment in __cpuhp_setup_state_cpuslocked()	2024-07-05 09:14:48 +02:00
cpu_pm.c	PM: cpu: Make notifier chain use a raw_spinlock_t	2021-08-16 18:55:32 +02:00
crash_core.c	kernel/crash_core: suppress unknown crashkernel parameter warning	2021-12-29 12:28:49 +01:00
crash_dump.c	…
cred.c	cred: switch to using atomic_long_t	2023-12-20 15:17:37 +01:00
delayacct.c	…
dma.c	…
exec_domain.c	…
exit.c	mm: optimize the redundant loop of mm_update_owner_next()	2024-07-18 13:07:32 +02:00
extable.c	…
fail_function.c	kernel/fail_function: fix memory leak with using debugfs_lookup()	2023-03-11 13:57:38 +01:00
fork.c	kernel/fork: beware of __put_task_struct() calling context	2023-09-23 11:09:55 +02:00
freezer.c	…
gen_kheaders.sh	kheaders: explicitly define file modes for archived headers	2024-07-05 09:14:37 +02:00
groups.c	…
hung_task.c	…
iomem.c	…
irq_work.c	…
jump_label.c	…
kallsyms.c	kallsyms: Make kallsyms_on_each_symbol generally available	2023-12-13 18:36:45 +01:00
kcmp.c	…
kcov.c	kcov: properly check for softirq context	2024-08-19 05:45:46 +02:00
kexec.c	kernel: kexec: copy user-array safely	2023-11-28 16:56:16 +00:00
kexec_core.c	kexec: fix a memory leak in crash_shrink_memory()	2023-07-23 13:46:52 +02:00
kexec_elf.c	…
kexec_file.c	kexec: support purgatories with .text.hot sections	2023-06-21 15:59:14 +02:00
kexec_internal.h	panic, kexec: make __crash_kexec() NMI safe	2023-04-20 12:13:57 +02:00
kheaders.c	kheaders: Use array declaration instead of char	2023-05-11 23:00:17 +09:00
kmod.c	…
kprobes.c	kprobes: Fix to check symbol prefixes correctly	2024-08-19 05:45:42 +02:00
ksysfs.c	kexec: turn all kexec_mutex acquisitions into trylocks	2023-04-20 12:13:57 +02:00
kthread.c	exit: Implement kthread_exit	2024-04-10 16:18:55 +02:00
latencytop.c	…
module-internal.h	…
module.c	NFSD: Remove svc_serv_ops::svo_module	2024-04-10 16:19:01 +02:00
module_signature.c	…
module_signing.c	…
notifier.c	notifier: Remove atomic_notifier_call_chain_robust()	2021-08-16 18:55:32 +02:00
nsproxy.c	memcg: enable accounting for new namesapces and struct nsproxy	2021-09-03 09:58:12 -07:00
padata.c	padata: Fix possible divide-by-0 panic in padata_mt_helper()	2024-08-19 05:45:47 +02:00
panic.c	panic: Flush kernel log buffer at the end	2024-04-13 13:01:43 +02:00
params.c	params: lift param_set_uint_minmax to common code	2021-08-16 14:42:22 +02:00
pid.c	…
pid_namespace.c	zap_pid_ns_processes: clear TIF_NOTIFY_SIGNAL along with TIF_SIGPENDING	2024-07-05 09:14:24 +02:00
profile.c	profiling: remove profile=sleep support	2024-08-19 05:45:39 +02:00
ptrace.c	ptrace: Reimplement PTRACE_KILL by always sending SIGKILL	2022-06-09 10:22:29 +02:00
range.c	…
reboot.c	kernel/reboot: emergency_restart: Set correct system_state	2023-11-28 16:56:31 +00:00
regset.c	…
relay.c	relayfs: fix out-of-bounds access in relay_file_read	2023-05-11 23:00:18 +09:00
resource.c	dax/kmem: Fix leak of memory-hotplug resources	2023-03-10 09:40:08 +01:00
resource_kunit.c	…
rseq.c	rseq: Remove broken uapi field layout on 32-bit little endian	2022-04-08 14:23:10 +02:00
scftorture.c	scftorture: Forgive memory-allocation failure if KASAN	2023-09-23 11:09:55 +02:00
scs.c	scs: Release kasan vmalloc poison in scs_free process	2021-11-18 19:16:29 +01:00
seccomp.c	seccomp: Invalidate seccomp mode to catch death failures	2022-02-16 12:56:38 +01:00
signal.c	kernel: rerun task_work while freezing in get_signal()	2024-08-19 05:45:22 +02:00
smp.c	smp: Add missing destroy_work_on_stack() call in smp_call_on_cpu()	2024-09-12 11:07:49 +02:00
smpboot.c	…
smpboot.h	…
softirq.c	softirq: Fix suspicious RCU usage in __do_softirq()	2024-06-16 13:39:15 +02:00
stackleak.c	gcc-plugins/stackleak: Use noinstr in favor of notrace	2022-02-23 12:03:07 +01:00
stacktrace.c	stacktrace: move filter_irq_stacks() to kernel/stacktrace.c	2022-04-13 20:59:28 +02:00
static_call.c	static_call: Don't make __static_call_return0 static	2022-04-13 20:59:28 +02:00
static_call_inline.c	static_call: Don't make __static_call_return0 static	2022-04-13 20:59:28 +02:00
stop_machine.c	…
sys.c	getrusage: use sig->stats_lock rather than lock_task_sighand()	2024-03-15 10:48:22 -04:00
sys_ni.c	syscalls: fix compat_sys_io_pgetevents_time64 usage	2024-07-05 09:14:50 +02:00
sysctl-test.c	…
sysctl.c	sched/rt: Disallow writing invalid values to sched_rt_period_us	2024-03-01 13:21:43 +01:00
task_work.c	task_work: Introduce task_work_cancel() again	2024-08-19 05:45:13 +02:00
taskstats.c	…
test_kprobes.c	…
torture.c	torture: Fix hang during kthread shutdown phase	2023-08-30 16:18:19 +02:00
tracepoint.c	tracepoint: Fix kerneldoc comments	2021-08-16 11:39:51 -04:00
tsacct.c	taskstats: Cleanup the use of task->exit_code	2022-01-27 11:05:35 +01:00
ucount.c	ucounts: Handle wrapping in is_ucounts_overlimit	2022-02-23 12:03:20 +01:00
uid16.c	…
uid16.h	…
umh.c	…
up.c	…
user-return-notifier.c	…
user.c	fs/epoll: use a per-cpu counter for user's watches count	2021-09-08 11:50:27 -07:00
user_namespace.c	ucounts: Fix systemd LimitNPROC with private users regression	2022-03-08 19:12:42 +01:00
usermode_driver.c	…
utsname.c	…
utsname_sysctl.c	…
watch_queue.c	kernel: watch_queue: copy user-array safely	2023-11-28 16:56:16 +00:00
watchdog.c	watchdog: move softlockup_panic back to early_param	2023-11-28 16:56:28 +00:00
watchdog_hld.c	watchdog/perf: properly initialize the turbo mode timestamp and rearm counter	2024-08-19 05:45:20 +02:00
workqueue.c	workqueue: Improve scalability of workqueue watchdog touch	2024-09-12 11:07:52 +02:00
workqueue_internal.h	workqueue: Assign a color to barrier work items	2021-08-17 07:49:10 -10:00