Johannes Weiner 36b238d571 psi: Optimize switching tasks inside shared cgroups ... When switching tasks running on a CPU, the psi state of a cgroup containing both of these tasks does not change. Right now, we don't exploit that, and can perform many unnecessary state changes in nested hierarchies, especially when most activity comes from one leaf cgroup. This patch implements an optimization where we only update cgroups whose state actually changes during a task switch. These are all cgroups that contain one task but not the other, up to the first shared ancestor. When both tasks are in the same group, we don't need to update anything at all. We can identify the first shared ancestor by walking the groups of the incoming task until we see TSK_ONCPU set on the local CPU; that's the first group that also contains the outgoing task. The new psi_task_switch() is similar to psi_task_change(). To allow code reuse, move the task flag maintenance code into a new function and the poll/avg worker wakeups into the shared psi_group_change(). Suggested-by: Peter Zijlstra <peterz@infradead.org> Signed-off-by: Johannes Weiner <hannes@cmpxchg.org> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lkml.kernel.org/r/20200316191333.115523-3-hannes@cmpxchg.org		2020-03-20 13:06:19 +01:00
..
bpf	bpf: Fix a potential deadlock with bpf_map_do_batch	2020-02-19 16:01:25 -08:00
cgroup	Merge branch 'for-5.6-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup	2020-02-10 17:07:05 -08:00
configs	…
debug	Revert "kdb: Get rid of confusing diag msg from "rd" if current task has no regs"	2020-02-06 11:40:09 +00:00
dma	dma-direct: improve DMA mask overflow reporting	2020-02-05 18:53:41 +01:00
events	A set of fixes and improvements for the perf subsystem:	2020-02-09 12:04:09 -08:00
gcov	Revert "um: Enable CONFIG_CONSTRUCTORS"	2020-01-19 22:42:06 +01:00
irq	genirq/proc: Reject invalid affinity masks (again)	2020-02-14 09:43:17 +01:00
livepatch	New tracing features:	2019-11-27 11:42:01 -08:00
locking	proc: convert everything to "struct proc_ops"	2020-02-04 03:05:26 +00:00
power	PM / hibernate: fix typo "reserverd_size" -> "reserved_size"	2020-02-20 11:58:01 +01:00
printk	printk: fix exclusive_console replaying	2020-01-02 16:15:04 +01:00
rcu	rcu: Forgive slow expedited grace periods at boot time	2020-01-25 12:00:40 -08:00
sched	psi: Optimize switching tasks inside shared cgroups	2020-03-20 13:06:19 +01:00
time	y2038: remove unused time32 interfaces	2020-02-21 11:22:15 -08:00
trace	block-5.6-2020-02-28	2020-02-28 11:43:30 -08:00
.gitignore	…
Kconfig.freezer	…
Kconfig.hz	…
Kconfig.locks	sched/rt, locking: Use CONFIG_PREEMPTION	2019-12-08 14:37:36 +01:00
Kconfig.preempt	…
Makefile	kcov: ignore fault-inject and stacktrace	2020-01-31 10:30:41 -08:00
acct.c	acct: stop using get_seconds()	2019-12-18 18:07:31 +01:00
async.c	…
audit.c	audit: always check the netlink payload length in audit_receive_msg()	2020-02-24 16:38:57 -05:00
audit.h	…
audit_fsnotify.c	…
audit_tree.c	…
audit_watch.c	…
auditfilter.c	audit: fix error handling in audit_data_to_entry()	2020-02-22 20:36:47 -05:00
auditsc.c	…
backtracetest.c	…
bounds.c	…
capability.c	…
compat.c	y2038: remove unused time32 interfaces	2020-02-21 11:22:15 -08:00
configs.c	proc: convert everything to "struct proc_ops"	2020-02-04 03:05:26 +00:00
context_tracking.c	…
cpu.c	Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip	2020-01-28 10:07:09 -08:00
cpu_pm.c	…
crash_core.c	…
crash_dump.c	…
cred.c	Merge branch 'dhowells' (patches from DavidH)	2020-01-14 09:56:31 -08:00
delayacct.c	…
dma.c	…
elfcore.c	…
exec_domain.c	…
exit.c	for-linus-2020-01-03	2020-01-03 11:17:14 -08:00
extable.c	bpf: Allow to resolve bpf trampoline and dispatcher in unwind	2020-01-25 07:12:40 -08:00
fail_function.c	…
fork.c	hmm related patches for 5.6	2020-01-29 19:56:50 -08:00
freezer.c	…
futex.c	futex: Fix kernel-doc notation warning	2020-01-09 13:23:40 +01:00
gen_kheaders.sh	…
groups.c	…
hung_task.c	…
iomem.c	…
irq_work.c	…
jump_label.c	…
kallsyms.c	Kbuild updates for v5.6 (2nd)	2020-02-09 16:05:50 -08:00
kcmp.c	…
kcov.c	kcov: remote coverage support	2019-12-04 19:44:14 -08:00
kexec.c	kexec: add machine_kexec_post_load()	2020-01-08 16:32:55 +00:00
kexec_core.c	kexec: add machine_kexec_post_load()	2020-01-08 16:32:55 +00:00
kexec_elf.c	…
kexec_file.c	kexec: add machine_kexec_post_load()	2020-01-08 16:32:55 +00:00
kexec_internal.h	kexec: add machine_kexec_post_load()	2020-01-08 16:32:55 +00:00
kheaders.c	…
kmod.c	…
kprobes.c	kprobes: Fix optimize_kprobe()/unoptimize_kprobe() cancellation logic	2020-01-09 12:40:13 +01:00
ksysfs.c	…
kthread.c	…
latencytop.c	proc: convert everything to "struct proc_ops"	2020-02-04 03:05:26 +00:00
module-internal.h	…
module.c	proc: convert everything to "struct proc_ops"	2020-02-04 03:05:26 +00:00
module_signature.c	…
module_signing.c	…
notifier.c	kernel/notifier.c: remove blocking_notifier_chain_cond_register()	2019-12-04 19:44:12 -08:00
nsproxy.c	ns: Introduce Time Namespace	2020-01-14 12:20:48 +01:00
padata.c	padata: update documentation	2019-12-11 16:37:02 +08:00
panic.c	…
params.c	…
pid.c	pid: Implement pidfd_getfd syscall	2020-01-13 21:49:36 +01:00
pid_namespace.c	…
profile.c	proc: convert everything to "struct proc_ops"	2020-02-04 03:05:26 +00:00
ptrace.c	ptrace: reintroduce usage of subjective credentials in ptrace_has_cap()	2020-01-18 13:51:39 +01:00
range.c	…
reboot.c	…
relay.c	…
resource.c	…
rseq.c	rseq: Reject unknown flags on rseq unregister	2019-12-25 10:41:20 +01:00
seccomp.c	seccomp: Check that seccomp_notif is zeroed out by the user	2020-01-02 13:03:45 -08:00
signal.c	signal: avoid double atomic counter increments for user accounting	2020-02-26 09:54:03 -08:00
smp.c	smp: Remove superfluous cond_func check in smp_call_function_many_cond()	2020-01-28 15:43:00 +01:00
smpboot.c	…
smpboot.h	…
softirq.c	…
stackleak.c	…
stacktrace.c	…
stop_machine.c	stop_machine: Make stop_cpus() static	2020-01-17 10:19:21 +01:00
sys.c	prctl: PR_{G,S}ET_IO_FLUSHER to support controlling memory reclaim	2020-01-28 10:09:51 +01:00
sys_ni.c	…
sysctl-test.c	kunit: allow kunit tests to be loaded as a module	2020-01-09 16:42:29 -07:00
sysctl.c	s390: remove obsolete ieee_emulation_warnings	2020-02-19 13:51:46 +01:00
sysctl_binary.c	sysctl: Remove the sysctl system call	2019-11-26 13:03:56 -06:00
task_work.c	…
taskstats.c	taskstats: fix data-race	2019-12-04 15:18:39 +01:00
test_kprobes.c	…
torture.c	…
tracepoint.c	…
tsacct.c	tsacct: add 64-bit btime field	2019-12-18 18:07:31 +01:00
ucount.c	…
uid16.c	…
uid16.h	…
umh.c	…
up.c	smp/up: Make smp_call_function_single() match SMP semantics	2020-02-07 15:34:12 +01:00
user-return-notifier.c	…
user.c	…
user_namespace.c	…
utsname.c	…
utsname_sysctl.c	…
watchdog.c	watchdog/softlockup: Enforce that timestamp is valid on boot	2020-01-17 11:19:22 +01:00
watchdog_hld.c	…
workqueue.c	Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip	2020-01-28 10:07:09 -08:00
workqueue_internal.h	…