WSL2-Linux-Kernel/arch/m68k
Peter Xu d92725256b mm: avoid unnecessary page fault retires on shared memory types
I observed that for each of the shared file-backed page faults, we're very
likely to retry one more time for the 1st write fault upon no page.  It's
because we'll need to release the mmap lock for dirty rate limit purpose
with balance_dirty_pages_ratelimited() (in fault_dirty_shared_page()).

Then after that throttling we return VM_FAULT_RETRY.

We did that probably because VM_FAULT_RETRY is the only way we can return
to the fault handler at that time telling it we've released the mmap lock.

However that's not ideal because it's very likely the fault does not need
to be retried at all since the pgtable was well installed before the
throttling, so the next continuous fault (including taking mmap read lock,
walk the pgtable, etc.) could be in most cases unnecessary.

It's not only slowing down page faults for shared file-backed, but also add
more mmap lock contention which is in most cases not needed at all.

To observe this, one could try to write to some shmem page and look at
"pgfault" value in /proc/vmstat, then we should expect 2 counts for each
shmem write simply because we retried, and vm event "pgfault" will capture
that.

To make it more efficient, add a new VM_FAULT_COMPLETED return code just to
show that we've completed the whole fault and released the lock.  It's also
a hint that we should very possibly not need another fault immediately on
this page because we've just completed it.

This patch provides a ~12% perf boost on my aarch64 test VM with a simple
program sequentially dirtying 400MB shmem file being mmap()ed and these are
the time it needs:

  Before: 650.980 ms (+-1.94%)
  After:  569.396 ms (+-1.38%)

I believe it could help more than that.

We need some special care on GUP and the s390 pgfault handler (for gmap
code before returning from pgfault), the rest changes in the page fault
handlers should be relatively straightforward.

Another thing to mention is that mm_account_fault() does take this new
fault as a generic fault to be accounted, unlike VM_FAULT_RETRY.

I explicitly didn't touch hmm_vma_fault() and break_ksm() because they do
not handle VM_FAULT_RETRY even with existing code, so I'm literally keeping
them as-is.

Link: https://lkml.kernel.org/r/20220530183450.42886-1-peterx@redhat.com
Signed-off-by: Peter Xu <peterx@redhat.com>
Acked-by: Geert Uytterhoeven <geert@linux-m68k.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Acked-by: Vineet Gupta <vgupta@kernel.org>
Acked-by: Guo Ren <guoren@kernel.org>
Acked-by: Max Filippov <jcmvbkbc@gmail.com>
Acked-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Acked-by: Michael Ellerman <mpe@ellerman.id.au> (powerpc)
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Alistair Popple <apopple@nvidia.com>
Reviewed-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>	[arm part]
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Stafford Horne <shorne@gmail.com>
Cc: David S. Miller <davem@davemloft.net>
Cc: Johannes Berg <johannes@sipsolutions.net>
Cc: Brian Cain <bcain@quicinc.com>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Richard Weinberger <richard@nod.at>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Janosch Frank <frankja@linux.ibm.com>
Cc: Albert Ou <aou@eecs.berkeley.edu>
Cc: Anton Ivanov <anton.ivanov@cambridgegreys.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Sven Schnelle <svens@linux.ibm.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: James Bottomley <James.Bottomley@HansenPartnership.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Jonas Bonn <jonas@southpole.se>
Cc: Will Deacon <will@kernel.org>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Michal Simek <monstr@monstr.eu>
Cc: Matt Turner <mattst88@gmail.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: David Hildenbrand <david@redhat.com>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Palmer Dabbelt <palmer@dabbelt.com>
Cc: Stefan Kristiansson <stefan.kristiansson@saunalahti.fi>
Cc: Paul Walmsley <paul.walmsley@sifive.com>
Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
Cc: Chris Zankel <chris@zankel.net>
Cc: Hugh Dickins <hughd@google.com>
Cc: Dinh Nguyen <dinguyen@kernel.org>
Cc: Rich Felker <dalias@libc.org>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
Cc: Helge Deller <deller@gmx.de>
Cc: Yoshinori Sato <ysato@users.osdn.me>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2022-06-16 19:48:27 -07:00
..
68000 m68knommu: fix ucsimm sparse warnings 2022-03-07 14:51:12 +10:00
amiga m68k: Add asm/config.h 2022-02-21 12:00:20 +01:00
apollo m68k: Add asm/config.h 2022-02-21 12:00:20 +01:00
atari for-5.18/block-2022-03-18 2022-03-21 16:48:55 -07:00
bvme6000 for-5.18/block-2022-03-18 2022-03-21 16:48:55 -07:00
coldfire m68k: fix typos in comments 2022-05-16 13:18:30 +10:00
configs m68k: defconfig: Update defconfigs for v5.18-rc1 2022-04-25 14:15:02 +02:00
emu m68k: Switch to new sys-off handler API 2022-05-19 19:30:30 +02:00
fpsp040 signal/m68k: Use force_sigsegv(SIGSEGV) in fpsp040_die 2021-08-26 09:59:15 -05:00
hp300 m68k: removed unused "mach_get_ss" 2022-05-25 00:04:11 +10:00
ifpsp060 m68k: fpsp040,ifpsp060: Remove meaningless EXTRA_LDFLAGS 2021-04-06 09:33:20 +02:00
include More power management updates for 5.19-rc1 2022-05-30 11:37:26 -07:00
kernel This set of changes updates init and user mode helper tasks to be 2022-06-03 16:03:05 -07:00
lib net: unexport csum_and_copy_{from,to}_user 2022-04-29 14:37:59 -07:00
mac m68k: Switch to new sys-off handler API 2022-05-19 19:30:30 +02:00
math-emu m68k: math-emu: Fix typos in comments 2022-05-09 13:13:59 +02:00
mm mm: avoid unnecessary page fault retires on shared memory types 2022-06-16 19:48:27 -07:00
mvme16x for-5.18/block-2022-03-18 2022-03-21 16:48:55 -07:00
mvme147 for-5.18/block-2022-03-18 2022-03-21 16:48:55 -07:00
q40 m68k: removed unused "mach_get_ss" 2022-05-25 00:04:11 +10:00
sun3 m68k: Remove set_fs() 2021-09-24 13:35:07 +02:00
sun3x m68k: Remove set_fs() 2021-09-24 13:35:07 +02:00
tools/amiga m68k: tools: Replace zero-length array with flexible-array member 2020-05-25 10:55:56 +02:00
virt m68k: virt: Switch to new sys-off handler API 2022-05-25 14:39:32 +02:00
Kbuild m68k: Introduce a virtual m68k machine 2022-04-11 11:48:01 +02:00
Kconfig hardening updates for v5.18-rc1 2022-03-21 19:32:04 -07:00
Kconfig.bus m68k: coldfire: drop ISA_DMA_API support 2022-05-16 13:18:30 +10:00
Kconfig.cpu m68knommu: changes for linux 5.19 2022-05-30 10:56:18 -07:00
Kconfig.debug m68k: Fix Kconfig indentation 2020-03-09 11:12:19 +01:00
Kconfig.devices treewide: replace '---help---' in Kconfig files with 'help' 2020-06-14 01:57:21 +09:00
Kconfig.machine m68knommu: changes for linux 5.19 2022-05-30 10:56:18 -07:00
Makefile kbuild: factor out the common installation code into scripts/install.sh 2022-05-11 21:45:53 +09:00
install.sh kbuild: factor out the common installation code into scripts/install.sh 2022-05-11 21:45:53 +09:00