From: "Russell King (Oracle)" <linux@armlinux•org.uk>
To: Peter Xu <peterx@redhat•com>
Cc: Alistair Popple <apopple@nvidia•com>,
linux-ia64@vger•kernel.org, linux-xtensa@linux-xtensa•org,
Peter Zijlstra <peterz@infradead•org>,
Dave Hansen <dave.hansen@linux•intel.com>,
David Hildenbrand <david@redhat•com>,
linux-mips@vger•kernel.org,
"James E . J . Bottomley" <James.Bottomley@hansenpartnership•com>,
linux-mm@kvack•org, Rich Felker <dalias@libc•org>,
Paul Mackerras <paulus@samba•org>,
"H . Peter Anvin" <hpa@zytor•com>,
sparclinux@vger•kernel.org,
Alexander Gordeev <agordeev@linux•ibm.com>,
Will Deacon <will@kernel•org>, Stafford Horne <shorne@gmail•com>,
Anton Ivanov <anton.ivanov@cambridgegreys•com>,
Andrea Arcangeli <aarcange@redhat•com>,
linux-s390@vger•kernel.org, linux-snps-arc@lists•infradead.org,
Janosch Frank <frankja@linux•ibm.com>,
Yoshinori Sato <ysato@users•sourceforge.jp>,
linux-sh@vger•kernel.org, Richard Weinberger <richard@nod•at>,
Helge Deller <deller@gmx•de>,
x86@kernel•org, Hugh Dickins <hughd@google•com>,
linux-csky@vger•kernel.org, Ingo Molnar <ming o@kernel•org>,
linux-alpha@vger•kernel.org, Ingo Molnar <mingo@redhat•com>,
Geert Uytterhoeven <geert@linux-m68k•org>,
linux-arm-kernel@lists•infradead.org,
Vineet Gupta <vgupta@kernel•org>,
Matt Turner <mattst88@gmail•com>,
Christian Borntraeger <borntraeger@linux•ibm.com>,
Catalin Marinas <catalin.marinas@arm•com>,
Jonas Bonn <jonas@southpole•se>,
Albert Ou <aou@eecs•berkeley.edu>,
Vasily Gorbik <gor@linux•ibm.com>,
Chris Zankel <chris@zankel•net>,
Heiko Carstens <hca@linux•ibm.com>,
Johannes Weiner <hannes@cmpxchg•org>,
linux-um@lists•infradead.org, Nicholas Piggin <npiggin@gmail•com>,
Stefan Kristiansson <stefan.kristiansson@saunalahti•fi>,
linux-m68k@lists•linux-m68k.org, openrisc@lists•librecores.org,
Borislav Petkov <bp@alien8•de>, Al Viro <viro@zeniv•linux.org.uk>,
Andy Lutomirski <luto@kernel•org>,
Paul Walmsley <paul.walmsley@sifive•com>,
Thomas Gleixner <tglx@linutronix•de>,
Andrew Morton <akpm@linux-foundation•org>,
Vlastimil Babka <vbabka@suse•cz>,
Richard Henderson <rt h@twiddle•net>,
Brian Cain <bcain@quicinc•com>,
Micha
Subject: Re: [PATCH v5] mm: Avoid unnecessary page fault retires on shared memory types
Date: Mon, 30 May 2022 23:26:09 +0100 [thread overview]
Message-ID: <YpVEgWHzzH3ZtVzA@shell.armlinux.org.uk> (raw)
In-Reply-To: <20220530183450.42886-1-peterx@redhat.com>
l Simek <monstr@monstr•eu>, Thomas Bogendoerfer <tsbogend@alpha•franken.de>, linux-parisc@vger•kernel.org, Max Filippov <jcmvbkbc@gmail•com>, linux-kernel@vger•kernel.org, Dinh Nguyen <dinguyen@kernel•org>, linux-riscv@lists•infradead.org, Palmer Dabbelt <palmer@dabbelt•com>, Sven Schnelle <svens@linux•ibm.com>, Guo Ren <guoren@kernel•org>, linux-hexagon@vger•kernel.org, Ivan Kokshaysky <ink@jurassic•park.msu.ru>, Johannes Berg <johannes@sipsolutions•net>, linuxppc-dev@lists•ozlabs.org, "David S . Miller" <davem@davemloft•net>
Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists•ozlabs.org
Sender: "Linuxppc-dev" <linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists•ozlabs.org>
On Mon, May 30, 2022 at 02:34:50PM -0400, Peter Xu wrote:
> I observed that for each of the shared file-backed page faults, we're very
> likely to retry one more time for the 1st write fault upon no page. It's
> because we'll need to release the mmap lock for dirty rate limit purpose
> with balance_dirty_pages_ratelimited() (in fault_dirty_shared_page()).
>
> Then after that throttling we return VM_FAULT_RETRY.
>
> We did that probably because VM_FAULT_RETRY is the only way we can return
> to the fault handler at that time telling it we've released the mmap lock.
>
> However that's not ideal because it's very likely the fault does not need
> to be retried at all since the pgtable was well installed before the
> throttling, so the next continuous fault (including taking mmap read lock,
> walk the pgtable, etc.) could be in most cases unnecessary.
>
> It's not only slowing down page faults for shared file-backed, but also add
> more mmap lock contention which is in most cases not needed at all.
>
> To observe this, one could try to write to some shmem page and look at
> "pgfault" value in /proc/vmstat, then we should expect 2 counts for each
> shmem write simply because we retried, and vm event "pgfault" will capture
> that.
>
> To make it more efficient, add a new VM_FAULT_COMPLETED return code just to
> show that we've completed the whole fault and released the lock. It's also
> a hint that we should very possibly not need another fault immediately on
> this page because we've just completed it.
>
> This patch provides a ~12% perf boost on my aarch64 test VM with a simple
> program sequentially dirtying 400MB shmem file being mmap()ed and these are
> the time it needs:
>
> Before: 650.980 ms (+-1.94%)
> After: 569.396 ms (+-1.38%)
>
> I believe it could help more than that.
>
> We need some special care on GUP and the s390 pgfault handler (for gmap
> code before returning from pgfault), the rest changes in the page fault
> handlers should be relatively straightforward.
>
> Another thing to mention is that mm_account_fault() does take this new
> fault as a generic fault to be accounted, unlike VM_FAULT_RETRY.
>
> I explicitly didn't touch hmm_vma_fault() and break_ksm() because they do
> not handle VM_FAULT_RETRY even with existing code, so I'm literally keeping
> them as-is.
>
> Acked-by: Geert Uytterhoeven <geert@linux-m68k•org>
> Acked-by: Peter Zijlstra (Intel) <peterz@infradead•org>
> Acked-by: Johannes Weiner <hannes@cmpxchg•org>
> Acked-by: Vineet Gupta <vgupta@kernel•org>
> Acked-by: Guo Ren <guoren@kernel•org>
> Acked-by: Max Filippov <jcmvbkbc@gmail•com>
> Acked-by: Christian Borntraeger <borntraeger@linux•ibm.com>
> Acked-by: Michael Ellerman <mpe@ellerman•id.au> (powerpc)
> Acked-by: Catalin Marinas <catalin.marinas@arm•com>
> Reviewed-by: Alistair Popple <apopple@nvidia•com>
> Reviewed-by: Ingo Molnar <mingo@kernel•org>
> Signed-off-by: Peter Xu <peterx@redhat•com>
For:
> diff --git a/arch/arm/mm/fault.c b/arch/arm/mm/fault.c
> index a062e07516dd..46cccd6bf705 100644
> --- a/arch/arm/mm/fault.c
> +++ b/arch/arm/mm/fault.c
> @@ -322,6 +322,10 @@ do_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs)
> return 0;
> }
>
> + /* The fault is fully completed (including releasing mmap lock) */
> + if (fault & VM_FAULT_COMPLETED)
> + return 0;
> +
> if (!(fault & VM_FAULT_ERROR)) {
> if (fault & VM_FAULT_RETRY) {
> flags |= FAULT_FLAG_TRIED;
Acked-by: Russell King (Oracle) <rmk+kernel@armlinux•org.uk>
Thanks!
--
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!
next prev parent reply other threads:[~2022-05-31 21:48 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-05-30 18:34 [PATCH v5] mm: Avoid unnecessary page fault retires on shared memory types Peter Xu
2022-05-30 22:26 ` Russell King (Oracle) [this message]
2022-05-31 7:59 ` Heiko Carstens
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YpVEgWHzzH3ZtVzA@shell.armlinux.org.uk \
--to=linux@armlinux$(echo .)org.uk \
--cc=James.Bottomley@hansenpartnership$(echo .)com \
--cc=aarcange@redhat$(echo .)com \
--cc=agordeev@linux$(echo .)ibm.com \
--cc=anton.ivanov@cambridgegreys$(echo .)com \
--cc=apopple@nvidia$(echo .)com \
--cc=dalias@libc$(echo .)org \
--cc=dave.hansen@linux$(echo .)intel.com \
--cc=david@redhat$(echo .)com \
--cc=deller@gmx$(echo .)de \
--cc=frankja@linux$(echo .)ibm.com \
--cc=hpa@zytor$(echo .)com \
--cc=hughd@google$(echo .)com \
--cc=linux-csky@vger$(echo .)kernel.org \
--cc=linux-ia64@vger$(echo .)kernel.org \
--cc=linux-mips@vger$(echo .)kernel.org \
--cc=linux-mm@kvack$(echo .)org \
--cc=linux-s390@vger$(echo .)kernel.org \
--cc=linux-sh@vger$(echo .)kernel.org \
--cc=linux-snps-arc@lists$(echo .)infradead.org \
--cc=linux-xtensa@linux-xtensa$(echo .)org \
--cc=paulus@samba$(echo .)org \
--cc=peterx@redhat$(echo .)com \
--cc=peterz@infradead$(echo .)org \
--cc=richard@nod$(echo .)at \
--cc=shorne@gmail$(echo .)com \
--cc=sparclinux@vger$(echo .)kernel.org \
--cc=will@kernel$(echo .)org \
--cc=x86@kernel$(echo .)org \
--cc=ysato@users$(echo .)sourceforge.jp \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox