[PATCH V2 4/7] mm/gup: Add FOLL_LONGTERM capability to GUP fast

public inbox for linuxppc-dev@ozlabs.org 
 help / color / mirror / Atom feed

From: ira.weiny@intel•com
To: linux-mips@vger•kernel.org, linux-kernel@vger•kernel.org,
	kvm-ppc@vger•kernel.org, linuxppc-dev@lists•ozlabs.org,
	linux-s390@vger•kernel.org, linux-sh@vger•kernel.org,
	sparclinux@vger•kernel.org, kvm@vger•kernel.org,
	linux-fpga@vger•kernel.org, dri-devel@lists•freedesktop.org,
	linux-rdma@vger•kernel.org, linux-media@vger•kernel.org,
	linux-scsi@vger•kernel.org, devel@driverdev•osuosl.org,
	virtualization@lists•linux-foundation.org,
	netdev@vger•kernel.org, linux-fbdev@vger•kernel.org,
	xen-devel@lists•xenproject.org, devel@lists•orangefs.org,
	linux-mm@kvack•org, ceph-devel@vger•kernel.org,
	rds-devel@oss•oracle.com
Cc: "Martin Brandenburg" <martin@omnibond•com>,
	"Rich Felker" <dalias@libc•org>,
	"David Hildenbrand" <david@redhat•com>,
	"David Airlie" <airlied@linux•ie>,
	"Jason Wang" <jasowang@redhat•com>,
	"Kai Mäkisara" <Kai.Makisara@kolumbus•fi>,
	"Michal Hocko" <mhocko@suse•com>,
	"Ira Weiny" <ira.weiny@intel•com>,
	"Ben Chan" <benchan@chromium•org>,
	"Rob Springer" <rspringer@google•com>,
	"Todd Poynor" <toddpoynor@google•com>,
	"Yoshinori Sato" <ysato@users•sourceforge.jp>,
	"Joerg Roedel" <joro@8bytes•org>,
	"Michael S. Tsirkin" <mst@redhat•com>,
	"Jason Gunthorpe" <jgg@ziepe•ca>,
	"Ingo Molnar" <mingo@redhat•com>, "Wu Hao" <hao.wu@intel•com>,
	"Alan Tull" <atull@kernel•org>,
	"John Hubbard" <jhubbard@nvidia•com>,
	"James E.J. Bottomley" <jejb@linux•ibm.com>,
	"Alex Williamson" <alex.williamson@redhat•com>,
	"Stefano Stabellini" <sstabellini@kernel•org>,
	"Borislav Petkov" <bp@alien8•de>,
	"Alexander Viro" <viro@zeniv•linux.org.uk>,
	"Thomas Gleixner" <tglx@linutronix•de>,
	"Mauro Carvalho Chehab" <mchehab@kernel•org>,
	"Peter Zijlstra" <peterz@infradead•org>,
	"Martin K. Petersen" <martin.petersen@oracle•com>,
	"Dennis Dalessandro" <dennis.dalessandro@intel•com>,
	"Cornelia Huck" <cohuck@redhat•com>,
	"Jens Wiklander" <jens.wiklander@linaro•org>,
	"Alexandre Bounine" <alex.bou9@gmail•com>,
	"Bartlomiej Zolnierkiewicz" <b.zolnierkie@samsung•com>,
	"Daniel Vetter" <daniel@ffwll•ch>,
	"Moritz Fischer" <mdf@kernel•org>,
	"Andrew Morton" <akpm@linux-foundation•org>,
	"David S. Miller" <davem@davemloft•net>,
	"Kirill A. Shutemov" <kirill.shutemov@linux•intel.com>,
	"Christian Benvenuti" <benve@cisco•com>
Subject: [PATCH V2 4/7] mm/gup: Add FOLL_LONGTERM capability to GUP fast
Date: Wed, 13 Feb 2019 15:04:52 -0800	[thread overview]
Message-ID: <20190213230455.5605-5-ira.weiny@intel.com> (raw)
In-Reply-To: <20190213230455.5605-1-ira.weiny@intel.com>

From: Ira Weiny <ira.weiny@intel•com>

DAX pages were previously unprotected from longterm pins when users
called get_user_pages_fast().

Use the new FOLL_LONGTERM flag to check for DEVMAP pages and fall
back to regular GUP processing if a DEVMAP page is encountered.

Signed-off-by: Ira Weiny <ira.weiny@intel•com>
---
 mm/gup.c | 24 +++++++++++++++++++++---
 1 file changed, 21 insertions(+), 3 deletions(-)

diff --git a/mm/gup.c b/mm/gup.c
index 6f32d36b3c5b..f7e759c523bb 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -1439,6 +1439,9 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end,
 			goto pte_unmap;
 
 		if (pte_devmap(pte)) {
+			if (unlikely(flags & FOLL_LONGTERM))
+				goto pte_unmap;
+
 			pgmap = get_dev_pagemap(pte_pfn(pte), pgmap);
 			if (unlikely(!pgmap)) {
 				undo_dev_pagemap(nr, nr_start, pages);
@@ -1578,8 +1581,11 @@ static int gup_huge_pmd(pmd_t orig, pmd_t *pmdp, unsigned long addr,
 	if (!pmd_access_permitted(orig, flags & FOLL_WRITE))
 		return 0;
 
-	if (pmd_devmap(orig))
+	if (pmd_devmap(orig)) {
+		if (unlikely(flags & FOLL_LONGTERM))
+			return 0;
 		return __gup_device_huge_pmd(orig, pmdp, addr, end, pages, nr);
+	}
 
 	refs = 0;
 	page = pmd_page(orig) + ((addr & ~PMD_MASK) >> PAGE_SHIFT);
@@ -1904,8 +1910,20 @@ int get_user_pages_fast(unsigned long start, int nr_pages,
 		start += nr << PAGE_SHIFT;
 		pages += nr;
 
-		ret = get_user_pages_unlocked(start, nr_pages - nr, pages,
-					      gup_flags);
+		if (gup_flags & FOLL_LONGTERM) {
+			down_read(&current->mm->mmap_sem);
+			ret = __gup_longterm_locked(current, current->mm,
+						    start, nr_pages - nr,
+						    pages, NULL, gup_flags);
+			up_read(&current->mm->mmap_sem);
+		} else {
+			/*
+			 * retain FAULT_FOLL_ALLOW_RETRY optimization if
+			 * possible
+			 */
+			ret = get_user_pages_unlocked(start, nr_pages - nr,
+						      pages, gup_flags);
+		}
 
 		/* Have to be a bit careful with return values */
 		if (nr > 0) {
-- 
2.20.1

next prev parent reply	other threads:[~2019-02-13 23:24 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <20190211201643.7599-1-ira.weiny@intel.com>
2019-02-13 23:04 ` [PATCH V2 0/7] Add FOLL_LONGTERM to GUP fast and use it ira.weiny
2019-02-13 23:04   ` [PATCH V2 1/7] mm/gup: Replace get_user_pages_longterm() with FOLL_LONGTERM ira.weiny
2019-02-13 23:04   ` [PATCH V2 2/7] mm/gup: Change write parameter to flags in fast walk ira.weiny
2019-02-13 23:04   ` [PATCH V2 3/7] mm/gup: Change GUP fast to use flags rather than a write 'bool' ira.weiny
2019-02-13 23:11     ` Jason Gunthorpe
2019-02-13 23:52       ` Ira Weiny
2019-02-13 23:04   ` ira.weiny [this message]
2019-02-13 23:04   ` [PATCH V2 5/7] IB/hfi1: Use the new FOLL_LONGTERM flag to get_user_pages_fast() ira.weiny
2019-02-13 23:04   ` [PATCH V2 6/7] IB/qib: " ira.weiny
2019-02-13 23:04   ` [PATCH V2 7/7] IB/mthca: " ira.weiny

find likely ancestor, descendant, or conflicting patches for this message:
( dfblob:6f32d36b3c5 dfblob:f7e759c523b )
 OR (
bs:"[PATCH V2 4/7] mm/gup: Add FOLL_LONGTERM capability to GUP fast" )
	(help)

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190213230455.5605-5-ira.weiny@intel.com \
    --to=ira.weiny@intel$(echo .)com \
    --cc=Kai.Makisara@kolumbus$(echo .)fi \
    --cc=airlied@linux$(echo .)ie \
    --cc=akpm@linux-foundation$(echo .)org \
    --cc=alex.bou9@gmail$(echo .)com \
    --cc=alex.williamson@redhat$(echo .)com \
    --cc=atull@kernel$(echo .)org \
    --cc=b.zolnierkie@samsung$(echo .)com \
    --cc=benchan@chromium$(echo .)org \
    --cc=benve@cisco$(echo .)com \
    --cc=bp@alien8$(echo .)de \
    --cc=ceph-devel@vger$(echo .)kernel.org \
    --cc=cohuck@redhat$(echo .)com \
    --cc=dalias@libc$(echo .)org \
    --cc=daniel@ffwll$(echo .)ch \
    --cc=davem@davemloft$(echo .)net \
    --cc=david@redhat$(echo .)com \
    --cc=dennis.dalessandro@intel$(echo .)com \
    --cc=devel@driverdev$(echo .)osuosl.org \
    --cc=devel@lists$(echo .)orangefs.org \
    --cc=dri-devel@lists$(echo .)freedesktop.org \
    --cc=hao.wu@intel$(echo .)com \
    --cc=jasowang@redhat$(echo .)com \
    --cc=jejb@linux$(echo .)ibm.com \
    --cc=jens.wiklander@linaro$(echo .)org \
    --cc=jgg@ziepe$(echo .)ca \
    --cc=jhubbard@nvidia$(echo .)com \
    --cc=joro@8bytes$(echo .)org \
    --cc=kirill.shutemov@linux$(echo .)intel.com \
    --cc=kvm-ppc@vger$(echo .)kernel.org \
    --cc=kvm@vger$(echo .)kernel.org \
    --cc=linux-fbdev@vger$(echo .)kernel.org \
    --cc=linux-fpga@vger$(echo .)kernel.org \
    --cc=linux-kernel@vger$(echo .)kernel.org \
    --cc=linux-media@vger$(echo .)kernel.org \
    --cc=linux-mips@vger$(echo .)kernel.org \
    --cc=linux-mm@kvack$(echo .)org \
    --cc=linux-rdma@vger$(echo .)kernel.org \
    --cc=linux-s390@vger$(echo .)kernel.org \
    --cc=linux-scsi@vger$(echo .)kernel.org \
    --cc=linux-sh@vger$(echo .)kernel.org \
    --cc=linuxppc-dev@lists$(echo .)ozlabs.org \
    --cc=martin.petersen@oracle$(echo .)com \
    --cc=martin@omnibond$(echo .)com \
    --cc=mchehab@kernel$(echo .)org \
    --cc=mdf@kernel$(echo .)org \
    --cc=mhocko@suse$(echo .)com \
    --cc=mingo@redhat$(echo .)com \
    --cc=mst@redhat$(echo .)com \
    --cc=netdev@vger$(echo .)kernel.org \
    --cc=peterz@infradead$(echo .)org \
    --cc=rds-devel@oss$(echo .)oracle.com \
    --cc=rspringer@google$(echo .)com \
    --cc=sparclinux@vger$(echo .)kernel.org \
    --cc=sstabellini@kernel$(echo .)org \
    --cc=tglx@linutronix$(echo .)de \
    --cc=toddpoynor@google$(echo .)com \
    --cc=viro@zeniv$(echo .)linux.org.uk \
    --cc=virtualization@lists$(echo .)linux-foundation.org \
    --cc=xen-devel@lists$(echo .)xenproject.org \
    --cc=ysato@users$(echo .)sourceforge.jp \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

Be sure your reply has a Subject: header at the top and a blank line before the message body.

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox