From: Sean Christopherson <seanjc@google•com>
To: Paolo Bonzini <pbonzini@redhat•com>,
Marc Zyngier <maz@kernel•org>,
Oliver Upton <oliver.upton@linux•dev>,
Sean Christopherson <seanjc@google•com>
Cc: kvm@vger•kernel.org, linux-arm-kernel@lists•infradead.org,
kvmarm@lists•linux.dev, linux-kernel@vger•kernel.org,
Ira Weiny <ira.weiny@intel•com>, Gavin Shan <gshan@redhat•com>,
Shivank Garg <shivankg@amd•com>,
Vlastimil Babka <vbabka@suse•cz>,
Xiaoyao Li <xiaoyao.li@intel•com>,
David Hildenbrand <david@redhat•com>,
Fuad Tabba <tabba@google•com>,
Ackerley Tng <ackerleytng@google•com>,
Tao Chan <chentao@kylinos•cn>,
James Houghton <jthoughton@google•com>
Subject: [PATCH v17 18/24] KVM: arm64: Handle guest_memfd-backed guest page faults
Date: Tue, 29 Jul 2025 15:54:49 -0700 [thread overview]
Message-ID: <20250729225455.670324-19-seanjc@google.com> (raw)
In-Reply-To: <20250729225455.670324-1-seanjc@google.com>
From: Fuad Tabba <tabba@google•com>
Add arm64 architecture support for handling guest page faults on memory
slots backed by guest_memfd.
This change introduces a new function, gmem_abort(), which encapsulates
the fault handling logic specific to guest_memfd-backed memory. The
kvm_handle_guest_abort() entry point is updated to dispatch to
gmem_abort() when a fault occurs on a guest_memfd-backed memory slot (as
determined by kvm_slot_has_gmem()).
Until guest_memfd gains support for huge pages, the fault granule for
these memory regions is restricted to PAGE_SIZE.
Reviewed-by: Gavin Shan <gshan@redhat•com>
Reviewed-by: James Houghton <jthoughton@google•com>
Reviewed-by: Marc Zyngier <maz@kernel•org>
Signed-off-by: Fuad Tabba <tabba@google•com>
Signed-off-by: Sean Christopherson <seanjc@google•com>
---
arch/arm64/kvm/mmu.c | 86 ++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 83 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index b3eacb400fab..8c82df80a835 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -1512,6 +1512,82 @@ static void adjust_nested_fault_perms(struct kvm_s2_trans *nested,
*prot |= kvm_encode_nested_level(nested);
}
+#define KVM_PGTABLE_WALK_MEMABORT_FLAGS (KVM_PGTABLE_WALK_HANDLE_FAULT | KVM_PGTABLE_WALK_SHARED)
+
+static int gmem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
+ struct kvm_s2_trans *nested,
+ struct kvm_memory_slot *memslot, bool is_perm)
+{
+ bool write_fault, exec_fault, writable;
+ enum kvm_pgtable_walk_flags flags = KVM_PGTABLE_WALK_MEMABORT_FLAGS;
+ enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R;
+ struct kvm_pgtable *pgt = vcpu->arch.hw_mmu->pgt;
+ unsigned long mmu_seq;
+ struct page *page;
+ struct kvm *kvm = vcpu->kvm;
+ void *memcache;
+ kvm_pfn_t pfn;
+ gfn_t gfn;
+ int ret;
+
+ ret = prepare_mmu_memcache(vcpu, true, &memcache);
+ if (ret)
+ return ret;
+
+ if (nested)
+ gfn = kvm_s2_trans_output(nested) >> PAGE_SHIFT;
+ else
+ gfn = fault_ipa >> PAGE_SHIFT;
+
+ write_fault = kvm_is_write_fault(vcpu);
+ exec_fault = kvm_vcpu_trap_is_exec_fault(vcpu);
+
+ VM_WARN_ON_ONCE(write_fault && exec_fault);
+
+ mmu_seq = kvm->mmu_invalidate_seq;
+ /* Pairs with the smp_wmb() in kvm_mmu_invalidate_end(). */
+ smp_rmb();
+
+ ret = kvm_gmem_get_pfn(kvm, memslot, gfn, &pfn, &page, NULL);
+ if (ret) {
+ kvm_prepare_memory_fault_exit(vcpu, fault_ipa, PAGE_SIZE,
+ write_fault, exec_fault, false);
+ return ret;
+ }
+
+ writable = !(memslot->flags & KVM_MEM_READONLY);
+
+ if (nested)
+ adjust_nested_fault_perms(nested, &prot, &writable);
+
+ if (writable)
+ prot |= KVM_PGTABLE_PROT_W;
+
+ if (exec_fault ||
+ (cpus_have_final_cap(ARM64_HAS_CACHE_DIC) &&
+ (!nested || kvm_s2_trans_executable(nested))))
+ prot |= KVM_PGTABLE_PROT_X;
+
+ kvm_fault_lock(kvm);
+ if (mmu_invalidate_retry(kvm, mmu_seq)) {
+ ret = -EAGAIN;
+ goto out_unlock;
+ }
+
+ ret = KVM_PGT_FN(kvm_pgtable_stage2_map)(pgt, fault_ipa, PAGE_SIZE,
+ __pfn_to_phys(pfn), prot,
+ memcache, flags);
+
+out_unlock:
+ kvm_release_faultin_page(kvm, page, !!ret, writable);
+ kvm_fault_unlock(kvm);
+
+ if (writable && !ret)
+ mark_page_dirty_in_slot(kvm, memslot, gfn);
+
+ return ret != -EAGAIN ? ret : 0;
+}
+
static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
struct kvm_s2_trans *nested,
struct kvm_memory_slot *memslot, unsigned long hva,
@@ -1536,7 +1612,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R;
struct kvm_pgtable *pgt;
struct page *page;
- enum kvm_pgtable_walk_flags flags = KVM_PGTABLE_WALK_HANDLE_FAULT | KVM_PGTABLE_WALK_SHARED;
+ enum kvm_pgtable_walk_flags flags = KVM_PGTABLE_WALK_MEMABORT_FLAGS;
if (fault_is_perm)
fault_granule = kvm_vcpu_trap_get_perm_fault_granule(vcpu);
@@ -1961,8 +2037,12 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
VM_WARN_ON_ONCE(kvm_vcpu_trap_is_permission_fault(vcpu) &&
!write_fault && !kvm_vcpu_trap_is_exec_fault(vcpu));
- ret = user_mem_abort(vcpu, fault_ipa, nested, memslot, hva,
- esr_fsc_is_permission_fault(esr));
+ if (kvm_slot_has_gmem(memslot))
+ ret = gmem_abort(vcpu, fault_ipa, nested, memslot,
+ esr_fsc_is_permission_fault(esr));
+ else
+ ret = user_mem_abort(vcpu, fault_ipa, nested, memslot, hva,
+ esr_fsc_is_permission_fault(esr));
if (ret == 0)
ret = 1;
out:
--
2.50.1.552.g942d659e1b-goog
next prev parent reply other threads:[~2025-07-29 23:45 UTC|newest]
Thread overview: 58+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-07-29 22:54 [PATCH v17 00/24] KVM: Enable mmap() for guest_memfd Sean Christopherson
2025-07-29 22:54 ` [PATCH v17 01/24] KVM: Rename CONFIG_KVM_PRIVATE_MEM to CONFIG_KVM_GUEST_MEMFD Sean Christopherson
2025-07-29 22:54 ` [PATCH v17 02/24] KVM: x86: Have all vendor neutral sub-configs depend on KVM_X86, not just KVM Sean Christopherson
2025-07-31 8:08 ` Fuad Tabba
2025-07-29 22:54 ` [PATCH v17 03/24] KVM: x86: Select KVM_GENERIC_PRIVATE_MEM directly from KVM_SW_PROTECTED_VM Sean Christopherson
2025-07-31 8:08 ` Fuad Tabba
2025-07-29 22:54 ` [PATCH v17 04/24] KVM: x86: Select TDX's KVM_GENERIC_xxx dependencies iff CONFIG_KVM_INTEL_TDX=y Sean Christopherson
2025-07-31 8:07 ` Fuad Tabba
2025-07-29 22:54 ` [PATCH v17 05/24] KVM: Rename CONFIG_KVM_GENERIC_PRIVATE_MEM to CONFIG_HAVE_KVM_ARCH_GMEM_POPULATE Sean Christopherson
2025-07-29 22:54 ` [PATCH v17 06/24] KVM: Rename kvm_slot_can_be_private() to kvm_slot_has_gmem() Sean Christopherson
2025-07-29 22:54 ` [PATCH v17 07/24] KVM: Fix comments that refer to slots_lock Sean Christopherson
2025-07-29 22:54 ` [PATCH v17 08/24] KVM: Fix comment that refers to kvm uapi header path Sean Christopherson
2025-07-29 22:54 ` [PATCH v17 09/24] KVM: x86: Enable KVM_GUEST_MEMFD for all 64-bit builds Sean Christopherson
2025-07-29 22:54 ` [PATCH v17 10/24] KVM: guest_memfd: Add plumbing to host to map guest_memfd pages Sean Christopherson
2025-07-29 22:54 ` [PATCH v17 11/24] KVM: guest_memfd: Track guest_memfd mmap support in memslot Sean Christopherson
2025-07-29 22:54 ` [PATCH v17 12/24] KVM: x86/mmu: Rename .private_max_mapping_level() to .gmem_max_mapping_level() Sean Christopherson
2025-07-31 8:15 ` Fuad Tabba
2025-07-31 8:29 ` David Hildenbrand
2025-07-31 8:33 ` Fuad Tabba
2025-07-29 22:54 ` [PATCH v17 13/24] KVM: x86/mmu: Hoist guest_memfd max level/order helpers "up" in mmu.c Sean Christopherson
2025-07-31 7:59 ` David Hildenbrand
2025-07-31 8:06 ` Fuad Tabba
2025-07-29 22:54 ` [PATCH v17 14/24] KVM: x86/mmu: Enforce guest_memfd's max order when recovering hugepages Sean Christopherson
2025-07-30 7:33 ` Xiaoyao Li
2025-07-31 8:06 ` David Hildenbrand
2025-07-31 8:10 ` Fuad Tabba
2025-07-29 22:54 ` [PATCH v17 15/24] KVM: x86/mmu: Extend guest_memfd's max mapping level to shared mappings Sean Christopherson
2025-07-30 7:36 ` Xiaoyao Li
2025-07-31 8:01 ` David Hildenbrand
2025-07-31 8:05 ` Fuad Tabba
2025-07-29 22:54 ` [PATCH v17 16/24] KVM: x86/mmu: Handle guest page faults for guest_memfd with shared memory Sean Christopherson
2025-07-30 7:37 ` Xiaoyao Li
2025-07-29 22:54 ` [PATCH v17 17/24] KVM: arm64: Refactor user_mem_abort() Sean Christopherson
2025-07-29 22:54 ` Sean Christopherson [this message]
2025-07-29 22:54 ` [PATCH v17 19/24] KVM: arm64: nv: Handle VNCR_EL2-triggered faults backed by guest_memfd Sean Christopherson
2025-07-29 22:54 ` [PATCH v17 20/24] KVM: arm64: Enable support for guest_memfd backed memory Sean Christopherson
2025-07-29 22:54 ` [PATCH v17 21/24] KVM: Allow and advertise support for host mmap() on guest_memfd files Sean Christopherson
2025-07-29 22:54 ` [PATCH v17 22/24] KVM: selftests: Do not use hardcoded page sizes in guest_memfd test Sean Christopherson
2025-07-30 11:04 ` Xiaoyao Li
2025-07-29 22:54 ` [PATCH v17 23/24] KVM: selftests: guest_memfd mmap() test when mmap is supported Sean Christopherson
2025-07-30 11:39 ` Xiaoyao Li
2025-07-30 12:57 ` Sean Christopherson
2025-07-31 7:49 ` Xiaoyao Li
2025-08-07 8:12 ` Shivank Garg
2025-07-29 22:54 ` [PATCH v17 24/24] KVM: selftests: Add guest_memfd testcase to fault-in on !mmap()'d memory Sean Christopherson
2025-07-30 8:20 ` Xiaoyao Li
2025-07-30 15:51 ` Fuad Tabba
2026-03-30 6:21 ` Zenghui Yu
2026-04-17 16:47 ` Sean Christopherson
2026-05-12 7:28 ` Zenghui Yu
2026-05-12 15:53 ` Sean Christopherson
2025-07-30 21:34 ` [PATCH v17 00/24] KVM: Enable mmap() for guest_memfd Ackerley Tng
2025-07-30 22:44 ` Ackerley Tng
2025-08-27 8:43 ` Paolo Bonzini
2025-08-27 12:57 ` Sean Christopherson
2025-08-27 13:08 ` Marc Zyngier
2025-08-27 13:11 ` Paolo Bonzini
2025-08-27 13:14 ` Marc Zyngier
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250729225455.670324-19-seanjc@google.com \
--to=seanjc@google$(echo .)com \
--cc=ackerleytng@google$(echo .)com \
--cc=chentao@kylinos$(echo .)cn \
--cc=david@redhat$(echo .)com \
--cc=gshan@redhat$(echo .)com \
--cc=ira.weiny@intel$(echo .)com \
--cc=jthoughton@google$(echo .)com \
--cc=kvm@vger$(echo .)kernel.org \
--cc=kvmarm@lists$(echo .)linux.dev \
--cc=linux-arm-kernel@lists$(echo .)infradead.org \
--cc=linux-kernel@vger$(echo .)kernel.org \
--cc=maz@kernel$(echo .)org \
--cc=oliver.upton@linux$(echo .)dev \
--cc=pbonzini@redhat$(echo .)com \
--cc=shivankg@amd$(echo .)com \
--cc=tabba@google$(echo .)com \
--cc=vbabka@suse$(echo .)cz \
--cc=xiaoyao.li@intel$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox