From: Sascha Bischoff <Sascha.Bischoff@arm•com>
To: "linux-arm-kernel@lists•infradead.org"
<linux-arm-kernel@lists•infradead.org>,
"kvmarm@lists•linux.dev" <kvmarm@lists•linux.dev>,
"kvm@vger•kernel.org" <kvm@vger•kernel.org>
Cc: nd <nd@arm•com>, "maz@kernel•org" <maz@kernel•org>,
"oliver.upton@linux•dev" <oliver.upton@linux•dev>,
Joey Gouly <Joey.Gouly@arm•com>,
Suzuki Poulose <Suzuki.Poulose@arm•com>,
"yuzenghui@huawei•com" <yuzenghui@huawei•com>,
"peter.maydell@linaro•org" <peter.maydell@linaro•org>,
"lpieralisi@kernel•org" <lpieralisi@kernel•org>,
Timothy Hayes <Timothy.Hayes@arm•com>
Subject: [PATCH v2 39/39] Documentation: KVM: Add the VGICv5 IRS save/restore sequences
Date: Thu, 21 May 2026 15:02:34 +0000 [thread overview]
Message-ID: <20260521144846.1899475-40-sascha.bischoff@arm.com> (raw)
In-Reply-To: <20260521144846.1899475-1-sascha.bischoff@arm.com>
When saving/restoring the state of the GICv5 IRS, it is important that
it happens in the correct order. Failure to do so will almost
certainly result in failing to restore a guest that is capable of
handling interrupts correctly.
On a save, the ISTs must be saved prior to saving the guest's memory
as the guest's LPI IST is written to guest memory. Conversely, on
restore the guest's memory must be restored prior to restoring the
ISTs.
It is important to restore the IRS MMIO registers by first restoring
the IRS_IDx registers as they define the capabilities of the IRS, and
are used as part of creating and managing ISTs and SPIs.
In order to restore the ISTs themselves, the IRS_IST_CFGR must be
restored prior to the IRS_IST_BASER. KVM uses these restored registers
when KVM_DEV_ARM_VGIC_GRP_IST is restored to determine whether a guest
LPI IST exists, how large it must be, and where the guest-provided
migration storage lives. The host LPI IST is allocated and populated
as part of restoring KVM_DEV_ARM_VGIC_GRP_IST.
At this stage the remaining MMIO registers can be restored. The SPI
IST gets extracted from a userspace provided buffer, and is
transferred to the host-allocated SPI IST. The LPI IST is extracted
from guest memory, and is written to the host-allocated LPI IST.
As a general rule, the IRS_*_STATUSR registers can be ignored on
restore. They are not userspace writable.
Signed-off-by: Sascha Bischoff <sascha.bischoff@arm•com>
---
.../virt/kvm/devices/arm-vgic-v5.rst | 45 +++++++++++++++++++
1 file changed, 45 insertions(+)
diff --git a/Documentation/virt/kvm/devices/arm-vgic-v5.rst b/Documentation/virt/kvm/devices/arm-vgic-v5.rst
index 0ee0fe9308fc9..188851f22f9eb 100644
--- a/Documentation/virt/kvm/devices/arm-vgic-v5.rst
+++ b/Documentation/virt/kvm/devices/arm-vgic-v5.rst
@@ -232,3 +232,48 @@ Groups:
or tracking pending interrupts
-ETIMEDOUT An IRS save/VM operation timed out
=========== ============================================================
+
+IRS Save Sequence:
+------------------
+
+The following operations are required when saving the virtual GICv5 IRS:
+
+a) Save the ISTs by issuing KVM_GET_DEVICE_ATTR on KVM_DEV_ARM_VGIC_GRP_IST.
+b) Save the IRS MMIO register state by issuing KVM_GET_DEVICE_ATTR on
+ KVM_DEV_ARM_VGIC_GRP_IRS_REGS.
+
+These two steps may be performed in either order. However, the guest memory
+must be serialised after the ISTs have been saved, as saving the LPI IST writes
+the IST state back into guest memory.
+
+IRS Restore Sequence:
+---------------------
+
+The following ordering must be followed when restoring the virtual GICv5 and
+IRS:
+
+a) Create vCPUs.
+b) Provide the IRS base address by issuing KVM_SET_DEVICE_ATTR on
+ KVM_DEV_ARM_VGIC_GRP_ADDR
+c) Restore the number of SPIs by issuing KVM_SET_DEVICE_ATTR on
+ KVM_DEV_ARM_VGIC_GRP_NR_IRQS.
+d) Initialise the GIC - this sets up the default state and creates the SPI
+ IST - by issuing KVM_SET_DEVICE_ATTR on KVM_DEV_ARM_VGIC_GRP_CTRL with
+ KVM_DEV_ARM_VGIC_CTRL_INIT
+e) Restore guest memory.
+f) Restore the IRS MMIO register state by issuing KVM_SET_DEVICE_ATTR on
+ KVM_DEV_ARM_VGIC_GRP_IRS_REGS. KVM uses the restored IRS_IST_CFGR and
+ IRS_IST_BASER state to allocate the LPI IST during the following step.
+g) Restore the ISTs by issuing KVM_SET_DEVICE_ATTR on
+ KVM_DEV_ARM_VGIC_GRP_IST.
+
+The number of SPIs must be restored before VGIC initialization because
+initialization allocates the SPI state and fixes the SPI range exposed by the
+IRS ID registers.
+
+The various ``*_STATUSR`` registers are observational state in the current KVM
+implementation. Userspace may save them for validation or debugging purposes,
+but they are not required as restore input and do not need to be replayed during
+restore.
+
+Then vCPUs can be started.
--
2.34.1
prev parent reply other threads:[~2026-05-21 15:03 UTC|newest]
Thread overview: 47+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-05-21 14:49 [PATCH v2 00/39] KVM: arm64: Add GICv5 IRS support Sascha Bischoff
2026-05-21 14:49 ` [PATCH v2 01/39] irqchip/gic-v5: Allow KVM setup without a maintenance IRQ Sascha Bischoff
2026-05-21 14:49 ` [PATCH v2 02/39] irqchip/gic-v5: Provide OF IRS config frame attrs to KVM Sascha Bischoff
2026-05-21 14:50 ` [PATCH v2 03/39] irqchip/gic-v5: Setup gic_kvm_info on ACPI hosts Sascha Bischoff
2026-05-27 10:51 ` Marc Zyngier
2026-05-29 14:33 ` Sascha Bischoff
2026-05-28 7:14 ` Lorenzo Pieralisi
2026-05-29 14:41 ` Sascha Bischoff
2026-05-21 14:50 ` [PATCH v2 04/39] KVM: arm64: gic-v5: Define remaining IRS MMIO registers Sascha Bischoff
2026-05-21 14:50 ` [PATCH v2 05/39] arm64/sysreg: Add GICv5 GIC VDPEND and VDRCFG encodings Sascha Bischoff
2026-05-21 14:51 ` [PATCH v2 06/39] arm64/sysreg: Update ICC_CR0_EL1 with LINK and LINK_IDLE fields Sascha Bischoff
2026-05-21 14:51 ` [PATCH v2 07/39] KVM: arm64: gic-v5: Extract host IRS caps from IRS config frame Sascha Bischoff
2026-05-21 14:51 ` [PATCH v2 08/39] KVM: arm64: gic-v5: Add VPE doorbell domain Sascha Bischoff
2026-05-21 14:52 ` [PATCH v2 09/39] KVM: arm64: gic-v5: Create & manage VM and VPE tables Sascha Bischoff
2026-05-21 14:52 ` [PATCH v2 10/39] KVM: arm64: gic-v5: Introduce guest IST alloc and management Sascha Bischoff
2026-05-21 14:52 ` [PATCH v2 11/39] KVM: arm64: gic-v5: Implement VMT/vIST IRS MMIO Ops Sascha Bischoff
2026-05-21 14:53 ` [PATCH v2 12/39] KVM: arm64: gic-v5: Keep GICv5 vCPU limit model-specific Sascha Bischoff
2026-05-21 14:53 ` [PATCH v2 13/39] KVM: arm64: gic-v5: Implement VPE IRS MMIO Ops Sascha Bischoff
2026-05-21 14:53 ` [PATCH v2 14/39] KVM: arm64: gic-v5: Set up VMTEs and VPE doorbells Sascha Bischoff
2026-05-21 14:54 ` [PATCH v2 15/39] KVM: arm64: gic-v5: Add resident/non-resident hyp calls Sascha Bischoff
2026-05-21 14:54 ` [PATCH v2 16/39] KVM: arm64: gic-v5: Request doorbells when VPEs enter WFI Sascha Bischoff
2026-05-21 14:54 ` [PATCH v2 17/39] KVM: arm64: gic-v5: Introduce struct vgic_v5_irs and IRS base address Sascha Bischoff
2026-05-21 14:55 ` [PATCH v2 18/39] KVM: arm64: gic-v5: Add IRS IODEV support to MMIO handlers Sascha Bischoff
2026-05-21 14:55 ` [PATCH v2 19/39] KVM: arm64: gic-v5: Add KVM_VGIC_V5_ADDR_TYPE_IRS to UAPI Sascha Bischoff
2026-05-21 14:55 ` [PATCH v2 20/39] KVM: arm64: gic-v5: Add GICv5 IRS IODEV and MMIO emulation Sascha Bischoff
2026-05-21 14:56 ` [PATCH v2 21/39] KVM: arm64: gic-v5: Initialise per-VM IRS state Sascha Bischoff
2026-05-21 14:56 ` [PATCH v2 22/39] KVM: arm64: gic-v5: Register the IRS IODEV Sascha Bischoff
2026-05-21 14:57 ` [PATCH v2 23/39] KVM: arm64: gic-v5: Set IRICHPPIDIS based on IRS enable state Sascha Bischoff
2026-05-21 14:57 ` [PATCH v2 24/39] KVM: arm64: selftests: Update vGICv5 selftest to set IRS address Sascha Bischoff
2026-05-21 14:57 ` [PATCH v2 25/39] KVM: arm64: gic-v5: Introduce SPI AP list Sascha Bischoff
2026-05-21 14:58 ` [PATCH v2 26/39] KVM: arm64: gic-v5: Add GIC VDPEND and GIC VDRCFG hyp calls Sascha Bischoff
2026-05-21 14:58 ` [PATCH v2 27/39] KVM: arm64: gic-v5: Track SPI state for in-flight SPIs Sascha Bischoff
2026-05-21 14:58 ` [PATCH v2 28/39] KVM: arm64: gic: Introduce set_pending_state() to irq_op Sascha Bischoff
2026-05-21 14:59 ` [PATCH v2 29/39] KVM: arm64: gic-v5: Support SPI injection Sascha Bischoff
2026-05-26 13:41 ` Vladimir Murzin
2026-05-28 14:59 ` Sascha Bischoff
2026-05-21 14:59 ` [PATCH v2 30/39] Documentation: KVM: Extend VGICv5 docs for KVM_VGIC_V5_ADDR_TYPE_IRS Sascha Bischoff
2026-05-21 14:59 ` [PATCH v2 31/39] KVM: arm64: gic-v5: Add GICv5 SPI injection to irqfd Sascha Bischoff
2026-06-04 10:51 ` Vladimir Murzin
2026-05-21 15:00 ` [PATCH v2 32/39] KVM: arm64: gic-v5: Mask per-vcpu PPI state in vgic_v5_finalize_ppi_state() Sascha Bischoff
2026-05-21 15:00 ` [PATCH v2 33/39] KVM: arm64: gic-v5: Add GICv5 EL1 sysreg userspace accessors Sascha Bischoff
2026-05-21 15:00 ` [PATCH v2 34/39] KVM: arm64: gic-v5: Handle userspace accesses to IRS MMIO region Sascha Bischoff
2026-05-21 15:01 ` [PATCH v2 35/39] KVM: arm64: gic-v5: Implement save/restore mechanisms for ISTs Sascha Bischoff
2026-05-21 15:01 ` [PATCH v2 36/39] Documentation: KVM: Document KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS for VGICv5 Sascha Bischoff
2026-05-21 15:01 ` [PATCH v2 37/39] Documentation: KVM: Add KVM_DEV_ARM_VGIC_GRP_IRS_REGS to VGICv5 docs Sascha Bischoff
2026-05-21 15:02 ` [PATCH v2 38/39] Documentation: KVM: Add docs for KVM_DEV_ARM_VGIC_GRP_IST Sascha Bischoff
2026-05-21 15:02 ` Sascha Bischoff [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260521144846.1899475-40-sascha.bischoff@arm.com \
--to=sascha.bischoff@arm$(echo .)com \
--cc=Joey.Gouly@arm$(echo .)com \
--cc=Suzuki.Poulose@arm$(echo .)com \
--cc=Timothy.Hayes@arm$(echo .)com \
--cc=kvm@vger$(echo .)kernel.org \
--cc=kvmarm@lists$(echo .)linux.dev \
--cc=linux-arm-kernel@lists$(echo .)infradead.org \
--cc=lpieralisi@kernel$(echo .)org \
--cc=maz@kernel$(echo .)org \
--cc=nd@arm$(echo .)com \
--cc=oliver.upton@linux$(echo .)dev \
--cc=peter.maydell@linaro$(echo .)org \
--cc=yuzenghui@huawei$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox