From: Suzuki K Poulose <suzuki.poulose@arm•com>
To: Marc Zyngier <maz@kernel•org>, Steven Price <steven.price@arm•com>
Cc: kvm@vger•kernel.org, kvmarm@lists•linux.dev,
Catalin Marinas <catalin.marinas@arm•com>,
Will Deacon <will@kernel•org>, James Morse <james.morse@arm•com>,
Oliver Upton <oliver.upton@linux•dev>,
Zenghui Yu <yuzenghui@huawei•com>,
linux-arm-kernel@lists•infradead.org,
linux-kernel@vger•kernel.org, Joey Gouly <joey.gouly@arm•com>,
Alexandru Elisei <alexandru.elisei@arm•com>,
Christoffer Dall <christoffer.dall@arm•com>,
Fuad Tabba <tabba@google•com>,
linux-coco@lists•linux.dev,
Ganapatrao Kulkarni <gankulkarni@os•amperecomputing.com>,
Gavin Shan <gshan@redhat•com>,
Shanker Donthineni <sdonthineni@nvidia•com>,
Alper Gun <alpergun@google•com>,
"Aneesh Kumar K . V" <aneesh.kumar@kernel•org>,
Emi Kisanuki <fj0570is@fujitsu•com>,
Vishal Annapurve <vannapurve@google•com>,
WeiLin.Chang@arm•com, Lorenzo.Pieralisi2@arm•com
Subject: Re: [PATCH v14 08/44] arm64: RMI: Ensure that the RMM has GPT entries for memory
Date: Thu, 21 May 2026 16:39:27 +0100 [thread overview]
Message-ID: <e175cb7b-b4fa-4139-b46d-1986e2372d16@arm.com> (raw)
In-Reply-To: <868q9cx4ac.wl-maz@kernel.org>
On 21/05/2026 14:47, Marc Zyngier wrote:
> On Wed, 13 May 2026 14:17:16 +0100,
> Steven Price <steven.price@arm•com> wrote:
>>
>> The RMM maintains the state of all the granules in the system to make
>> sure that the host is abiding by the rules. This state can be maintained
>> at different granularity, per page (TRACKING_FINE) or per region
>> (TRACKING_COARSE). The region size depends on the underlying
>> "RMI_GRANULE_SIZE". For a "coarse" region all pages in the region must
>> be of the same state, this implies we need to have "fine" tracking for
>> DRAM, so that we can delegated individual pages.
>>
>> For now we only support a statically carved out memory for tracking
>> granules for the "fine" regions. This can be extended in the future to
>> allow modifying the tracking granularity and remove the need for a
>> static allocation.
>>
>> Similarly, the firmware may create L0 GPT entries describing the total
>> address space. But if we change the "PAS" (Physical Address Space) of a
>> granule then the firmware may need to create L1 tables to track the PAS
>> at a finer granularity.
>>
>> Note: support is currently missing for SROs which means that if the RMM
>> needs memory donating this will fail (and render CCA unusable in Linux).
>> This effectively means that the L1 GPT tables must be created before
>> Linux starts.
>>
>> Signed-off-by: Steven Price <steven.price@arm•com>
>> ---
>> Changes since v13:
>> * Moved out of KVM
>> ---
>> arch/arm64/include/asm/rmi_cmds.h | 2 +
>> arch/arm64/kernel/rmi.c | 103 ++++++++++++++++++++++++++++++
>> 2 files changed, 105 insertions(+)
>>
>> diff --git a/arch/arm64/include/asm/rmi_cmds.h b/arch/arm64/include/asm/rmi_cmds.h
>> index 9179934925c5..9078a2920a7c 100644
>> --- a/arch/arm64/include/asm/rmi_cmds.h
>> +++ b/arch/arm64/include/asm/rmi_cmds.h
>> @@ -33,6 +33,8 @@ struct rmi_sro_state {
>> } while (RMI_RETURN_STATUS(res.a0) == RMI_BUSY || \
>> RMI_RETURN_STATUS(res.a0) == RMI_BLOCKED)
>>
>> +bool rmi_is_available(void);
>> +
>> unsigned long rmi_sro_execute(struct rmi_sro_state *sro, gfp_t gfp);
>> void rmi_sro_free(struct rmi_sro_state *sro);
>>
>> diff --git a/arch/arm64/kernel/rmi.c b/arch/arm64/kernel/rmi.c
>> index a14ead5dedda..52a415e99500 100644
>> --- a/arch/arm64/kernel/rmi.c
>> +++ b/arch/arm64/kernel/rmi.c
>> @@ -7,6 +7,8 @@
>>
>> #include <asm/rmi_cmds.h>
>>
>> +static bool arm64_rmi_is_available;
>> +
>> unsigned long rmm_feat_reg0;
>> unsigned long rmm_feat_reg1;
>>
>> @@ -88,6 +90,102 @@ static int rmi_configure(void)
>> return 0;
>> }
>>
>> +/*
>> + * For now we set the tracking_region_size to 0 for RMI_RMM_CONFIG_SET().
>> + * TODO: Support other tracking sizes (via Kconfig option).
>> + */
>> +#ifdef CONFIG_PAGE_SIZE_4KB
>> +#define RMM_GRANULE_TRACKING_SIZE SZ_1G
>> +#elif defined(CONFIG_PAGE_SIZE_16KB)
>> +#define RMM_GRANULE_TRACKING_SIZE SZ_32M
>> +#elif defined(CONFIG_PAGE_SIZE_64KB)
>> +#define RMM_GRANULE_TRACKING_SIZE SZ_512M
>> +#endif
>
> Basically, a level 2 mapping. Which means this whole block really is:
>
> #define RMM_GRANULE_TRAKING_SIZE (2 * PAGE_SHIFT - 3)
>
> (adjust for D128 as needed).
True,
>
>> +
>> +/*
>> + * Make sure the area is tracked by RMM at FINE granularity.
>> + * We do not support changing the tracking yet.
>> + */
>> +static int rmi_verify_memory_tracking(phys_addr_t start, phys_addr_t end)
>> +{
>> + while (start < end) {
>> + unsigned long ret, category, state, next;
>> +
>> + ret = rmi_granule_tracking_get(start, end, &category, &state, &next);
>> + if (ret != RMI_SUCCESS ||
>> + state != RMI_TRACKING_FINE ||
>> + category != RMI_MEM_CATEGORY_CONVENTIONAL) {
>> + /* TODO: Set granule tracking in this case */
>> + pr_err("Granule tracking for region isn't fine/conventional: %llx",
>> + start);
>> + return -ENODEV;
>
> How is this triggered? Do we really need to spam the console with
> this? A PA doesn't mean much, and there is no context (stack trace).
This could be triggered if the RMM doesn't have static carveout
for tracking the DRAM granules. (state != RMI_TRACKING_FINE).
This not worth WARN_ONCE(), we could simply not enable KVM.
We plan to add support for donating memory to the RMM in
the future. (Primarily we don't yet have an RMM implementation
that does dynamic management via SRO. This can be added later
as a separate series)
>
> If that's not expected, turn this into a WARN_ONCE().
>
>> + }
>> + start = next;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static unsigned long rmi_l0gpt_size(void)
>> +{
>> + return 1UL << (30 + FIELD_GET(RMI_FEATURE_REGISTER_1_L0GPTSZ,
>> + rmm_feat_reg1));
>> +}
>> +
>> +static int rmi_create_gpts(phys_addr_t start, phys_addr_t end)
>> +{
>> + unsigned long l0gpt_sz = rmi_l0gpt_size();
>> +
>> + start = ALIGN_DOWN(start, l0gpt_sz);
>> + end = ALIGN(end, l0gpt_sz);
>> +
>> + while (start < end) {
>> + int ret = rmi_gpt_l1_create(start);
>> +
>> + /*
>> + * Make sure the L1 GPT tables are created for the region.
>> + * RMI_ERROR_GPT indicates the L1 table already exists.
>> + */
>> + if (ret && ret != RMI_ERROR_GPT) {
>> + /*
>> + * FIXME: Handle SRO so that memory can be donated for
>> + * the tables.
>> + */
>> + pr_err("GPT Level1 table missing for %llx\n", start);
>> + return -ENOMEM;
>
> If any of this fails, where is the cleanup done? Is that part of the
> missing SRO support that's indicated in the commit message?
>
For now, there is no cleanup required. What we essentially do here is
making sure that the GPT tables have been created upto L1 (i.e.,
by checking ret == RMI_ERROR_GPT).
We do not donate any memory now, but only support RMMs with static
memory carved out for L1 GPT. Support for dynamic RMMs could be added as
a separate series, at which point, we could defer the table creation to
the actual use case (e.g, RMI_GRANULE_DELEGATE).
Clean up would be required when we donate memory to the RMM.
>> + }
>> + start += l0gpt_sz;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static int rmi_init_metadata(void)
>> +{
>> + phys_addr_t start, end;
>> + const struct memblock_region *r;
>> +
>> + for_each_mem_region(r) {
>> + int ret;
>> +
>> + start = memblock_region_memory_base_pfn(r) << PAGE_SHIFT;
>> + end = memblock_region_memory_end_pfn(r) << PAGE_SHIFT;
>> + ret = rmi_verify_memory_tracking(start, end);
>> + if (ret)
>> + return ret;
>> + ret = rmi_create_gpts(start, end);
>> + if (ret)
>> + return ret;
>> + }
>
> How does this work with, say, memory hotplug?
Good point, we need a hook for hotpug to make sure this is taken care
of. As mentioned above, when we add support for RMM with support for
dynamic Tracking/GPT with SRO, this could be deferred to the actual
use (handling RMI return codes, RMI_ERROR_TRACKING/RMI_ERROR_GPT)
Suzuki
>
>> +
>> + return 0;
>> +}
>> +
>> +bool rmi_is_available(void)
>> +{
>> + return arm64_rmi_is_available;
>> +}
>> +
>> static int __init arm64_init_rmi(void)
>> {
>> /* Continue without realm support if we can't agree on a version */
>> @@ -101,6 +199,11 @@ static int __init arm64_init_rmi(void)
>>
>> if (rmi_configure())
>> return 0;
>> + if (rmi_init_metadata())
>> + return 0;
>> +
>> + arm64_rmi_is_available = true;
>> + pr_info("RMI configured");
>>
>> return 0;
>> }
>
> Thanks,
>
> M.
>
next prev parent reply other threads:[~2026-05-21 15:39 UTC|newest]
Thread overview: 123+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-05-13 13:17 [PATCH v14 00/44] arm64: Support for Arm CCA in KVM Steven Price
2026-05-13 13:17 ` [PATCH v14 01/44] kvm: arm64: Include kvm_emulate.h in kvm/arm_psci.h Steven Price
2026-05-21 10:19 ` Marc Zyngier
2026-05-21 15:11 ` Steven Price
2026-05-13 13:17 ` [PATCH v14 02/44] kvm: arm64: Avoid including linux/kvm_host.h in kvm_pgtable.h Steven Price
2026-05-21 10:26 ` Marc Zyngier
2026-05-21 15:11 ` Steven Price
2026-05-13 13:17 ` [PATCH v14 03/44] arm64: RME: Handle Granule Protection Faults (GPFs) Steven Price
2026-05-21 12:25 ` Marc Zyngier
2026-05-21 15:15 ` Steven Price
2026-05-13 13:17 ` [PATCH v14 04/44] arm64: RMI: Add SMC definitions for calling the RMM Steven Price
2026-05-18 7:08 ` Gavin Shan
2026-05-20 16:01 ` Steven Price
2026-05-21 12:40 ` Marc Zyngier
2026-05-21 14:50 ` Suzuki K Poulose
2026-05-21 15:33 ` Steven Price
2026-05-22 9:58 ` Marc Zyngier
2026-06-03 10:15 ` Steven Price
2026-05-13 13:17 ` [PATCH v14 05/44] arm64: RMI: Add wrappers for RMI calls Steven Price
2026-05-19 5:35 ` Aneesh Kumar K.V
2026-05-21 15:44 ` Steven Price
2026-05-21 0:21 ` Gavin Shan
2026-05-21 15:44 ` Steven Price
2026-05-21 12:49 ` Marc Zyngier
2026-05-21 15:44 ` Steven Price
2026-05-13 13:17 ` [PATCH v14 06/44] arm64: RMI: Check for RMI support at init Steven Price
2026-05-21 0:39 ` Gavin Shan
2026-05-21 15:49 ` Steven Price
2026-05-25 6:58 ` Gavin Shan
2026-06-03 10:57 ` Steven Price
2026-05-21 13:02 ` Marc Zyngier
2026-06-03 10:57 ` Steven Price
2026-05-13 13:17 ` [PATCH v14 07/44] arm64: RMI: Configure the RMM with the host's page size Steven Price
2026-05-21 0:51 ` Gavin Shan
2026-05-21 22:36 ` Suzuki K Poulose
2026-05-21 13:30 ` Marc Zyngier
2026-05-21 14:53 ` Suzuki K Poulose
2026-06-03 15:48 ` Steven Price
2026-05-13 13:17 ` [PATCH v14 08/44] arm64: RMI: Ensure that the RMM has GPT entries for memory Steven Price
2026-05-19 5:55 ` Aneesh Kumar K.V
2026-06-03 15:48 ` Steven Price
2026-05-21 0:58 ` Gavin Shan
2026-06-03 15:48 ` Steven Price
2026-05-21 13:47 ` Marc Zyngier
2026-05-21 14:24 ` Marc Zyngier
2026-05-21 15:39 ` Suzuki K Poulose [this message]
2026-06-03 15:48 ` Steven Price
2026-05-13 13:17 ` [PATCH v14 09/44] arm64: RMI: Provide functions to delegate/undelegate ranges of memory Steven Price
2026-05-21 13:59 ` Marc Zyngier
2026-05-21 16:01 ` Suzuki K Poulose
2026-05-22 10:02 ` Marc Zyngier
2026-06-04 14:43 ` Steven Price
2026-05-13 13:17 ` [PATCH v14 10/44] arm64: RMI: Add support for SRO Steven Price
2026-05-14 8:01 ` Aneesh Kumar K.V
2026-05-14 9:33 ` Steven Price
2026-05-19 6:02 ` Aneesh Kumar K.V
2026-06-04 15:19 ` Steven Price
2026-05-21 4:38 ` Gavin Shan
2026-06-04 15:19 ` Steven Price
2026-05-21 14:35 ` Marc Zyngier
2026-06-04 15:19 ` Steven Price
2026-05-13 13:17 ` [PATCH v14 11/44] arm64: RMI: Check for RMI support at KVM init Steven Price
2026-05-13 13:17 ` [PATCH v14 12/44] arm64: RMI: Check for LPA2 support Steven Price
2026-05-13 13:17 ` [PATCH v14 13/44] arm64: RMI: Define the user ABI Steven Price
2026-05-26 22:17 ` Wei-Lin Chang
2026-06-04 15:27 ` Steven Price
2026-05-27 15:21 ` Marc Zyngier
2026-06-02 11:15 ` Suzuki K Poulose
2026-06-04 15:27 ` Steven Price
2026-05-13 13:17 ` [PATCH v14 14/44] arm64: RMI: Basic infrastructure for creating a realm Steven Price
2026-05-19 6:31 ` Aneesh Kumar K.V
2026-05-28 7:10 ` Marc Zyngier
2026-06-02 14:49 ` Suzuki K Poulose
2026-06-04 15:55 ` Steven Price
2026-05-13 13:17 ` [PATCH v14 15/44] kvm: arm64: Don't expose unsupported capabilities for realm guests Steven Price
2026-05-13 13:17 ` [PATCH v14 16/44] KVM: arm64: Allow passing machine type in KVM creation Steven Price
2026-05-13 13:17 ` [PATCH v14 17/44] arm64: RMI: RTT tear down Steven Price
2026-05-19 6:54 ` Aneesh Kumar K.V
2026-05-26 22:27 ` Wei-Lin Chang
2026-05-26 22:32 ` Wei-Lin Chang
2026-05-13 13:17 ` [PATCH v14 18/44] arm64: RMI: Activate realm on first VCPU run Steven Price
2026-05-13 13:17 ` [PATCH v14 19/44] arm64: RMI: Allocate/free RECs to match vCPUs Steven Price
2026-05-26 22:39 ` Wei-Lin Chang
2026-05-13 13:17 ` [PATCH v14 20/44] arm64: RMI: Support for the VGIC in realms Steven Price
2026-05-28 4:07 ` Gavin Shan
2026-05-13 13:17 ` [PATCH v14 21/44] KVM: arm64: Support timers in realm RECs Steven Price
2026-05-28 4:11 ` Gavin Shan
2026-05-13 13:17 ` [PATCH v14 22/44] arm64: RMI: Handle realm enter/exit Steven Price
2026-05-28 4:38 ` Gavin Shan
2026-05-13 13:17 ` [PATCH v14 23/44] arm64: RMI: Handle RMI_EXIT_RIPAS_CHANGE Steven Price
2026-05-19 9:40 ` Aneesh Kumar K.V
2026-05-27 10:52 ` Wei-Lin Chang
2026-05-13 13:17 ` [PATCH v14 24/44] KVM: arm64: Handle realm MMIO emulation Steven Price
2026-05-28 5:03 ` Gavin Shan
2026-05-13 13:17 ` [PATCH v14 25/44] KVM: arm64: Expose support for private memory Steven Price
2026-05-13 13:17 ` [PATCH v14 26/44] arm64: RMI: Allow populating initial contents Steven Price
2026-05-28 5:30 ` Gavin Shan
2026-05-13 13:17 ` [PATCH v14 27/44] arm64: RMI: Set RIPAS of initial memslots Steven Price
2026-05-19 10:02 ` Aneesh Kumar K.V
2026-05-19 10:13 ` Suzuki K Poulose
2026-05-19 12:55 ` Aneesh Kumar K.V
2026-05-19 13:06 ` Suzuki K Poulose
2026-05-13 13:17 ` [PATCH v14 28/44] arm64: RMI: Create the realm descriptor Steven Price
2026-05-26 22:47 ` Wei-Lin Chang
2026-05-28 5:51 ` Gavin Shan
2026-05-13 13:17 ` [PATCH v14 29/44] arm64: RMI: Runtime faulting of memory Steven Price
2026-05-13 13:17 ` [PATCH v14 30/44] KVM: arm64: Handle realm VCPU load Steven Price
2026-05-13 13:17 ` [PATCH v14 31/44] KVM: arm64: Validate register access for a Realm VM Steven Price
2026-05-13 13:17 ` [PATCH v14 32/44] KVM: arm64: Handle Realm PSCI requests Steven Price
2026-05-28 6:55 ` Gavin Shan
2026-05-13 13:17 ` [PATCH v14 33/44] KVM: arm64: WARN on injected undef exceptions Steven Price
2026-05-13 13:17 ` [PATCH v14 34/44] arm64: RMI: allow userspace to inject aborts Steven Price
2026-05-13 13:17 ` [PATCH v14 35/44] arm64: RMI: support RSI_HOST_CALL Steven Price
2026-05-13 13:17 ` [PATCH v14 36/44] arm64: RMI: Allow checking SVE on VM instance Steven Price
2026-05-13 13:17 ` [PATCH v14 37/44] arm64: RMI: Prevent Device mappings for Realms Steven Price
2026-05-19 10:25 ` Aneesh Kumar K.V
2026-05-13 13:17 ` [PATCH v14 38/44] arm64: RMI: Propagate number of breakpoints and watchpoints to userspace Steven Price
2026-05-13 13:17 ` [PATCH v14 39/44] arm64: RMI: Set breakpoint parameters through SET_ONE_REG Steven Price
2026-05-13 13:17 ` [PATCH v14 40/44] arm64: RMI: Propagate max SVE vector length from RMM Steven Price
2026-05-13 13:17 ` [PATCH v14 41/44] arm64: RMI: Configure max SVE vector length for a Realm Steven Price
2026-05-13 13:17 ` [PATCH v14 42/44] arm64: RMI: Provide register list for unfinalized RMI RECs Steven Price
2026-05-13 13:17 ` [PATCH v14 43/44] arm64: RMI: Provide accurate register list Steven Price
2026-05-13 13:17 ` [PATCH v14 44/44] arm64: RMI: Enable realms to be created Steven Price
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e175cb7b-b4fa-4139-b46d-1986e2372d16@arm.com \
--to=suzuki.poulose@arm$(echo .)com \
--cc=Lorenzo.Pieralisi2@arm$(echo .)com \
--cc=WeiLin.Chang@arm$(echo .)com \
--cc=alexandru.elisei@arm$(echo .)com \
--cc=alpergun@google$(echo .)com \
--cc=aneesh.kumar@kernel$(echo .)org \
--cc=catalin.marinas@arm$(echo .)com \
--cc=christoffer.dall@arm$(echo .)com \
--cc=fj0570is@fujitsu$(echo .)com \
--cc=gankulkarni@os$(echo .)amperecomputing.com \
--cc=gshan@redhat$(echo .)com \
--cc=james.morse@arm$(echo .)com \
--cc=joey.gouly@arm$(echo .)com \
--cc=kvm@vger$(echo .)kernel.org \
--cc=kvmarm@lists$(echo .)linux.dev \
--cc=linux-arm-kernel@lists$(echo .)infradead.org \
--cc=linux-coco@lists$(echo .)linux.dev \
--cc=linux-kernel@vger$(echo .)kernel.org \
--cc=maz@kernel$(echo .)org \
--cc=oliver.upton@linux$(echo .)dev \
--cc=sdonthineni@nvidia$(echo .)com \
--cc=steven.price@arm$(echo .)com \
--cc=tabba@google$(echo .)com \
--cc=vannapurve@google$(echo .)com \
--cc=will@kernel$(echo .)org \
--cc=yuzenghui@huawei$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox