From: andre.przywara@arm•com (Andre Przywara)
To: linux-arm-kernel@lists•infradead.org
Subject: [PATCH v4 15/19] arm/arm64: KVM: add virtual GICv3 distributor emulation
Date: Tue, 02 Dec 2014 17:32:45 +0000 [thread overview]
Message-ID: <547DF7BD.1000503@arm.com> (raw)
In-Reply-To: <547DF181.8060402@arm.com>
On 02/12/14 17:06, Marc Zyngier wrote:
> On 02/12/14 16:24, Andre Przywara wrote:
>> Hej Christoffer,
>>
>> On 30/11/14 08:30, Christoffer Dall wrote:
>>> On Fri, Nov 28, 2014 at 03:24:11PM +0000, Andre Przywara wrote:
>>>> Hej Christoffer,
>>>>
>>>> On 25/11/14 10:41, Christoffer Dall wrote:
>>>>> Hi Andre,
>>>>>
>>>>> On Mon, Nov 24, 2014 at 04:00:46PM +0000, Andre Przywara wrote:
>>>>>
>>>>
>>
>> [...]
>>
>>>>>>>> +
>>>>>>>> + if (!is_in_range(mmio->phys_addr, mmio->len, rdbase,
>>>>>>>> + GIC_V3_REDIST_SIZE * nrcpus))
>>>>>>>> + return false;
>>>>>>>
>>>>>>> Did you think more about the contiguous allocation issue here or can you
>>>>>>> give me a pointer to the requirement in the spec?
>>>>>>
>>>>>> 5.4.1 Re-Distributor Addressing
>>>>>>
>>>>>
>>>>> Section 5.4.1 talks about the pages within a single re-distributor having
>>>>> to be contiguous, not all the re-deistributor regions having to be
>>>>> contiguous, right?
>>>>
>>>> Ah yes, you are right. But I still think it does not matter:
>>>> 1) We are "implementing" the GICv3. So as the spec does not forbid this,
>>>> we just state that the redistributor register maps for each VCPU are
>>>> contiguous. Also we create the FDT accordingly. I will add a comment in
>>>> the documentation to state this.
>>>>
>>>> 2) The kernel's GICv3 DT bindings assume this allocation is the default.
>>>> Although Marc added bindings to work around this (stride), it seems much
>>>> more logical to me to not use it.
>>>
>>> I don't disagree (and never have) with the fact that it is up to us to
>>> decide.
>>>
>>> My original question, which we haven't talked about yet, is if it is
>>> *reasonable* to assume that all re-distributor regions will always be
>>> contiguous?
>>>
>>> How will you handle VCPU hotplug for example?
>>
>> As kvmtool does not support hotplug, I haven't thought about this yet.
>> To me it looks like userland should just use maxcpus for the allocation.
>> If I get the current QEMU code right, there is room for 127 GICv3 VCPUs
>> (2*64K per VCPU + 64K for the distributor in 16M space) at the moment.
>> Kvmtool uses a different mapping, which allows to share 1G with virtio,
>> so the limit is around 8000ish VCPUs here.
>> Are there any issues with changing the QEMU virt mapping later?
>> Migration, maybe?
>> If the UART, the RTC and the virtio regions are moved more towards the
>> beginning of the 256MB PCI mapping, then there should be space for a bit
>> less than 1024 VCPUs, if I get this right.
>>
>>> Where in the guest
>>> physical memory map of our various virt machines should these regions
>>> sit so that we can allocate anough re-distributors for VCPUs etc.?
>>
>> Various? Are there other mappings than those described in hw/arm/virt.c?
>>
>>> I just want to make sure we're not limiting ourselves by some amount of
>>> functionality or ABI (redistributor base addresses) that will be hard to
>>> expand in the future.
>>
>> If we are flexible with the mapping at VM creation time, QEMU could just
>> use a mapping depending on max_cpus:
>> < 128 VCPUs: use the current mapping
>> 128 <= x < 1020: use a more compressed mapping
>>> = 1020: map the redistributor somewhere above 4 GB
>>
>> As the device tree binding for GICv3 just supports a stride value, we
>> don't have any other real options beside this, right? So how I see this,
>> a contiguous mapping (with possible holes) is the only way.
>
> Not really. The GICv3 binding definitely supports having several regions
> for the redistributors (see the binding documentation). This allows for
> the pathological case where you have N regions for N CPUs. Not that we
> ever want to go there, really.
Ah yes, thanks for pointing that out. I was mixing this up with the
stride parameter, which is independent of this. Sorry for that.
So from a userland point of view we probably would like to have the
first n VCPU's redistributors mapped at their current places and allow
for more VCPUs to use memory above 4 GB.
Which would require quite some changes to the code to support this in a
very flexible way. I think this could be much easier if we confine
ourselves to two regions (one contiguous lower (< 4 GB) and one
contiguous upper region (>4 GB)), so we don't need to support arbitrary
per VCPU addresses, but could just use the 1st or 2nd map depending on
the VCPU number.
Is this too hackish?
If not, I would add another vgic_addr type (like
KVM_VGIC_V3_ADDR_TYPE_REDIST_UPPER or so) to be used from userland and
use that in the handle_mmio region detection.
Let me know if that sounds reasonable.
Cheers,
Andre.
next prev parent reply other threads:[~2014-12-02 17:32 UTC|newest]
Thread overview: 80+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-11-14 10:07 [PATCH v4 00/19] KVM GICv3 emulation Andre Przywara
2014-11-14 10:07 ` [PATCH v4 01/19] arm/arm64: KVM: rework MPIDR assignment and add accessors Andre Przywara
2014-11-18 10:35 ` Eric Auger
2014-11-23 9:34 ` Christoffer Dall
2014-11-14 10:07 ` [PATCH v4 02/19] arm/arm64: KVM: pass down user space provided GIC type into vGIC code Andre Przywara
2014-11-18 10:36 ` Eric Auger
2014-11-14 10:07 ` [PATCH v4 03/19] arm/arm64: KVM: refactor vgic_handle_mmio() function Andre Przywara
2014-11-18 10:35 ` Eric Auger
2014-11-14 10:07 ` [PATCH v4 04/19] arm/arm64: KVM: wrap 64 bit MMIO accesses with two 32 bit ones Andre Przywara
2014-11-18 10:36 ` Eric Auger
2014-11-23 9:42 ` Christoffer Dall
2014-11-24 13:50 ` Andre Przywara
2014-11-24 14:40 ` Christoffer Dall
2014-11-14 10:07 ` [PATCH v4 05/19] arm/arm64: KVM: introduce per-VM ops Andre Przywara
2014-11-23 9:58 ` Christoffer Dall
2014-11-14 10:07 ` [PATCH v4 06/19] arm/arm64: KVM: move kvm_register_device_ops() into vGIC probing Andre Przywara
2014-11-18 10:43 ` Eric Auger
2014-11-18 10:58 ` Eric Auger
2014-11-18 11:03 ` Andre Przywara
2014-11-14 10:07 ` [PATCH v4 07/19] arm/arm64: KVM: dont rely on a valid GICH base address Andre Przywara
2014-11-14 10:07 ` [PATCH v4 08/19] arm/arm64: KVM: make the maximum number of vCPUs a per-VM value Andre Przywara
2014-11-23 13:21 ` Christoffer Dall
2014-12-08 14:10 ` Andre Przywara
2014-11-14 10:07 ` [PATCH v4 09/19] arm/arm64: KVM: make the value of ICC_SRE_EL1 a per-VM variable Andre Przywara
2014-11-14 10:07 ` [PATCH v4 10/19] arm/arm64: KVM: refactor MMIO accessors Andre Przywara
2014-11-14 10:07 ` [PATCH v4 11/19] arm/arm64: KVM: refactor/wrap vgic_set/get_attr() Andre Przywara
2014-11-23 13:27 ` Christoffer Dall
2014-11-14 10:07 ` [PATCH v4 12/19] arm/arm64: KVM: add vgic.h header file Andre Przywara
2014-11-18 14:07 ` Eric Auger
2014-11-18 15:24 ` Andre Przywara
2014-11-23 13:29 ` Christoffer Dall
2014-11-14 10:07 ` [PATCH v4 13/19] arm/arm64: KVM: split GICv2 specific emulation code from vgic.c Andre Przywara
2014-11-23 13:32 ` Christoffer Dall
2014-11-14 10:07 ` [PATCH v4 14/19] arm/arm64: KVM: add opaque private pointer to MMIO data Andre Przywara
2014-11-23 13:33 ` Christoffer Dall
2014-11-14 10:07 ` [PATCH v4 15/19] arm/arm64: KVM: add virtual GICv3 distributor emulation Andre Przywara
2014-11-14 11:07 ` Christoffer Dall
2014-11-17 13:58 ` Andre Przywara
2014-11-17 23:46 ` Christoffer Dall
2014-11-18 15:57 ` Eric Auger
2014-11-23 14:38 ` Christoffer Dall
2014-11-24 16:00 ` Andre Przywara
2014-11-25 10:41 ` Christoffer Dall
2014-11-28 15:24 ` Andre Przywara
2014-11-30 8:30 ` Christoffer Dall
2014-12-02 16:24 ` Andre Przywara
2014-12-02 17:06 ` Marc Zyngier
2014-12-02 17:32 ` Andre Przywara [this message]
2014-12-03 10:30 ` Christoffer Dall
2014-12-03 10:47 ` Andre Przywara
2014-12-03 11:06 ` Christoffer Dall
2014-12-03 10:29 ` Christoffer Dall
2014-12-03 10:44 ` Marc Zyngier
2014-12-03 11:07 ` Christoffer Dall
2014-12-03 10:28 ` Christoffer Dall
2014-12-03 11:10 ` Andre Przywara
2014-12-03 11:28 ` Arnd Bergmann
2014-12-03 11:39 ` Peter Maydell
2014-12-03 12:03 ` Andre Przywara
2014-12-03 13:14 ` Arnd Bergmann
2014-12-04 9:34 ` Christoffer Dall
2014-12-04 10:02 ` Eric Auger
2014-11-14 10:08 ` [PATCH v4 16/19] arm64: GICv3: introduce symbolic names for GICv3 ICC_SGI1R_EL1 fields Andre Przywara
2014-11-23 14:43 ` Christoffer Dall
2014-11-14 10:08 ` [PATCH v4 17/19] arm64: KVM: add SGI generation register emulation Andre Przywara
2014-11-23 15:08 ` Christoffer Dall
2014-11-24 16:37 ` Andre Przywara
2014-11-25 11:03 ` Christoffer Dall
2014-11-28 15:40 ` Andre Przywara
2014-11-30 8:45 ` Christoffer Dall
2014-12-03 17:50 ` Andre Przywara
2014-12-03 20:22 ` Christoffer Dall
2014-11-14 10:08 ` [PATCH v4 18/19] arm/arm64: KVM: enable kernel side of GICv3 emulation Andre Przywara
2014-11-24 9:09 ` Christoffer Dall
2014-11-24 17:41 ` Andre Przywara
2014-11-25 11:08 ` Christoffer Dall
2014-11-14 10:08 ` [PATCH v4 19/19] arm/arm64: KVM: allow userland to request a virtual GICv3 Andre Przywara
2014-11-24 9:39 ` Christoffer Dall
2014-11-24 9:33 ` [PATCH v4 00/19] KVM GICv3 emulation Eric Auger
2014-11-24 17:46 ` Andre Przywara
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=547DF7BD.1000503@arm.com \
--to=andre.przywara@arm$(echo .)com \
--cc=linux-arm-kernel@lists$(echo .)infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox