From: catalin.marinas@arm•com (Catalin Marinas)
To: linux-arm-kernel@lists•infradead.org
Subject: [RFC PATCH] arm64: KVM: honor cacheability attributes on S2 page fault
Date: Fri, 11 Oct 2013 16:44:02 +0100 [thread overview]
Message-ID: <20131011154401.GE15803@arm.com> (raw)
In-Reply-To: <CAAhSdy2nfnDuAZW6XWz0Xc4ehcU9YcHKLv7WVSCgk3+b8AD7Hw@mail.gmail.com>
On Fri, Oct 11, 2013 at 04:32:48PM +0100, Anup Patel wrote:
> On Fri, Oct 11, 2013 at 8:29 PM, Marc Zyngier <marc.zyngier@arm•com> wrote:
> > On 11/10/13 15:50, Anup Patel wrote:
> >> On Fri, Oct 11, 2013 at 8:07 PM, Catalin Marinas
> >> <catalin.marinas@arm•com> wrote:
> >>> On Fri, Oct 11, 2013 at 03:27:16PM +0100, Anup Patel wrote:
> >>>> On Fri, Oct 11, 2013 at 6:08 PM, Catalin Marinas
> >>>> <catalin.marinas@arm•com> wrote:
> >>>>> On Thu, Oct 10, 2013 at 05:09:03PM +0100, Anup Patel wrote:
> >>>>>> Coming back to where we started, the actual problem was that when
> >>>>>> Guest starts booting it sees wrong contents because it is runs with
> >>>>>> MMU disable and correct contents are still in external L3 cache of X-Gene.
> >>>>>
> >>>>> That's one of the problems and I think the easiest to solve. Note that
> >>>>> contents could still be in the L1/L2 (inner) cache since whole cache
> >>>>> flushing by set/way isn't guaranteed in an MP context.
> >>>>>
> >>>>>> How about reconsidering the approach of flushing Guest RAM (entire or
> >>>>>> portion of it) to PoC by VA once before the first run of a VCPU ?
> >>>>>
> >>>>> Flushing the entire guest RAM is not possible by set/way
> >>>>> (architecturally) and not efficient by VA (though some benchmark would
> >>>>> be good). Marc's patch defers this flushing when a page is faulted in
> >>>>> (at stage 2) and I think it covers the initial boot.
> >>>>>
> >>>>>> OR
> >>>>>> We can also have KVM API using which user space can flush portions
> >>>>>> of Guest RAM before running the VCPU. (I think this was a suggestion
> >>>>>> from Marc Z initially)
> >>>>>
> >>>>> This may not be enough. It indeed flushes the kernel image that gets
> >>>>> loaded but the kernel would write other pages (bss, page tables etc.)
> >>>>> with MMU disabled and those addresses may contain dirty cache lines that
> >>>>> have not been covered by the initial kvmtool flush. So you basically
> >>>>> need all guest non-cacheable accesses to be flushed.
> >>>>>
> >>>>> The other problems are the cacheable aliases that I mentioned, so even
> >>>>> though the guest does non-cacheable accesses with the MMU off, the
> >>>>> hardware can still allocate into the cache via the other mappings. In
> >>>>> this case the guest needs to invalidate the areas of memory that it
> >>>>> wrote with caches off (or just use the DC bit to force memory accesses
> >>>>> with MMU off to be cacheable).
> >>>>
> >>>> Having looked at all the approaches, I would vote for the approach taken
> >>>> by this patch.
> >>>
> >>> But this patch alone doesn't solve the other issues. OTOH, the DC bit
> >>> would solve your initial problem and a few others.
> >>
> >> DC bit might solve the initial problem but it can be problematic because
> >> setting DC bit would mean Guest would have Caching ON even when Guest
> >> MMU is disabled. This will be more problematic if Guest is running a
> >> bootloader (uboot, grub, UEFI, ..) which does pass-through access to a
> >> DMA-capable device and we will have to change the bootloader in this
> >> case and put explicit flushes in bootloader for running inside Guest.
> >
> > Well, as Catalin mentioned, we'll have to do some cache maintenance in
> > the guest in any case.
>
> This would also mean that we will have to change Guest bootloader for
> running as Guest under KVM ARM64.
Yes.
> In x86 world, everything that can run natively also runs as Guest OS
> even if the Guest has pass-through devices.
I guess on x86 the I/O is also coherent, in which case we could use the
DC bit.
--
Catalin
next prev parent reply other threads:[~2013-10-11 15:44 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-09-10 9:51 [RFC PATCH] arm64: KVM: honor cacheability attributes on S2 page fault Marc Zyngier
2013-09-11 12:21 ` Anup Patel
2013-09-11 12:35 ` Marc Zyngier
2013-09-11 19:38 ` Christoffer Dall
2013-10-10 4:51 ` Anup Patel
2013-10-10 8:39 ` Marc Zyngier
2013-10-10 11:24 ` Catalin Marinas
2013-10-10 16:09 ` Anup Patel
2013-10-11 12:38 ` Catalin Marinas
2013-10-11 14:27 ` Anup Patel
2013-10-11 14:37 ` Catalin Marinas
2013-10-11 14:50 ` Anup Patel
2013-10-11 14:59 ` Marc Zyngier
2013-10-11 15:32 ` Anup Patel
2013-10-11 15:44 ` Catalin Marinas [this message]
2013-10-12 18:24 ` Anup Patel
2013-10-15 14:38 ` Catalin Marinas
2013-10-17 4:19 ` Anup Patel
2013-10-17 11:16 ` Catalin Marinas
2013-10-19 14:45 ` Christoffer Dall
2013-10-20 9:06 ` Catalin Marinas
2013-09-11 18:06 ` Peter Maydell
2013-09-11 19:25 ` Christoffer Dall
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20131011154401.GE15803@arm.com \
--to=catalin.marinas@arm$(echo .)com \
--cc=linux-arm-kernel@lists$(echo .)infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox