From: catalin.marinas@arm•com (Catalin Marinas)
To: linux-arm-kernel@lists•infradead.org
Subject: [PATCH] ARM64: KVM: Fix coherent_icache_guest_page() for host with external L3-cache.
Date: Fri, 30 Aug 2013 14:02:38 +0100 [thread overview]
Message-ID: <20130830130238.GB4650@arm.com> (raw)
In-Reply-To: <CAAhSdy0Hm1k+mWGGxUOzAA=5yVGNOERfo=36jcE1cvkUM=xtVg@mail.gmail.com>
On Fri, Aug 30, 2013 at 11:44:30AM +0100, Anup Patel wrote:
> On Fri, Aug 30, 2013 at 3:22 PM, Catalin Marinas
> <catalin.marinas@arm•com> wrote:
> > On Fri, Aug 16, 2013 at 07:11:51PM +0100, Anup Patel wrote:
> >> On Fri, Aug 16, 2013 at 11:20 PM, Christoffer Dall
> >> <christoffer.dall@linaro•org> wrote:
> >> > On Fri, Aug 16, 2013 at 11:12:08PM +0530, Anup Patel wrote:
> >> >> Discussion here is about getting KVM ARM64 working in-presence
> >> >> of an external L3-cache (i.e. not part of CPU). Before starting a VCPU
> >> >> user-space typically loads images to guest RAM so, in-presence of
> >> >> huge L3-cache (few MBs). When the VCPU starts running some of the
> >> >> contents guest RAM will be still in L3-cache and VCPU runs with
> >> >> MMU off (i.e. cacheing off) hence VCPU will bypass L3-cache and
> >> >> see incorrect contents. To solve this problem we need to flush the
> >> >> guest RAM contents before they are accessed by first time by VCPU.
> >> >>
> >> > ok, I'm with you that far.
> >> >
> >> > But is it also not true that we need to decide between:
> >> >
> >> > A.1: Flush the entire guest RAM before running the VCPU
> >> > A.2: Flush the pages as we fault them in
> >>
> >> Yes, thats the decision we have to make.
> >>
> >> >
> >> > And (independently):
> >> >
> >> > B.1: Use __flush_dcache_range
> >> > B.2: Use something else + outer cache framework for arm64
> >>
> >> This would be __flush_dcache_all() + outer cache flush all.
> >
> > We need to be careful here since the __flush_dcache_all() operation uses
> > cache maintenance by set/way and these are *local* to a CPU (IOW not
> > broadcast). Do you have any guarantee that dirty cache lines don't
> > migrate between CPUs and __flush_dcache_all() wouldn't miss them?
> > Architecturally we don't, so this is not a safe operation that would
> > guarantee L1 cache flushing (we probably need to revisit some of the
> > __flush_dcache_all() calls in KVM, I haven't looked into this).
> >
> > So I think we are left to the range operation where the DC ops to PoC
> > would be enough for your L3.
>
> If __flush_dcache_all() is *local" to a CPU then I guess DC ops to PoC
> by range would be the only option.
Yes. In the (upcoming) ARM ARMv8 there is a clear note that set/way
operations to flush the whole cache must not be used for the maintenance
of large buffer but only during power-down/power-up code sequences.
> > An outer cache flush all is probably only needed for cpuidle/suspend
> > (the booting part should be handled by the boot loader).
>
> Yes, cpuidle/suspend would definitely require outer cache maintenance.
>
> For KVM, we can avoid flushing d-cache to PoC every time in
> coherent_icache_guest_page() by only doing it when Guest MMU is
> turned-off. This may reduce the performance penalty.
That's for the KVM guys to decide ;)
--
Catalin
next prev parent reply other threads:[~2013-08-30 13:02 UTC|newest]
Thread overview: 58+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-08-14 11:47 [PATCH] ARM64: KVM: Fix coherent_icache_guest_page() for host with external L3-cache Pranavkumar Sawargaonkar
2013-08-14 12:04 ` Marc Zyngier
2013-08-14 14:22 ` Anup Patel
2013-08-14 15:06 ` Alexander Graf
2013-08-14 15:34 ` Marc Zyngier
2013-08-14 15:41 ` Peter Maydell
2013-08-14 15:57 ` Marc Zyngier
2013-08-14 16:36 ` Anup Patel
2013-08-14 15:23 ` Marc Zyngier
2013-08-14 15:35 ` Peter Maydell
2013-08-14 15:49 ` Marc Zyngier
2013-08-14 17:34 ` Christoffer Dall
2013-08-15 4:44 ` Marc Zyngier
2013-08-15 16:58 ` Christoffer Dall
2013-08-14 15:36 ` Anup Patel
2013-08-15 4:52 ` Marc Zyngier
2013-08-15 6:26 ` Anup Patel
2013-08-15 8:31 ` Marc Zyngier
2013-08-15 13:31 ` Anup Patel
2013-08-15 14:47 ` Marc Zyngier
2013-08-15 15:13 ` Anup Patel
2013-08-15 15:37 ` Marc Zyngier
2013-08-15 15:45 ` Anup Patel
2013-08-15 16:53 ` Christoffer Dall
2013-08-16 5:02 ` Anup Patel
2013-08-16 6:57 ` Anup Patel
2013-08-16 17:19 ` Christoffer Dall
2013-08-16 17:42 ` Anup Patel
2013-08-16 17:50 ` Christoffer Dall
2013-08-16 18:06 ` Christoffer Dall
2013-08-16 18:20 ` Anup Patel
2013-08-16 18:11 ` Anup Patel
2013-08-16 18:20 ` Christoffer Dall
2013-08-30 9:52 ` Catalin Marinas
2013-08-30 10:44 ` Anup Patel
2013-08-30 13:02 ` Catalin Marinas [this message]
2013-08-30 13:21 ` Marc Zyngier
2013-08-30 14:04 ` Catalin Marinas
2013-08-30 14:22 ` Marc Zyngier
2013-08-30 14:30 ` Will Deacon
2013-08-30 14:52 ` Anup Patel
2013-08-30 15:12 ` Marc Zyngier
2013-08-29 10:52 ` Catalin Marinas
2013-08-29 12:31 ` Anup Patel
2013-08-29 12:53 ` Catalin Marinas
2013-08-29 16:02 ` Anup Patel
2013-08-30 9:44 ` Catalin Marinas
2013-08-30 10:36 ` Anup Patel
2013-08-30 12:52 ` Catalin Marinas
2013-08-16 17:14 ` Christoffer Dall
[not found] ` <CALrVBkvEP1Q0mKpv8ViOTLRvW2ks18MQXgmurSBHn+aJcz+=gw@mail.gmail.com>
2013-08-16 17:28 ` Christoffer Dall
2013-08-16 17:42 ` Christoffer Dall
2013-08-15 8:39 ` Pranavkumar Sawargaonkar
2013-08-15 15:42 ` Marc Zyngier
2013-08-14 14:23 ` Sudeep KarkadaNagesha
2013-08-14 14:35 ` Anup Patel
2013-08-14 17:37 ` Christoffer Dall
-- strict thread matches above, loose matches on Subject: below --
2013-08-14 11:45 Pranavkumar Sawargaonkar
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20130830130238.GB4650@arm.com \
--to=catalin.marinas@arm$(echo .)com \
--cc=linux-arm-kernel@lists$(echo .)infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox