Subject: Re: [PATCH v2] powerpc: slightly improve cache helpers

From: Nathan Chancellor <[email-masked]>
Date: Sun, 21 Jul 2019 00:58:46 -0700

On Fri, Jul 19, 2019 at 09:04:55AM -0700, Nathan Chancellor wrote:
> On Fri, Jul 19, 2019 at 10:23:03AM -0500, Segher Boessenkool wrote:
> > On Thu, Jul 18, 2019 at 08:24:56PM -0700, Nathan Chancellor wrote:
> > > On Mon, Jul 08, 2019 at 11:49:52PM -0700, Nathan Chancellor wrote:
> > > > On Tue, Jul 09, 2019 at 07:04:43AM +0200, Christophe Leroy wrote:
> > > > > Is that a Clang bug ?
> > > > 
> > > > No idea, it happens with clang-8 and clang-9 though (pretty sure there
> > > > were fixes for PowerPC in clang-8 so something before it probably won't
> > > > work but I haven't tried).
> > > > 
> > > > > 
> > > > > Do you have a disassembly of the code both with and without this patch in
> > > > > order to compare ?
> > > > 
> > > > I can give you whatever disassembly you want (or I can upload the raw
> > > > files if that is easier).
> > > 
> > > What disassembly/files did you need to start taking a look at this? I
> > > can upload/send whatever you need.
> > 
> > A before and after of *only this patch*.  And then look at what changed;
> > it maybe be obvious what is the problem to you, as well, so look at it
> > yourself first?
> > 
> > 
> > Segher

Hi Segher,

Looks like the problematic function is dcbz, as there is no segfault
when only that function is reverted to a
pre-6c5875843b87c3adea2beade9d1b8b3d4523900a state.

I was able to expose a singular problematic callsite using the attached
patch (let me know if that is insufficient).

I have attached the disassembly of arch/powerpc/kernel/mem.o with
clear_page (working) and broken_clear_page (broken), along with the side
by side diff. My assembly knowledge is fairly limited as it stands and
it is certainly not up to snuff on PowerPC so I have no idea what I am
looking for. Please let me know if anything immediately looks off or if
there is anything else I can do to help out.

Cheers,
Nathan
>From 3d7f79f7601c312d47245141185bea7defffffa4 Mon Sep 17 00:00:00 2001
From: Nathan Chancellor <[email-masked]>
Date: Sat, 20 Jul 2019 23:37:55 -0700
Subject: [PATCH] powerpc: Test broken dcbz

Signed-off-by: Nathan Chancellor <[email-masked]>
---
 arch/powerpc/include/asm/cache.h   |  2 +-
 arch/powerpc/include/asm/page_32.h | 13 +++++++++++++
 arch/powerpc/mm/mem.c              |  2 +-
 3 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/arch/powerpc/include/asm/cache.h b/arch/powerpc/include/asm/cache.h
index b3388d95f451..ba76291b4d4d 100644
--- a/arch/powerpc/include/asm/cache.h
+++ b/arch/powerpc/include/asm/cache.h
@@ -107,7 +107,7 @@ extern void _set_L3CR(unsigned long);
 
 static inline void dcbz(void *addr)
 {
-	__asm__ __volatile__ ("dcbz %y0" : : "Z"(*(u8 *)addr) : "memory");
+	__asm__ __volatile__ ("dcbz 0, %0" : : "r"(addr) : "memory");
 }
 
 static inline void dcbi(void *addr)
diff --git a/arch/powerpc/include/asm/page_32.h b/arch/powerpc/include/asm/page_32.h
index 683dfbc67ca8..125c2ba7cd79 100644
--- a/arch/powerpc/include/asm/page_32.h
+++ b/arch/powerpc/include/asm/page_32.h
@@ -40,6 +40,19 @@ typedef unsigned long long pte_basic_t;
 typedef unsigned long pte_basic_t;
 #endif
 
+static inline void broken_dcbz(void *addr)
+{
+	__asm__ __volatile__ ("dcbz %y0" : : "Z"(*(u8 *)addr) : "memory");
+}
+
+static inline void broken_clear_page(void *addr)
+{
+	unsigned int i;
+
+	for (i = 0; i < PAGE_SIZE / L1_CACHE_BYTES; i++, addr += L1_CACHE_BYTES)
+		broken_dcbz(addr);
+}
+
 /*
  * Clear page using the dcbz instruction, which doesn't cause any
  * memory traffic (except to write out any cache lines which get
diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
index 9259337d7374..7bb88e7a2e4c 100644
--- a/arch/powerpc/mm/mem.c
+++ b/arch/powerpc/mm/mem.c
@@ -361,7 +361,7 @@ EXPORT_SYMBOL(flush_dcache_icache_page);
 
 void clear_user_page(void *page, unsigned long vaddr, struct page *pg)
 {
-	clear_page(page);
+	broken_clear_page(page);
 
 	/*
 	 * We shouldn't have to do this, but some versions of glibc
-- 
2.22.0