Subject: Re: [PATCH v8 00/10] Enable VAS

From: Michael Neuling <[email-masked]>
Date: Thu, 07 Sep 2017 17:00:30 +1000

So this is upstream now but it will cause a crash on boot with older skiboots
with: 

powernv-cpufreq: cpufreq pstate min 101 nominal 50 max 0
powernv-cpufreq: Workload Optimized Frequency is enabled in the platform
Disabling lock debugging due to kernel taint
Severe Machine check interrupt [Not recovered]
  NIP [c000000000098530]: reset_window_regs+0x20/0x220
  Initiator: CPU
  Error type: Unknown
opal: Machine check interrupt unrecoverable: MSR(RI=0)
opal: Hardware platform error: Unrecoverable Machine Check exception
CPU: 1 PID: 1 Comm: swapper/0 Tainted: G   M            4.13.0-rc7-00708-g8b680911e774-dirty #10
task: c000000f22680000 task.stack: c000000f22700000
NIP:  c000000000098530 LR: c000000000098758 CTR: 0000000000000000
REGS: c00000003ffebd80 TRAP: 0200   Tainted: G   M             (4.13.0-rc7-00708-g8b680911e774-dirty)
MSR:  9000000008349031 <SF,HV,EE,ME,IR,DR,LE>  CR: 24000224  XER: 00000000
CFAR: c000000000098754 DAR: 00000000100bef30 DSISR: 40000000 SOFTE: 0 
GPR00: c000000000098f44 c000000f22703a00 c000000000eff200 c000000f1cf861e0 
GPR04: c000000f22703a50 0000000000000001 0000000000000fff 0000000000000003 
GPR08: c00c0000842f0000 0000000000000001 0000000000000000 0000000000000000 
GPR12: 0000000000000000 c00000000fd40580 c000000000c03590 c000000000c1f428 
GPR16: c000000000c4a640 c000000000c31360 c000000000c92738 c000000000c92690 
GPR20: c000000000c926a0 c000000000c926f0 c000000f1d672940 0000000000000000 
GPR24: c000000000c92638 0000000000000000 c000000000e888b0 c000000000dce428 
GPR28: 0000000000000002 c000000f0e800000 c000000f22703a50 c000000f1cf861e0 
NIP [c000000000098530] reset_window_regs+0x20/0x220
LR [c000000000098758] init_winctx_regs+0x28/0x6c0
Call Trace:
[c000000f22703a00] [0000000000000002] 0x2 (unreliable)
[c000000f22703a30] [c000000000098f44] vas_rx_win_open.part.11+0x154/0x210
[c000000f22703ae0] [c000000000d668e8] nx842_powernv_init+0x6b4/0x824
[c000000f22703c[   38.412765557,0] OPAL: Reboot requested due to Platform error.
[   38.412828287,3] OPAL: Reboot requested due to Platform error.40] [c00000000000ca60] do_one_initcall+0x60/0x1c0

If you see this you need a new skiboot with at least these two patches:

    b503dcf16d vas: Set mmio enable bits in DD2
    a5c124072f vas: Set FIRs according to workbook

This is a community announcement brought to you by OzLabs. 
  OzLabs: Making Linux better since 1999

Mikey

On Mon, 2017-08-28 at 23:23 -0700, Sukadev Bhattiprolu wrote:
> Power9 introduces a hardware subsystem referred to as the Virtual
> Accelerator Switchboard (VAS). VAS allows kernel subsystems and user
> space processes to directly access the Nest Accelerator (NX) engines
> which implement compression and encryption algorithms in the hardware.
> 
> NX has been in Power processors since Power7+, but access to the NX
> engines was through the 'icswx' instruction which is only available
> to the kernel/hypervisor. Starting with Power9, access to the NX
> engines is provided to both kernel and user space processes through
> VAS.
> 
> The switchboard (i.e VAS) multiplexes accesses between "receivers" and
> "senders", where the "receivers" are typically the NX engines and
> "senders" are the kernel subsystems and user processors that wish to
> access the receivers (NX engines).  Once a sender is "connected" to
> a receiver through the switchboard, the senders can submit compression/
> encryption requests to the hardware using the new (PowerISA 3.0)
> "copy" and "paste" instructions.
> 
> In the initial OPAL and PowerNV kernel patchsets, the "senders" can
> only be kernel subsystems (eg NX-842 driver) and receivers can only
> be the NX-842 engine. Follow-on patch sets will allow senders/receivers
> to be user-space processes and receivers to be NX-GZIP engines.
> 
> Provides:
> 
> 	This kernel patch set configures the VAS subsystems and provides
> 	kernel interfaces to drivers like NX-842 to open receive and send
> 	windows in VAS and to submit compression requests to the NX engine.
> 
> Requires:
> 
> 	This patch set needs corresponding VAS/NX skiboot patches which
> 	were merged into skiboot tree. i.e skiboot must include:
> 	commit b503dcf ("vas: Set mmio enable bits in DD2")
> 
> Tests:
>         In-kernel compression requests were tested on DD1 and DD2 POWER9
> 	hardware using compression self-test module and the following
> 	NX-842 patch set from Haren Myneni:
> 
>         https://lists.ozlabs.org/pipermail/linuxppc-dev/2017-July/160620.html
> 
> 	and by dropping the last parameters to both vas_copy_crb() and
> 	vas_paste_crb() calls in drivers/crypto/nx/nx-842-powernv.c.
> 	See also PATCH 10/10.
> 
> Git Tree:
> 
>         https://github.com/sukadev/linux/ 
> 	Branch: vas-kern-v8
> 
> Thanks to input from Ben Herrenschmidt, Michael Neuling, Michael Ellerman
> and Haren Myneni.
> 
> Changelog[v8]:
> 	- [Michael Ellerman] Use kernel int types (u64, u32 etc); make VAS
> 	  a built-in rather than a module; drop unnecessary fields from
> 	  struct vas_instance; Update ISA references; use 0 or 1 with
> 	  SET_FIELD macros instead of bool; skip writing to SPARE registers;
> 	  minor cleanup of debug/error messages; retry if ida_get_new()
> 	  fails with EAGAIN; fix couple of leaks in ids in error handling;
> 	  drop vas_initialized() check; drop vas_win_id() and vas_paste_addr()
> 	  interfaces as they are not yet used; Set task_state() and fix
> 	  parameter to schedule_timeout(); Reuse existing copy/paste macros
> 	  drop unnecessary parameters and add cr0 to clobbers list
> 
> Changelog[v7]:
> 	- Drop support for user space send/receive FTW windows (will be
> 	  posted separately) Simplifies the rx-win-open interface a bit.
> 	- [Michael Ellerman] Move GET_FIELD/SET_FIELD macros from 
> 	  uapi/asm/vas.h to asm/vas.h.
> 
> Changelog[v6]
> 	- Add support for user space send/receive FTW windows
> 	- Add a new, NX-FTW driver which provides the FTW user interface
> 
> Changelog[v5]
> 	- [Ben Herrenschmidt] Make VAS a platform device in the device tree
> 	  and use the core platform functions to parse the VAS properties.
> 	  Map the VAS MMIO regions as non-cachable and paste regions as
> 	  cachable. Use CONFIG_PPC_VAS rather than CONFIG_VAS; Don't assume
> 	  VAS ids are sequential.
> 	- Copy the FIFO address as is into LFIFO_BAR (don't shift it).
> 
> Changelog[v4]
> 	Comments from Michael Neuling:
> 	- Move VAS code from drivers/misc/vas to arch/powerpc/platforms/powernv
> 	  since VAS only provides interfaces to other drivers like NX-842.
> 	- Drop vas-internal.h and use vas.h in separate dirs for VAS
> 	  internal, kernel API and user API
> 	- Rather than create 6 separate device tree properties windows
> 	  and window context, combine them into 6 "reg" properties.
> 	- Drop vas_window_reset() since windows are reset/cleared before
> 	  being assigned to kernel/users.
> 	- Use ilog2() and radix_enabled() helpers
> 
> Changelog[v3]
> 	- Rebase to v4.11-rc1
> 	- Add interfaces to initialize send/receive window attributes to
> 	  defaults that drivers can use (see arch/powerpc/include/asm/vas.h)
> 	- Modify interface vas_paste() to return 0 or error code
> 	- Fix a bug in setting Translation Control Mode (0b11 not 0x11)
> 	- Enable send-window-credit checking 
> 	- Reorg code  in vas_win_close()
> 	- Minor reorgs and tweaks to register field settings to make it
> 	  easier to add support for user space windows.
> 	- Skip writing to read-only registers
> 	- Start window indexing from 0 rather than 1
> 
> Changelog[v2]
> 	- Use vas-id, HVWC, UWC and paste address, entries from device tree
> 	  rather than defining/computing them in kernel and reorg code.
> 
> 
> Sukadev Bhattiprolu (10):
>   powerpc/vas: Define macros, register fields and structures
>   Move GET_FIELD/SET_FIELD to vas.h
>   powerpc/vas: Define vas_init() and vas_exit()
>   powerpc/vas: Define helpers to access MMIO regions
>   powerpc/vas: Define helpers to init window context
>   powerpc/vas: Define helpers to alloc/free windows
>   powerpc/vas: Define vas_rx_win_open() interface
>   powerpc/vas: Define vas_win_close() interface
>   powerpc/vas: Define vas_tx_win_open()
>   powerpc/vas: Define copy/paste interfaces
> 
>  .../devicetree/bindings/powerpc/ibm,vas.txt        |   23 +
>  MAINTAINERS                                        |    9 +
>  arch/powerpc/include/asm/ppc-opcode.h              |    2 +
>  arch/powerpc/include/asm/vas.h                     |  160 +++
>  arch/powerpc/platforms/powernv/Kconfig             |   14 +
>  arch/powerpc/platforms/powernv/Makefile            |    1 +
>  arch/powerpc/platforms/powernv/copy-paste.h        |   46 +
>  arch/powerpc/platforms/powernv/vas-window.c        | 1134
> ++++++++++++++++++++
>  arch/powerpc/platforms/powernv/vas.c               |  151 +++
>  arch/powerpc/platforms/powernv/vas.h               |  467 ++++++++
>  drivers/crypto/nx/nx-842-powernv.c                 |    7 +-
>  drivers/crypto/nx/nx-842.h                         |    5 -
>  12 files changed, 2011 insertions(+), 8 deletions(-)
>  create mode 100644 Documentation/devicetree/bindings/powerpc/ibm,vas.txt
>  create mode 100644 arch/powerpc/include/asm/vas.h
>  create mode 100644 arch/powerpc/platforms/powernv/copy-paste.h
>  create mode 100644 arch/powerpc/platforms/powernv/vas-window.c
>  create mode 100644 arch/powerpc/platforms/powernv/vas.c
>  create mode 100644 arch/powerpc/platforms/powernv/vas.h
>