Subject: Re: [DOC][PATCH] powerpc: Provide initial documentation for PAPR hcalls

From: Michael Ellerman <[email-masked]>
Date: Wed, 28 Aug 2019 23:24:30 +1000

Laurent Dufour <[email-masked]> writes:
> Le 27/08/2019 à 17:23, Vaibhav Jain a écrit :
>> This doc patch provides an initial description of the hcall op-codes
>> that are used by Linux kernel running as a guest (LPAR) on top of
>> PowerVM or any other sPAPR compliant hyper-visor (e.g qemu).
>> 
>> Apart from documenting the hcalls the doc-patch also provides a
>> rudimentary overview of how hcall ABI, how they are issued with the
>> Linux kernel and how information/control flows between the guest and
>> hypervisor.
>> 
>> Signed-off-by: Vaibhav Jain <[email-masked]>
>
> Hi Vaibhav,
>
> Thanks for documenting this.
>
> Besides my few remarks below, please consider:
>
> Reviewed-by: Laurent Dufour <[email-masked]>
>
>> ---
>> Change-log:
>> 
>> Initial version of this doc-patch was posted and reviewed as part of
>> the patch-series "[PATCH v5 0/4] powerpc/papr_scm: Workaround for
>> failure of drc bind after kexec"
>> https://patchwork.ozlabs.org/patch/1136022/. Changes introduced on top
>> the original patch:
>> 
>> * Replaced the of term PHYP with Hypervisor to indicate both
>> PowerVM/Qemu [Laurent]
>> * Emphasized that In/Out arguments to hcalls are in Big-endian format
>> [Laurent]
>> * Fixed minor word repetition, spell issues and grammatical error
>> [Michal, Mpe]
>> * Replaced various variant of term 'hcall' with a single
>> variant. [Mpe]
>> * Changed the documentation format from txt to ReST. [Mpe]
>> * Changed the name of documentation file to papr_hcalls.rst. [Mpe]
>> * Updated the section describing privileged operation by hypervisor
>> to be more accurate [Mpe].
>> * Fixed up mention of register notation used for describing
>> hcalls. [Mpe]
>> * s/NVDimm/NVDIMM [Mpe]
>> * Added section on return values from hcall [Mpe]
>> * Described H_CONTINUE return-value for long running hcalls.
>> ---
>>   Documentation/powerpc/papr_hcalls.rst | 200 ++++++++++++++++++++++++++
>>   1 file changed, 200 insertions(+)
>>   create mode 100644 Documentation/powerpc/papr_hcalls.rst
>> 
>> diff --git a/Documentation/powerpc/papr_hcalls.rst b/Documentation/powerpc/papr_hcalls.rst
>> new file mode 100644
>> index 000000000000..7afc0310de29
>> --- /dev/null
>> +++ b/Documentation/powerpc/papr_hcalls.rst
>> @@ -0,0 +1,200 @@
>> +===========================
>> +Hypercall Op-codes (hcalls)
>> +===========================
>> +
>> +Overview
>> +=========
>> +
>> +Virtualization on 64-bit Power Book3S Platforms is based on the PAPR
>> +specification [1]_ which describes the run-time environment for a guest
>> +operating system and how it should interact with the hypervisor for
>> +privileged operations. Currently there are two PAPR compliant hypervisors:
>> +
>> +- **IBM PowerVM (PHYP)**: IBM's proprietary hypervisor that supports AIX,
>> +  IBM-i and  Linux as supported guests (termed as Logical Partitions
>> +  or LPARS). It supports the full PAPR specification.
>> +
>> +- **Qemu/KVM**: Supports PPC64 linux guests running on a PPC64 linux host.
>> +  Though it only implements a subset of PAPR specification called LoPAPR [2]_.
>> +
>> +On PPC64 arch a guest kernel running on top of a PAPR hypervisor is called
>> +a *pSeries guest*. A pseries guest runs in a supervisor mode (HV=0) and must
>> +issue hypercalls to the hypervisor whenever it needs to perform an action
>> +that is hypervisor priviledged [3]_ or for other services managed by the
>> +hypervisor.
>> +
>> +Hence a Hypercall (hcall) is essentially a request by the pSeries guest
>> +asking hypervisor to perform a privileged operation on behalf of the guest. The
>> +guest issues a with necessary input operands. The hypervisor after performing
>                   ^ hcall ?
>
>> +the privilege operation returns a status code and output operands back to the
>> +guest.
>> +
>> +HCALL ABI
>> +=========
>> +The ABI specification for a hcall between a pSeries guest and PAPR hypervisor
>> +is covered in section 14.5.3 of ref [2]_. Switch to the  Hypervisor context is
>> +done via the instruction **HVCS** that expects the Opcode for hcall is set in *r3*
>> +and any in-arguments for the hcall are provided in registers *r4-r12* in
>> +Big-endian byte order.
> Indeed, register valuer are not byte ordered, only values passed through 
> buffer in memory are byte ordered.
>
> Should it be explicitly said that Big-endian order is only concerning data 
> stored in memory?
> What about something like that:
> "...any in-arguments for the hcall are provided in registers *r4-r12*. If 
> values have to be passed through a memory buffer, the data stored in that 
> buffer are in Big-endian order."

Yes that would be better.

I guess to be pedantic every structure passed in memory needs to be
defined in PAPR and could have some arbitrary ordering, but in practice
everything is big endian.

cheers