From: Jonathan Cameron <Jonathan.Cameron@Huawei•com>
To: Zhenzhong Duan <zhenzhong.duan@intel•com>
Cc: linmiaohe@huawei•com, alison.schofield@intel•com,
rafael@kernel•org, sathyanarayanan.kuppuswamy@intel•com,
linux-pci@vger•kernel.org, erwin.tsaur@intel•com,
linux-cxl@vger•kernel.org, linux-kernel@vger•kernel.org,
oohall@gmail•com, ira.weiny@intel•com, dave@stgolabs•net,
dave.jiang@intel•com, vishal.l.verma@intel•com,
Smita.KoralahalliChannabasappa@amd•com,
linux-acpi@vger•kernel.org, helgaas@kernel•org, lenb@kernel•org,
chao.p.peng@intel•com, rrichter@amd•com, yudong.wang@intel•com,
bp@alien8•de, qingshun.wang@linux•intel.com, bhelgaas@google•com,
dan.j.williams@intel•com, linux-edac@vger•kernel.org,
tony.luck@intel•com, feiting.wanyan@intel•com,
adam.c.preble@intel•com, mahesh@linux•ibm.com,
leoyang.li@nxp•com, lukas@wunner•de, james.morse@arm•com,
linuxppc-dev@lists•ozlabs.org, shiju.jose@huawei•com
Subject: Re: [PATCH v3 1/3] PCI/AER: Store UNCOR_STATUS bits that might be ANFE in aer_err_info
Date: Mon, 22 Apr 2024 17:16:29 +0100 [thread overview]
Message-ID: <20240422171629.00005675@Huawei.com> (raw)
In-Reply-To: <20240417061407.1491361-2-zhenzhong.duan@intel.com>
On Wed, 17 Apr 2024 14:14:05 +0800
Zhenzhong Duan <zhenzhong.duan@intel•com> wrote:
> In some cases the detector of a Non-Fatal Error(NFE) is not the most
> appropriate agent to determine the type of the error. For example,
> when software performs a configuration read from a non-existent
> device or Function, completer will send an ERR_NONFATAL Message.
> On some platforms, ERR_NONFATAL results in a System Error, which
> breaks normal software probing.
>
> Advisory Non-Fatal Error(ANFE) is a special case that can be used
> in above scenario. It is predominantly determined by the role of the
> detecting agent (Requester, Completer, or Receiver) and the specific
> error. In such cases, an agent with AER signals the NFE (if enabled)
> by sending an ERR_COR Message as an advisory to software, instead of
> sending ERR_NONFATAL.
>
> When processing an ANFE, ideally both correctable error(CE) status and
> uncorrectable error(UE) status should be cleared. However, there is no
> way to fully identify the UE associated with ANFE. Even worse, a Fatal
> Error(FE) or Non-Fatal Error(NFE) may set the same UE status bit as
> ANFE. Treating an ANFE as NFE will reproduce above mentioned issue,
> i.e., breaking softwore probing; treating NFE as ANFE will make us
> ignoring some UEs which need active recover operation. To avoid clearing
> UEs that are not ANFE by accident, the most conservative route is taken
> here: If any of the FE/NFE Detected bits is set in Device Status, do not
> touch UE status, they should be cleared later by the UE handler. Otherwise,
> a specific set of UEs that may be raised as ANFE according to the PCIe
> specification will be cleared if their corresponding severity is Non-Fatal.
>
> To achieve above purpose, store UNCOR_STATUS bits that might be ANFE
> in aer_err_info.anfe_status. So that those bits could be printed and
> processed later.
>
> Tested-by: Yudong Wang <yudong.wang@intel•com>
> Co-developed-by: "Wang, Qingshun" <qingshun.wang@linux•intel.com>
> Signed-off-by: "Wang, Qingshun" <qingshun.wang@linux•intel.com>
> Signed-off-by: Zhenzhong Duan <zhenzhong.duan@intel•com>
> ---
> drivers/pci/pci.h | 1 +
> drivers/pci/pcie/aer.c | 45 ++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 46 insertions(+)
>
> diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
> index 17fed1846847..3f9eb807f9fd 100644
> --- a/drivers/pci/pci.h
> +++ b/drivers/pci/pci.h
> @@ -412,6 +412,7 @@ struct aer_err_info {
>
> unsigned int status; /* COR/UNCOR Error Status */
> unsigned int mask; /* COR/UNCOR Error Mask */
> + unsigned int anfe_status; /* UNCOR Error Status for ANFE */
> struct pcie_tlp_log tlp; /* TLP Header */
> };
>
> diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c
> index ac6293c24976..27364ab4b148 100644
> --- a/drivers/pci/pcie/aer.c
> +++ b/drivers/pci/pcie/aer.c
> @@ -107,6 +107,12 @@ struct aer_stats {
> PCI_ERR_ROOT_MULTI_COR_RCV | \
> PCI_ERR_ROOT_MULTI_UNCOR_RCV)
>
> +#define AER_ERR_ANFE_UNC_MASK (PCI_ERR_UNC_POISON_TLP | \
> + PCI_ERR_UNC_COMP_TIME | \
> + PCI_ERR_UNC_COMP_ABORT | \
> + PCI_ERR_UNC_UNX_COMP | \
> + PCI_ERR_UNC_UNSUP)
> +
> static int pcie_aer_disable;
> static pci_ers_result_t aer_root_reset(struct pci_dev *dev);
>
> @@ -1196,6 +1202,41 @@ void aer_recover_queue(int domain, unsigned int bus, unsigned int devfn,
> EXPORT_SYMBOL_GPL(aer_recover_queue);
> #endif
>
> +static void anfe_get_uc_status(struct pci_dev *dev, struct aer_err_info *info)
> +{
> + u32 uncor_mask, uncor_status;
> + u16 device_status;
> + int aer = dev->aer_cap;
> +
> + if (pcie_capability_read_word(dev, PCI_EXP_DEVSTA, &device_status))
> + return;
> + /*
> + * Take the most conservative route here. If there are
> + * Non-Fatal/Fatal errors detected, do not assume any
> + * bit in uncor_status is set by ANFE.
> + */
> + if (device_status & (PCI_EXP_DEVSTA_NFED | PCI_EXP_DEVSTA_FED))
> + return;
> +
Is there not a race here? If we happen to get either an NFED or FED
between the read of device_status above and here we might pick up a status
that corresponds to that (and hence clear something we should not).
Or am I missing that race being close somewhere?
> + pci_read_config_dword(dev, aer + PCI_ERR_UNCOR_STATUS, &uncor_status);
> + pci_read_config_dword(dev, aer + PCI_ERR_UNCOR_MASK, &uncor_mask);
> + /*
> + * According to PCIe Base Specification Revision 6.1,
> + * Section 6.2.3.2.4, if an UNCOR error is raised as
> + * Advisory Non-Fatal error, it will match the following
> + * conditions:
> + * a. The severity of the error is Non-Fatal.
> + * b. The error is one of the following:
> + * 1. Poisoned TLP (Section 6.2.3.2.4.3)
> + * 2. Completion Timeout (Section 6.2.3.2.4.4)
> + * 3. Completer Abort (Section 6.2.3.2.4.1)
> + * 4. Unexpected Completion (Section 6.2.3.2.4.5)
> + * 5. Unsupported Request (Section 6.2.3.2.4.1)
> + */
> + info->anfe_status = uncor_status & ~uncor_mask & ~info->severity &
> + AER_ERR_ANFE_UNC_MASK;
> +}
> +
> /**
> * aer_get_device_error_info - read error status from dev and store it to info
> * @dev: pointer to the device expected to have a error record
> @@ -1213,6 +1254,7 @@ int aer_get_device_error_info(struct pci_dev *dev, struct aer_err_info *info)
>
> /* Must reset in this function */
> info->status = 0;
> + info->anfe_status = 0;
> info->tlp_header_valid = 0;
>
> /* The device might not support AER */
> @@ -1226,6 +1268,9 @@ int aer_get_device_error_info(struct pci_dev *dev, struct aer_err_info *info)
> &info->mask);
> if (!(info->status & ~info->mask))
> return 0;
> +
> + if (info->status & PCI_ERR_COR_ADV_NFAT)
> + anfe_get_uc_status(dev, info);
> } else if (type == PCI_EXP_TYPE_ROOT_PORT ||
> type == PCI_EXP_TYPE_RC_EC ||
> type == PCI_EXP_TYPE_DOWNSTREAM ||
next prev parent reply other threads:[~2024-04-22 16:33 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-17 6:14 [PATCH v3 0/3] PCI/AER: Handle Advisory Non-Fatal error Zhenzhong Duan
2024-04-17 6:14 ` [PATCH v3 1/3] PCI/AER: Store UNCOR_STATUS bits that might be ANFE in aer_err_info Zhenzhong Duan
2024-04-22 16:16 ` Jonathan Cameron [this message]
2024-04-23 2:25 ` Duan, Zhenzhong
2024-04-26 16:11 ` Jonathan Cameron
2024-04-28 3:31 ` Duan, Zhenzhong
2024-05-01 12:24 ` Jonathan Cameron
2024-05-03 11:10 ` Duan, Zhenzhong
2024-04-17 6:14 ` [PATCH v3 2/3] PCI/AER: Print UNCOR_STATUS bits that might be ANFE Zhenzhong Duan
2024-04-17 6:14 ` [PATCH v3 3/3] PCI/AER: Clear " Zhenzhong Duan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240422171629.00005675@Huawei.com \
--to=jonathan.cameron@huawei$(echo .)com \
--cc=Smita.KoralahalliChannabasappa@amd$(echo .)com \
--cc=adam.c.preble@intel$(echo .)com \
--cc=alison.schofield@intel$(echo .)com \
--cc=bhelgaas@google$(echo .)com \
--cc=bp@alien8$(echo .)de \
--cc=chao.p.peng@intel$(echo .)com \
--cc=dan.j.williams@intel$(echo .)com \
--cc=dave.jiang@intel$(echo .)com \
--cc=dave@stgolabs$(echo .)net \
--cc=erwin.tsaur@intel$(echo .)com \
--cc=feiting.wanyan@intel$(echo .)com \
--cc=helgaas@kernel$(echo .)org \
--cc=ira.weiny@intel$(echo .)com \
--cc=james.morse@arm$(echo .)com \
--cc=lenb@kernel$(echo .)org \
--cc=leoyang.li@nxp$(echo .)com \
--cc=linmiaohe@huawei$(echo .)com \
--cc=linux-acpi@vger$(echo .)kernel.org \
--cc=linux-cxl@vger$(echo .)kernel.org \
--cc=linux-edac@vger$(echo .)kernel.org \
--cc=linux-kernel@vger$(echo .)kernel.org \
--cc=linux-pci@vger$(echo .)kernel.org \
--cc=linuxppc-dev@lists$(echo .)ozlabs.org \
--cc=lukas@wunner$(echo .)de \
--cc=mahesh@linux$(echo .)ibm.com \
--cc=oohall@gmail$(echo .)com \
--cc=qingshun.wang@linux$(echo .)intel.com \
--cc=rafael@kernel$(echo .)org \
--cc=rrichter@amd$(echo .)com \
--cc=sathyanarayanan.kuppuswamy@intel$(echo .)com \
--cc=shiju.jose@huawei$(echo .)com \
--cc=tony.luck@intel$(echo .)com \
--cc=vishal.l.verma@intel$(echo .)com \
--cc=yudong.wang@intel$(echo .)com \
--cc=zhenzhong.duan@intel$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox