From: Nirmoy Das <nirmoyd@nvidia•com>
To: Nicolin Chen <nicolinc@nvidia•com>, <jgg@nvidia•com>,
<will@kernel•org>, <robin.murphy@arm•com>, <bhelgaas@google•com>
Cc: <joro@8bytes•org>, <praan@google•com>, <baolu.lu@linux•intel.com>,
<kevin.tian@intel•com>, <miko.lenczewski@arm•com>,
<linux-arm-kernel@lists•infradead.org>, <iommu@lists•linux.dev>,
<linux-kernel@vger•kernel.org>, <linux-pci@vger•kernel.org>,
<dan.j.williams@intel•com>, <jonathan.cameron@huawei•com>,
<vsethi@nvidia•com>, <linux-cxl@vger•kernel.org>
Subject: Re: [PATCH v3 1/3] PCI: Allow ATS to be always on for CXL.cache capable devices
Date: Sun, 8 Mar 2026 21:53:25 +0100 [thread overview]
Message-ID: <8dc03385-aede-4774-9951-e76fc5cb2628@nvidia.com> (raw)
In-Reply-To: <2d523f6d-a4a5-44f5-b588-e6ad520322ff@nvidia.com>
On 08.03.26 21:49, Nirmoy Das wrote:
>
> On 07.03.26 00:41, Nicolin Chen wrote:
>> Controlled by the IOMMU driver, ATS is usually enabled "on demand"
>> when a
>> device requests a translation service from its associated IOMMU HW
>> running
>> on the channel of a given PASID. This is working even when a device
>> has no
>> translation on its RID (i.e., the RID is IOMMU bypassed).
>>
>> However, certain PCIe devices require non-PASID ATS on their RID even
>> when
>> the RID is IOMMU bypassed. Call this "always on".
>>
>> For instance, the CXL spec notes in "3.2.5.13 Memory Type on CXL.cache":
>> "To source requests on CXL.cache, devices need to get the Host Physical
>> Address (HPA) from the Host by means of an ATS request on CXL.io."
>>
>> In other words, the CXL.cache capability requires ATS; otherwise, it
>> can't
>> access host physical memory.
>>
>> Introduce a new pci_ats_always_on() helper for the IOMMU driver to
>> scan a
>> PCI device and shift ATS policies between "on demand" and "always on".
>>
>> Add the support for CXL.cache devices first. Pre-CXL devices will be
>> added
>> in quirks.c file.
>>
>> Note that pci_ats_always_on() validates against pci_ats_supported(),
>> so we
>> ensure that untrusted devices (e.g. external ports) will not be
>> always on.
>> This maintains the existing ATS security policy regarding potential
>> side-
>> channel attacks via ATS.
>>
>> Cc: linux-cxl@vger•kernel.org
>> Suggested-by: Vikram Sethi <vsethi@nvidia•com>
>> Suggested-by: Jason Gunthorpe <jgg@nvidia•com>
>> Signed-off-by: Nicolin Chen <nicolinc@nvidia•com>
>
> Tested the series with a Type 2 CXL device.
>
> Tested-by: Nirmoy Das <nirmoyd@nvidia•com>
>
> Acked-by: Nirmoy Das <nirmoy@nvidia•com>
Sent with wrong email address
Acked-by: Nirmoy Das <nirmoyd@nvidia•com>
>
>> ---
>> include/linux/pci-ats.h | 3 +++
>> include/uapi/linux/pci_regs.h | 1 +
>> drivers/pci/ats.c | 42 +++++++++++++++++++++++++++++++++++
>> 3 files changed, 46 insertions(+)
>>
>> diff --git a/include/linux/pci-ats.h b/include/linux/pci-ats.h
>> index 75c6c86cf09dc..d14ba727d38b3 100644
>> --- a/include/linux/pci-ats.h
>> +++ b/include/linux/pci-ats.h
>> @@ -12,6 +12,7 @@ int pci_prepare_ats(struct pci_dev *dev, int ps);
>> void pci_disable_ats(struct pci_dev *dev);
>> int pci_ats_queue_depth(struct pci_dev *dev);
>> int pci_ats_page_aligned(struct pci_dev *dev);
>> +bool pci_ats_always_on(struct pci_dev *dev);
>> #else /* CONFIG_PCI_ATS */
>> static inline bool pci_ats_supported(struct pci_dev *d)
>> { return false; }
>> @@ -24,6 +25,8 @@ static inline int pci_ats_queue_depth(struct
>> pci_dev *d)
>> { return -ENODEV; }
>> static inline int pci_ats_page_aligned(struct pci_dev *dev)
>> { return 0; }
>> +static inline bool pci_ats_always_on(struct pci_dev *dev)
>> +{ return false; }
>> #endif /* CONFIG_PCI_ATS */
>> #ifdef CONFIG_PCI_PRI
>> diff --git a/include/uapi/linux/pci_regs.h
>> b/include/uapi/linux/pci_regs.h
>> index 14f634ab9350d..6ac45be1008b8 100644
>> --- a/include/uapi/linux/pci_regs.h
>> +++ b/include/uapi/linux/pci_regs.h
>> @@ -1349,6 +1349,7 @@
>> /* CXL r4.0, 8.1.3: PCIe DVSEC for CXL Device */
>> #define PCI_DVSEC_CXL_DEVICE 0
>> #define PCI_DVSEC_CXL_CAP 0xA
>> +#define PCI_DVSEC_CXL_CACHE_CAPABLE _BITUL(0)
>> #define PCI_DVSEC_CXL_MEM_CAPABLE _BITUL(2)
>> #define PCI_DVSEC_CXL_HDM_COUNT __GENMASK(5, 4)
>> #define PCI_DVSEC_CXL_CTRL 0xC
>> diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c
>> index ec6c8dbdc5e9c..cf262eb6e6890 100644
>> --- a/drivers/pci/ats.c
>> +++ b/drivers/pci/ats.c
>> @@ -205,6 +205,48 @@ int pci_ats_page_aligned(struct pci_dev *pdev)
>> return 0;
>> }
>> +/*
>> + * CXL r4.0, sec 3.2.5.13 Memory Type on CXL.cache notes: to source
>> requests on
>> + * CXL.cache, devices need to get the Host Physical Address (HPA)
>> from the Host
>> + * by means of an ATS request on CXL.io.
>> + *
>> + * In other world, CXL.cache devices cannot access host physical
>> memory without
>> + * ATS.
>> + */
>> +static bool pci_cxl_ats_always_on(struct pci_dev *pdev)
>> +{
>> + u16 cap = 0;
>> + int offset;
>> +
>> + offset = pci_find_dvsec_capability(pdev, PCI_VENDOR_ID_CXL,
>> + PCI_DVSEC_CXL_DEVICE);
>> + if (!offset)
>> + return false;
>> +
>> + pci_read_config_word(pdev, offset + PCI_DVSEC_CXL_CAP, &cap);
>> +
>> + return cap & PCI_DVSEC_CXL_CACHE_CAPABLE;
>> +}
>> +
>> +/**
>> + * pci_ats_always_on - Whether the PCI device requires ATS to be
>> always enabled
>> + * @pdev: the PCI device
>> + *
>> + * Returns true, if the PCI device requires ATS for basic functional
>> operation.
>> + */
>> +bool pci_ats_always_on(struct pci_dev *pdev)
>> +{
>> + if (pci_ats_disabled() || !pci_ats_supported(pdev))
>> + return false;
>> +
>> + /* A VF inherits its PF's requirement for ATS function */
>> + if (pdev->is_virtfn)
>> + pdev = pci_physfn(pdev);
>> +
>> + return pci_cxl_ats_always_on(pdev);
>> +}
>> +EXPORT_SYMBOL_GPL(pci_ats_always_on);
>> +
>> #ifdef CONFIG_PCI_PRI
>> void pci_pri_init(struct pci_dev *pdev)
>> {
>
next prev parent reply other threads:[~2026-03-08 20:54 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-03-06 23:41 [PATCH v3 0/3] Allow ATS to be always on for certain ATS-capable devices Nicolin Chen
2026-03-06 23:41 ` [PATCH v3 1/3] PCI: Allow ATS to be always on for CXL.cache capable devices Nicolin Chen
2026-03-08 20:49 ` Nirmoy Das
2026-03-08 20:53 ` Nirmoy Das [this message]
2026-03-09 11:48 ` Jonathan Cameron
2026-03-26 21:38 ` Bjorn Helgaas
2026-03-26 21:51 ` Jason Gunthorpe
2026-03-30 12:48 ` Jason Gunthorpe
2026-03-31 8:19 ` Tian, Kevin
2026-04-09 22:45 ` Nicolin Chen
2026-04-09 22:52 ` Jason Gunthorpe
2026-04-10 0:04 ` Nicolin Chen
2026-04-10 3:13 ` Tian, Kevin
2026-04-10 12:05 ` Jason Gunthorpe
2026-04-13 6:40 ` Tian, Kevin
2026-03-06 23:41 ` [PATCH v3 2/3] PCI: Allow ATS to be always on for pre-CXL devices Nicolin Chen
2026-03-08 20:50 ` Nirmoy Das
2026-03-08 20:54 ` Nirmoy Das
2026-03-09 11:50 ` Jonathan Cameron
2026-03-30 12:49 ` Jason Gunthorpe
2026-03-31 8:24 ` Tian, Kevin
2026-03-06 23:41 ` [PATCH v3 3/3] iommu/arm-smmu-v3: Allow ATS to be always on Nicolin Chen
2026-03-08 20:52 ` Nirmoy Das
2026-03-30 12:51 ` Jason Gunthorpe
2026-03-31 8:40 ` Tian, Kevin
2026-03-31 12:08 ` Jason Gunthorpe
2026-04-01 8:15 ` Tian, Kevin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=8dc03385-aede-4774-9951-e76fc5cb2628@nvidia.com \
--to=nirmoyd@nvidia$(echo .)com \
--cc=baolu.lu@linux$(echo .)intel.com \
--cc=bhelgaas@google$(echo .)com \
--cc=dan.j.williams@intel$(echo .)com \
--cc=iommu@lists$(echo .)linux.dev \
--cc=jgg@nvidia$(echo .)com \
--cc=jonathan.cameron@huawei$(echo .)com \
--cc=joro@8bytes$(echo .)org \
--cc=kevin.tian@intel$(echo .)com \
--cc=linux-arm-kernel@lists$(echo .)infradead.org \
--cc=linux-cxl@vger$(echo .)kernel.org \
--cc=linux-kernel@vger$(echo .)kernel.org \
--cc=linux-pci@vger$(echo .)kernel.org \
--cc=miko.lenczewski@arm$(echo .)com \
--cc=nicolinc@nvidia$(echo .)com \
--cc=praan@google$(echo .)com \
--cc=robin.murphy@arm$(echo .)com \
--cc=vsethi@nvidia$(echo .)com \
--cc=will@kernel$(echo .)org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox