From: Arnd Bergmann <arnd@arndb•de>
To: linuxppc-dev@ozlabs•org
Cc: Mark Nelson <markn@au1•ibm.com>,
Roland Dreier <rdreier@cisco•com>,
cbe-oss-dev@ozlabs•org
Subject: Re: [patch 9/9] powerpc/cell: Add DMA_ATTR_STRONG_ORDERING dma attribute and use in IOMMU code
Date: Tue, 15 Jul 2008 23:27:06 +0200 [thread overview]
Message-ID: <200807152327.06946.arnd@arndb.de> (raw)
In-Reply-To: <ada3amaevj5.fsf@cisco.com>
On Tuesday 15 July 2008, Roland Dreier wrote:
> Sorry for the late comments, I missed this when it went by before.
>=20
> =C2=A0> +DMA_ATTR_STRONG_ORDERING
> =C2=A0> +----------------------
> =C2=A0> +
> =C2=A0> +DMA_ATTR_STRONG_ORDERING specifies that previous reads and write=
s are
> =C2=A0> +performed in the order in which they're received by the IOMMU; t=
hus
> =C2=A0> +reads and writes may not pass each other.
>=20
> I don't understand what this is trying to say. =C2=A0What is "previous"
> referring to? What does "received by the IOMMU" mean -- do you mean=20
> issued onto the bus by the CPU?
This is all about inbound transfers, i.e. DMAs coming from the I/O bridge
into the CPU, both DMA read and DMA write.
The relevant paragraph in the specification is=20
"If the SO bits in the I/O page table entry =3D =E2=80=9811=E2=80=99 and th=
e IOIF S-bit is =E2=80=981=E2=80=99,
this READ or WRITE cannot be placed on the EIB until all previous READs and
WRITEs from this CVCID and IOID have gotten an ACK or NULL type snoop respo=
nse."
Normally, this is only true for accesses going to the same cache line,
accesses from one device to different cache lines that are issued in order
also send their response in-order (unless you get an I/O exception, which m=
eans
you're toast), but can arrive at the I/O location out of order.
My interpretation is that strong ordering basically turns our whole I/O
subsystem into single-issue non-posted accesses (all devices are currently
configured to use the same CVCID and IOID), so we really should not
do that.
> When you say "reads and writes may not=20
> pass each other," do you mean just that reads may not pass writes and
> writes may not pass reads, or do you mean that reads also can't pass
> reads and writes can't pass writes?
>=20
> Since I don't know exactly what this attribute does, I can't be sure,
> but it seems that making weak ordering the default is dangerous in that
> it breaks drivers that expect usual memory ordering semantics. =C2=A0Woul=
d it
> be safer/better to make strong ordering the default and then add a
> "WEAK_ORDERING" attribute that drivers can use as an optimization?
With all our existing hardware, the I/O bridge overrides the setting
to select weak ordering. The purpose of this patch is to change the
default so that the bridge does not force weak ordering any more and
some drivers are free to set strong ordering without impacting performance
on the other drivers.
Strong ordering is only active when both the bridge and the IOMMU enable
it, but for correctly written drivers, this only results in a slowdown.
Arnd <><
next prev parent reply other threads:[~2008-07-15 21:27 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-07-15 19:51 [patch 0/9] Cell patches for 2.6.27, version 2 arnd
2008-07-15 19:51 ` [patch 1/9] powerpc/cell/edac: log a syndrome code in case of correctable error arnd
2008-07-17 5:59 ` Benjamin Herrenschmidt
2008-07-17 18:35 ` Doug Thompson
2008-07-15 19:51 ` [patch 2/9] powerpc/axonram: use only one block device major number arnd
2008-07-15 19:51 ` [patch 3/9] powerpc/axonram: enable partitioning of the Axons DDR2 DIMMs arnd
2008-07-15 19:51 ` [patch 4/9] powerpc/cell/cpufreq: add spu aware cpufreq governor arnd
2008-07-15 19:51 ` [patch 5/9] powerpc/cell: cleanup sysreset_hack for IBM cell blades arnd
2008-07-15 19:51 ` [patch 6/9] powerpc/cell: add support for power button of future " arnd
2008-07-15 19:51 ` [patch 7/9] azfs: initial submit of azfs, a non-buffered filesystem arnd
2008-07-17 6:13 ` Benjamin Herrenschmidt
2008-07-22 9:49 ` [Cbe-oss-dev] " Christoph Hellwig
2008-07-15 19:51 ` [patch 8/9] powerpc/dma: use the struct dma_attrs in iommu code arnd
2008-07-15 19:51 ` [patch 9/9] powerpc/cell: Add DMA_ATTR_STRONG_ORDERING dma attribute and use in IOMMU code arnd
2008-07-15 20:34 ` Roland Dreier
2008-07-15 21:27 ` Arnd Bergmann [this message]
2008-07-16 2:18 ` Roland Dreier
2008-07-16 7:54 ` Arnd Bergmann
2008-07-17 6:20 ` [Cbe-oss-dev] " Benjamin Herrenschmidt
2008-07-17 14:53 ` Arnd Bergmann
2008-07-17 20:10 ` Benjamin Herrenschmidt
2008-07-17 20:10 ` Benjamin Herrenschmidt
2008-07-18 13:03 ` [PATCH] Add DMA_ATTR_WEAK_ORDERING dma attribute and use in Cell " Arnd Bergmann
2008-07-19 7:29 ` [Cbe-oss-dev] " Jeremy Kerr
2008-07-19 8:36 ` Arnd Bergmann
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=200807152327.06946.arnd@arndb.de \
--to=arnd@arndb$(echo .)de \
--cc=cbe-oss-dev@ozlabs$(echo .)org \
--cc=linuxppc-dev@ozlabs$(echo .)org \
--cc=markn@au1$(echo .)ibm.com \
--cc=rdreier@cisco$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox