public inbox for netdev@vger.kernel.org 
 help / color / mirror / Atom feed
From: Byungchul Park <byungchul@sk•com>
To: Bagas Sanjaya <bagasdotme@gmail•com>
Cc: linux-kernel@vger•kernel.org, kernel_team@skhynix•com,
	torvalds@linux-foundation•org, damien.lemoal@opensource•wdc.com,
	linux-ide@vger•kernel.org, adilger.kernel@dilger•ca,
	linux-ext4@vger•kernel.org, mingo@redhat•com,
	peterz@infradead•org, will@kernel•org, tglx@linutronix•de,
	rostedt@goodmis•org, joel@joelfernandes•org, sashal@kernel•org,
	daniel.vetter@ffwll•ch, duyuyang@gmail•com,
	johannes.berg@intel•com, tj@kernel•org, tytso@mit•edu,
	willy@infradead•org, david@fromorbit•com, amir73il@gmail•com,
	gregkh@linuxfoundation•org, kernel-team@lge•com,
	linux-mm@kvack•org, akpm@linux-foundation•org, mhocko@kernel•org,
	minchan@kernel•org, hannes@cmpxchg•org, vdavydov.dev@gmail•com,
	sj@kernel•org, jglisse@redhat•com, dennis@kernel•org,
	cl@linux•com, penberg@kernel•org, rientjes@google•com,
	vbabka@suse•cz, ngupta@vflare•org, linux-block@vger•kernel.org,
	josef@toxicpanda•com, linux-fsdevel@vger•kernel.org,
	jack@suse•cz, jlayton@kernel•org, dan.j.williams@intel•com,
	hch@infradead•org, djwong@kernel•org,
	dri-devel@lists•freedesktop.org, rodrigosiqueiramelo@gmail•com,
	melissa.srw@gmail•com, hamohammed.sa@gmail•com,
	harry.yoo@oracle•com, chris.p.wilson@intel•com,
	gwan-gyeong.mun@intel•com, max.byungchul.park@gmail•com,
	boqun.feng@gmail•com, longman@redhat•com,
	yunseong.kim@ericsson•com, ysk@kzalloc•com, yeoreum.yun@arm•com,
	netdev@vger•kernel.org, matthew.brost@intel•com,
	her0gyugyu@gmail•com, corbet@lwn•net, catalin.marinas@arm•com,
	bp@alien8•de, x86@kernel•org, hpa@zytor•com, luto@kernel•org,
	sumit.semwal@linaro•org, gustavo@padovan•org,
	christian.koenig@amd•com, andi.shyti@kernel•org, arnd@arndb•de,
	lorenzo.stoakes@oracle•com, Liam.Howlett@oracle•com,
	rppt@kernel•org, surenb@google•com, mcgrof@kernel•org,
	petr.pavlu@suse•com, da.gomez@kernel•org,
	samitolvanen@google•com, paulmck@kernel•org, frederic@kernel•org,
	neeraj.upadhyay@kernel•org, joelagnelf@nvidia•com,
	josh@joshtriplett•org, urezki@gmail•com,
	mathieu.desnoyers@efficios•com, jiangshanlai@gmail•com,
	qiang.zhang@linux•dev, juri.lelli@redhat•com,
	vincent.guittot@linaro•org, dietmar.eggemann@arm•com,
	bsegall@google•com, mgorman@suse•de, vschneid@redhat•com,
	chuck.lever@oracle•com, neil@brown•name, okorniev@redhat•com,
	Dai.Ngo@oracle•com, tom@talpey•com, trondmy@kernel•org,
	anna@kernel•org, kees@kernel•org, bigeasy@linutronix•de,
	clrkwllms@kernel•org, mark.rutland@arm•com,
	ada.coupriediaz@arm•com, kristina.martsenko@arm•com,
	wangkefeng.wang@huawei•com, broonie@kernel•org,
	kevin.brodsky@arm•com, dwmw@amazon•co.uk, shakeel.butt@linux•dev,
	ast@kernel•org, ziy@nvidia•com, yuzhao@google•com,
	baolin.wang@linux•alibaba.com, usamaarif642@gmail•com,
	joel.granados@kernel•org, richard.weiyang@gmail•com,
	geert+renesas@glider•be, tim.c.chen@linux•intel.com,
	linux@treblig•org, alexander.shishkin@linux•intel.com,
	lillian@star-ark•net, chenhuacai@kernel•org, francesco@valla•it,
	guoweikang.kernel@gmail•com, link@vivo•com, jpoimboe@kernel•org,
	masahiroy@kernel•org, brauner@kernel•org,
	thomas.weissschuh@linutronix•de, oleg@redhat•com,
	mjguzik@gmail•com, andrii@kernel•org, wangfushuai@baidu•com,
	linux-doc@vger•kernel.org, linux-arm-kernel@lists•infradead.org,
	linux-media@vger•kernel.org, linaro-mm-sig@lists•linaro.org,
	linux-i2c@vger•kernel.org, linux-arch@vger•kernel.org,
	linux-modules@vger•kernel.org, rcu@vger•kernel.org,
	linux-nfs@vger•kernel.org, linux-rt-devel@lists•linux.dev,
	2407018371@qq•com, dakr@kernel•org,
	miguel.ojeda.sandonis@gmail•com, neilb@ownmail•net,
	wsa+renesas@sang-engineering•com, dave.hansen@intel•com,
	geert@linux-m68k•org, ojeda@kernel•org, alex.gaynor@gmail•com,
	gary@garyguo•net, bjorn3_gh@protonmail•com, lossin@kernel•org,
	a.hindborg@kernel•org, aliceryhl@google•com, tmgross@umich•edu,
	rust-for-linux@vger•kernel.org
Subject: Re: [PATCH v18 25/42] dept: add documents for dept
Date: Mon, 15 Dec 2025 13:22:37 +0900	[thread overview]
Message-ID: <20251215042237.GA49936@system.software.com> (raw)
In-Reply-To: <aTN38kJjBftxnjm9@archie.me>

On Sat, Dec 06, 2025 at 07:25:22AM +0700, Bagas Sanjaya wrote:
> On Fri, Dec 05, 2025 at 04:18:38PM +0900, Byungchul Park wrote:
> > Add documents describing the concept and APIs of dept.
> > 
> > Signed-off-by: Byungchul Park <byungchul@sk•com>
> > ---
> >  Documentation/dev-tools/dept.rst     | 778 +++++++++++++++++++++++++++
> >  Documentation/dev-tools/dept_api.rst | 125 +++++
> 
> You forget to add toctree entries:

I'm sorry for late reply.

Thanks a lot!

> ---- >8 ----
> diff --git a/Documentation/dev-tools/index.rst b/Documentation/dev-tools/index.rst
> index 4b8425e348abd1..02c858f5ed1fa2 100644
> --- a/Documentation/dev-tools/index.rst
> +++ b/Documentation/dev-tools/index.rst
> @@ -22,6 +22,8 @@ Documentation/process/debugging/index.rst
>     clang-format
>     coccinelle
>     sparse
> +   dept
> +   dept_api
>     kcov
>     gcov
>     kasan
> 
> > +Lockdep detects a deadlock by checking lock acquisition order.  For
> > +example, a graph to track acquisition order built by lockdep might look
> > +like:
> > +
> > +.. literal::
> > +
> > +   A -> B -
> > +           \
> > +            -> E
> > +           /
> > +   C -> D -
> > +
> > +   where 'A -> B' means that acquisition A is prior to acquisition B
> > +   with A still held.
> 
> Use code-block directive for literal code blocks:

I will.

> ---- >8 ----
> diff --git a/Documentation/dev-tools/dept.rst b/Documentation/dev-tools/dept.rst
> index 333166464543d7..8394c4ea81bc2a 100644
> --- a/Documentation/dev-tools/dept.rst
> +++ b/Documentation/dev-tools/dept.rst
> @@ -10,7 +10,7 @@ Lockdep detects a deadlock by checking lock acquisition order.  For
>  example, a graph to track acquisition order built by lockdep might look
>  like:
>  
> -.. literal::
> +.. code-block::
>  
>     A -> B -
>             \
> @@ -25,7 +25,7 @@ Lockdep keeps adding each new acquisition order into the graph at
>  runtime.  For example, 'E -> C' will be added when the two locks have
>  been acquired in the order, E and then C.  The graph will look like:
>  
> -.. literal::
> +.. code-block::
>  
>         A -> B -
>                 \
> @@ -41,7 +41,7 @@ been acquired in the order, E and then C.  The graph will look like:
>  
>  This graph contains a subgraph that demonstrates a loop like:
>  
> -.. literal::
> +.. code-block::
>  
>                  -> E -
>                 /      \
> @@ -76,7 +76,7 @@ e.g. irq context, normal process context, wq worker context, or so on.
>  
>  Can lockdep detect the following deadlock?
>  
> -.. literal::
> +.. code-block::
>  
>     context X	   context Y	   context Z
>  
> @@ -91,7 +91,7 @@ Can lockdep detect the following deadlock?
>  
>  No.  What about the following?
>  
> -.. literal::
> +.. code-block::
>  
>     context X		   context Y
>  
> @@ -116,7 +116,7 @@ What leads a deadlock
>  A deadlock occurs when one or multi contexts are waiting for events that
>  will never happen.  For example:
>  
> -.. literal::
> +.. code-block::
>  
>     context X	   context Y	   context Z
>  
> @@ -148,7 +148,7 @@ In terms of dependency:
>  
>  Dependency graph reflecting this example will look like:
>  
> -.. literal::
> +.. code-block::
>  
>      -> C -> A -> B -
>     /                \
> @@ -171,7 +171,7 @@ Introduce DEPT
>  DEPT(DEPendency Tracker) tracks wait and event instead of lock
>  acquisition order so as to recognize the following situation:
>  
> -.. literal::
> +.. code-block::
>  
>     context X	   context Y	   context Z
>  
> @@ -186,7 +186,7 @@ acquisition order so as to recognize the following situation:
>  and builds up a dependency graph at runtime that is similar to lockdep.
>  The graph might look like:
>  
> -.. literal::
> +.. code-block::
>  
>      -> C -> A -> B -
>     /                \
> @@ -199,7 +199,7 @@ DEPT keeps adding each new dependency into the graph at runtime.  For
>  example, 'B -> D' will be added when event D occurrence is a
>  prerequisite to reaching event B like:
>  
> -.. literal::
> +.. code-block::
>  
>     context W
>  
> @@ -211,7 +211,7 @@ prerequisite to reaching event B like:
>  
>  After the addition, the graph will look like:
>  
> -.. literal::
> +.. code-block::
>  
>                       -> D
>                      /
> @@ -236,7 +236,7 @@ How DEPT works
>  Let's take a look how DEPT works with the 1st example in the section
>  'Limitation of lockdep'.
>  
> -.. literal::
> +.. code-block::
>  
>     context X	   context Y	   context Z
>  
> @@ -256,7 +256,7 @@ event.
>  
>  Adding comments to describe DEPT's view in detail:
>  
> -.. literal::
> +.. code-block::
>  
>     context X	   context Y	   context Z
>  
> @@ -293,7 +293,7 @@ Adding comments to describe DEPT's view in detail:
>  
>  Let's build up dependency graph with this example.  Firstly, context X:
>  
> -.. literal::
> +.. code-block::
>  
>     context X
>  
> @@ -304,7 +304,7 @@ Let's build up dependency graph with this example.  Firstly, context X:
>  
>  There are no events to create dependency.  Next, context Y:
>  
> -.. literal::
> +.. code-block::
>  
>     context Y
>  
> @@ -332,7 +332,7 @@ event A cannot be triggered if wait B cannot be awakened by event B.
>  Therefore, we can say event A depends on event B, say, 'A -> B'.  The
>  graph will look like after adding the dependency:
>  
> -.. literal::
> +.. code-block::
>  
>     A -> B
>  
> @@ -340,7 +340,7 @@ graph will look like after adding the dependency:
>  
>  Lastly, context Z:
>  
> -.. literal::
> +.. code-block::
>  
>     context Z
>  
> @@ -362,7 +362,7 @@ triggered if wait A cannot be awakened by event A.  Therefore, we can
>  say event B depends on event A, say, 'B -> A'.  The graph will look like
>  after adding the dependency:
>  
> -.. literal::
> +.. code-block::
>  
>      -> A -> B -
>     /           \
> @@ -386,7 +386,7 @@ Interpret DEPT report
>  
>  The following is the same example in the section 'How DEPT works'.
>  
> -.. literal::
> +.. code-block::
>  
>     context X	   context Y	   context Z
>  
> @@ -425,7 +425,7 @@ We can simplify this by labeling each waiting point with [W], each
>  point where its event's context starts with [S] and each event with [E].
>  This example will look like after the labeling:
>  
> -.. literal::
> +.. code-block::
>  
>     context X	   context Y	   context Z
>  
> @@ -443,7 +443,7 @@ DEPT uses the symbols [W], [S] and [E] in its report as described above.
>  The following is an example reported by DEPT for a real problem in
>  practice.
>  
> -.. literal::
> +.. code-block::
>  
>     Link: https://lore.kernel.org/lkml/6383cde5-cf4b-facf-6e07-1378a485657d@I-love.SAKURA.ne.jp/#t
>     Link: https://lore.kernel.org/lkml/1674268856-31807-1-git-send-email-byungchul.park@lge.com/
> @@ -646,7 +646,7 @@ practice.
>  
>  Let's take a look at the summary that is the most important part.
>  
> -.. literal::
> +.. code-block::
>  
>     ---------------------------------------------------
>     summary
> @@ -669,7 +669,7 @@ Let's take a look at the summary that is the most important part.
>  
>  The summary shows the following scenario:
>  
> -.. literal::
> +.. code-block::
>  
>     context A	   context B	   context ?(unknown)
>  
> @@ -684,7 +684,7 @@ The summary shows the following scenario:
>  
>  Adding comments to describe DEPT's view in detail:
>  
> -.. literal::
> +.. code-block::
>  
>     context A	   context B	   context ?(unknown)
>  
> @@ -711,7 +711,7 @@ Adding comments to describe DEPT's view in detail:
>  
>  Let's build up dependency graph with this report. Firstly, context A:
>  
> -.. literal::
> +.. code-block::
>  
>     context A
>  
> @@ -735,7 +735,7 @@ unlock(&ni->ni_lock:0) depends on folio_unlock(&f1), say,
>  
>  The graph will look like after adding the dependency:
>  
> -.. literal::
> +.. code-block::
>  
>     unlock(&ni->ni_lock:0) -> folio_unlock(&f1)
>  
> @@ -743,7 +743,7 @@ The graph will look like after adding the dependency:
>  
>  Secondly, context B:
>  
> -.. literal::
> +.. code-block::
>  
>     context B
>  
> @@ -762,7 +762,7 @@ folio_unlock(&f1) depends on unlock(&ni->ni_lock:0), say,
>  
>  The graph will look like after adding the dependency:
>  
> -.. literal::
> +.. code-block::
>  
>      -> unlock(&ni->ni_lock:0) -> folio_unlock(&f1) -
>     /                                                \
> 
> > +Limitation of lockdep
> > +---------------------
> > +
> > +Lockdep deals with a deadlock by typical lock e.g. spinlock and mutex,
> > +that are supposed to be released within the acquisition context.
> > +However, when it comes to a deadlock by folio lock that is not supposed
> > +to be released within the acquisition context or other general
> > +synchronization mechanisms, lockdep doesn't work.
> > +
> > +NOTE:  In this document, 'context' refers to any type of unique context
> > +e.g. irq context, normal process context, wq worker context, or so on.
> > +
> > +Can lockdep detect the following deadlock?
> > +
> > +.. literal::
> > +
> > +   context X	   context Y	   context Z
> > +
> > +		   mutex_lock A
> > +   folio_lock B
> > +		   folio_lock B <- DEADLOCK
> > +				   mutex_lock A <- DEADLOCK
> > +				   folio_unlock B
> > +		   folio_unlock B
> > +		   mutex_unlock A
> > +				   mutex_unlock A
> > +
> > +No.  What about the following?
> > +
> > +.. literal::
> > +
> > +   context X		   context Y
> > +
> > +			   mutex_lock A
> > +   mutex_lock A <- DEADLOCK
> > +			   wait_for_complete B <- DEADLOCK
> > +   complete B
> > +			   mutex_unlock A
> > +   mutex_unlock A
> > +
> > +No.
> 
> One unanswered question from my v17 review [1]: You explain in "How DEPT works"
> section how DEPT detects deadlock in the first example (the former with three
> contexts). Can you do the same on the second example (the latter with two
> contexts)?

Did you mean to update the document with it?  I misunderstood what you
meant but sure I will update it as [1].

[1] https://lore.kernel.org/linux-doc/20251013012855.GB52546@system.software.com/

Thanks.

	Byungchul

> Thanks.
> 
> [1]: https://lore.kernel.org/linux-doc/aN84jKyrE1BumpLj@archie.me/
> 
> -- 
> An old man doll... just what I always wanted! - Clara



  reply	other threads:[~2025-12-15  4:38 UTC|newest]

Thread overview: 62+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-12-05  7:18 [PATCH v18 00/42] DEPT(DEPendency Tracker) Byungchul Park
2025-12-05  7:18 ` [PATCH v18 01/42] dept: implement " Byungchul Park
2025-12-05  7:18 ` [PATCH v18 02/42] dept: add single event dependency tracker APIs Byungchul Park
2025-12-05  7:18 ` [PATCH v18 03/42] dept: add lock " Byungchul Park
2025-12-05  7:18 ` [PATCH v18 04/42] dept: tie to lockdep and IRQ tracing Byungchul Park
2025-12-05  7:18 ` [PATCH v18 05/42] dept: add proc knobs to show stats and dependency graph Byungchul Park
2025-12-05  7:18 ` [PATCH v18 06/42] dept: distinguish each kernel context from another Byungchul Park
2025-12-05  7:18 ` [PATCH v18 07/42] dept: distinguish each work " Byungchul Park
2025-12-05  7:18 ` [PATCH v18 08/42] dept: add a mechanism to refill the internal memory pools on running out Byungchul Park
2025-12-05  7:18 ` [PATCH v18 09/42] dept: record the latest one out of consecutive waits of the same class Byungchul Park
2025-12-05  7:18 ` [PATCH v18 10/42] dept: apply sdt_might_sleep_{start,end}() to wait_for_completion()/complete() Byungchul Park
2025-12-05  7:18 ` [PATCH v18 11/42] dept: apply sdt_might_sleep_{start,end}() to swait Byungchul Park
2025-12-05  7:18 ` [PATCH v18 12/42] dept: apply sdt_might_sleep_{start,end}() to waitqueue wait Byungchul Park
2025-12-05  7:18 ` [PATCH v18 13/42] dept: apply sdt_might_sleep_{start,end}() to hashed-waitqueue wait Byungchul Park
2025-12-05  7:18 ` [PATCH v18 14/42] dept: apply sdt_might_sleep_{start,end}() to dma fence Byungchul Park
2025-12-05  7:18 ` [PATCH v18 15/42] dept: track timeout waits separately with a new Kconfig Byungchul Park
2025-12-05  7:18 ` [PATCH v18 16/42] dept: apply timeout consideration to wait_for_completion()/complete() Byungchul Park
2025-12-05  7:18 ` [PATCH v18 17/42] dept: apply timeout consideration to swait Byungchul Park
2025-12-05  7:18 ` [PATCH v18 18/42] dept: apply timeout consideration to waitqueue wait Byungchul Park
2025-12-05  7:18 ` [PATCH v18 19/42] dept: apply timeout consideration to hashed-waitqueue wait Byungchul Park
2025-12-05  7:18 ` [PATCH v18 20/42] dept: apply timeout consideration to dma fence wait Byungchul Park
2025-12-05  7:18 ` [PATCH v18 21/42] dept: make dept able to work with an external wgen Byungchul Park
2025-12-05  7:18 ` [PATCH v18 22/42] dept: track PG_locked with dept Byungchul Park
2025-12-05  7:18 ` [PATCH v18 23/42] dept: print staged wait's stacktrace on report Byungchul Park
2025-12-05  7:18 ` [PATCH v18 24/42] locking/lockdep: prevent various lockdep assertions when lockdep_off()'ed Byungchul Park
2025-12-05  7:18 ` [PATCH v18 25/42] dept: add documents for dept Byungchul Park
2025-12-06  0:25   ` Bagas Sanjaya
2025-12-15  4:22     ` Byungchul Park [this message]
2025-12-15  9:06       ` Bagas Sanjaya
2026-04-28 16:26     ` [PATCH] dept: update documentation function names to match implementation Yunseong Kim
2026-05-06  6:27       ` Byungchul Park
2025-12-05  7:18 ` [PATCH v18 26/42] cpu/hotplug: use a weaker annotation in AP thread Byungchul Park
2025-12-05  7:18 ` [PATCH v18 27/42] dept: assign dept map to mmu notifier invalidation synchronization Byungchul Park
2025-12-05  7:18 ` [PATCH v18 28/42] dept: assign unique dept_key to each distinct dma fence caller Byungchul Park
2025-12-05  7:18 ` [PATCH v18 29/42] dept: make dept aware of lockdep_set_lock_cmp_fn() annotation Byungchul Park
2025-12-05  7:18 ` [PATCH v18 30/42] dept: make dept stop from working on debug_locks_off() Byungchul Park
2025-12-05  7:18 ` [PATCH v18 31/42] dept: assign unique dept_key to each distinct wait_for_completion() caller Byungchul Park
2026-02-15  6:42   ` Dirk Behme
2026-02-23  0:32     ` Byungchul Park
2025-12-05  7:18 ` [PATCH v18 32/42] completion, dept: introduce init_completion_dmap() API Byungchul Park
2025-12-05  7:18 ` [PATCH v18 33/42] dept: introduce a new type of dependency tracking between multi event sites Byungchul Park
2025-12-05  7:18 ` [PATCH v18 34/42] dept: add module support for struct dept_event_site and dept_event_site_dep Byungchul Park
2026-01-07 12:19   ` Petr Pavlu
2026-02-13  5:50     ` Byungchul Park
2026-02-18 15:08       ` Petr Pavlu
2026-02-23  0:33         ` Byungchul Park
2025-12-05  7:18 ` [PATCH v18 35/42] dept: introduce event_site() to disable event tracking if it's recoverable Byungchul Park
2025-12-05  7:18 ` [PATCH v18 36/42] dept: implement a basic unit test for dept Byungchul Park
2025-12-05  7:18 ` [PATCH v18 37/42] dept: call dept_hardirqs_off() in local_irq_*() regardless of irq state Byungchul Park
2025-12-05  7:18 ` [PATCH v18 38/42] rcu/update: fix same dept key collision between various types of RCU Byungchul Park
2025-12-05  7:18 ` [PATCH v18 39/42] dept: introduce APIs to set page usage and use subclasses_evt for the usage Byungchul Park
2025-12-05  7:18 ` [PATCH v18 40/42] dept: track PG_writeback with dept Byungchul Park
2025-12-05  7:18 ` [PATCH v18 41/42] SUNRPC: relocate struct rcu_head to the first field of struct rpc_xprt Byungchul Park
2025-12-05  9:27   ` Jeff Layton
2025-12-15  5:15     ` Byungchul Park
2025-12-05  7:18 ` [PATCH v18 42/42] mm: percpu: increase PERCPU_DYNAMIC_SIZE_SHIFT on DEPT and large PAGE_SIZE Byungchul Park
2026-03-17  4:44 ` [PATCH v18 00/42] DEPT(DEPendency Tracker) Byungchul Park
2026-03-18  5:58   ` Yunseong Kim
2026-05-23 12:32 ` DEPT (the dependency tracker) as AI review prompt? (was: DEPT v18) Harry Yoo
2026-05-23 14:00   ` Yunseong Kim
2026-05-23 14:34     ` DEPT (the dependency tracker) as AI review prompt? Harry Yoo
2026-05-23 15:04       ` Yunseong Kim

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20251215042237.GA49936@system.software.com \
    --to=byungchul@sk$(echo .)com \
    --cc=2407018371@qq$(echo .)com \
    --cc=Dai.Ngo@oracle$(echo .)com \
    --cc=Liam.Howlett@oracle$(echo .)com \
    --cc=a.hindborg@kernel$(echo .)org \
    --cc=ada.coupriediaz@arm$(echo .)com \
    --cc=adilger.kernel@dilger$(echo .)ca \
    --cc=akpm@linux-foundation$(echo .)org \
    --cc=alex.gaynor@gmail$(echo .)com \
    --cc=alexander.shishkin@linux$(echo .)intel.com \
    --cc=aliceryhl@google$(echo .)com \
    --cc=amir73il@gmail$(echo .)com \
    --cc=andi.shyti@kernel$(echo .)org \
    --cc=andrii@kernel$(echo .)org \
    --cc=anna@kernel$(echo .)org \
    --cc=arnd@arndb$(echo .)de \
    --cc=ast@kernel$(echo .)org \
    --cc=bagasdotme@gmail$(echo .)com \
    --cc=baolin.wang@linux$(echo .)alibaba.com \
    --cc=bigeasy@linutronix$(echo .)de \
    --cc=bjorn3_gh@protonmail$(echo .)com \
    --cc=boqun.feng@gmail$(echo .)com \
    --cc=bp@alien8$(echo .)de \
    --cc=brauner@kernel$(echo .)org \
    --cc=broonie@kernel$(echo .)org \
    --cc=bsegall@google$(echo .)com \
    --cc=catalin.marinas@arm$(echo .)com \
    --cc=chenhuacai@kernel$(echo .)org \
    --cc=chris.p.wilson@intel$(echo .)com \
    --cc=christian.koenig@amd$(echo .)com \
    --cc=chuck.lever@oracle$(echo .)com \
    --cc=cl@linux$(echo .)com \
    --cc=clrkwllms@kernel$(echo .)org \
    --cc=corbet@lwn$(echo .)net \
    --cc=da.gomez@kernel$(echo .)org \
    --cc=dakr@kernel$(echo .)org \
    --cc=damien.lemoal@opensource$(echo .)wdc.com \
    --cc=dan.j.williams@intel$(echo .)com \
    --cc=daniel.vetter@ffwll$(echo .)ch \
    --cc=dave.hansen@intel$(echo .)com \
    --cc=david@fromorbit$(echo .)com \
    --cc=dennis@kernel$(echo .)org \
    --cc=dietmar.eggemann@arm$(echo .)com \
    --cc=djwong@kernel$(echo .)org \
    --cc=dri-devel@lists$(echo .)freedesktop.org \
    --cc=duyuyang@gmail$(echo .)com \
    --cc=dwmw@amazon$(echo .)co.uk \
    --cc=francesco@valla$(echo .)it \
    --cc=frederic@kernel$(echo .)org \
    --cc=gary@garyguo$(echo .)net \
    --cc=geert+renesas@glider$(echo .)be \
    --cc=geert@linux-m68k$(echo .)org \
    --cc=gregkh@linuxfoundation$(echo .)org \
    --cc=guoweikang.kernel@gmail$(echo .)com \
    --cc=gustavo@padovan$(echo .)org \
    --cc=gwan-gyeong.mun@intel$(echo .)com \
    --cc=hamohammed.sa@gmail$(echo .)com \
    --cc=hannes@cmpxchg$(echo .)org \
    --cc=harry.yoo@oracle$(echo .)com \
    --cc=hch@infradead$(echo .)org \
    --cc=her0gyugyu@gmail$(echo .)com \
    --cc=hpa@zytor$(echo .)com \
    --cc=jack@suse$(echo .)cz \
    --cc=jglisse@redhat$(echo .)com \
    --cc=jiangshanlai@gmail$(echo .)com \
    --cc=jlayton@kernel$(echo .)org \
    --cc=joel.granados@kernel$(echo .)org \
    --cc=joel@joelfernandes$(echo .)org \
    --cc=joelagnelf@nvidia$(echo .)com \
    --cc=johannes.berg@intel$(echo .)com \
    --cc=josef@toxicpanda$(echo .)com \
    --cc=josh@joshtriplett$(echo .)org \
    --cc=jpoimboe@kernel$(echo .)org \
    --cc=juri.lelli@redhat$(echo .)com \
    --cc=kees@kernel$(echo .)org \
    --cc=kernel-team@lge$(echo .)com \
    --cc=kernel_team@skhynix$(echo .)com \
    --cc=kevin.brodsky@arm$(echo .)com \
    --cc=kristina.martsenko@arm$(echo .)com \
    --cc=lillian@star-ark$(echo .)net \
    --cc=linaro-mm-sig@lists$(echo .)linaro.org \
    --cc=link@vivo$(echo .)com \
    --cc=linux-arch@vger$(echo .)kernel.org \
    --cc=linux-arm-kernel@lists$(echo .)infradead.org \
    --cc=linux-block@vger$(echo .)kernel.org \
    --cc=linux-doc@vger$(echo .)kernel.org \
    --cc=linux-ext4@vger$(echo .)kernel.org \
    --cc=linux-fsdevel@vger$(echo .)kernel.org \
    --cc=linux-i2c@vger$(echo .)kernel.org \
    --cc=linux-ide@vger$(echo .)kernel.org \
    --cc=linux-kernel@vger$(echo .)kernel.org \
    --cc=linux-media@vger$(echo .)kernel.org \
    --cc=linux-mm@kvack$(echo .)org \
    --cc=linux-modules@vger$(echo .)kernel.org \
    --cc=linux-nfs@vger$(echo .)kernel.org \
    --cc=linux-rt-devel@lists$(echo .)linux.dev \
    --cc=linux@treblig$(echo .)org \
    --cc=longman@redhat$(echo .)com \
    --cc=lorenzo.stoakes@oracle$(echo .)com \
    --cc=lossin@kernel$(echo .)org \
    --cc=luto@kernel$(echo .)org \
    --cc=mark.rutland@arm$(echo .)com \
    --cc=masahiroy@kernel$(echo .)org \
    --cc=mathieu.desnoyers@efficios$(echo .)com \
    --cc=matthew.brost@intel$(echo .)com \
    --cc=max.byungchul.park@gmail$(echo .)com \
    --cc=mcgrof@kernel$(echo .)org \
    --cc=melissa.srw@gmail$(echo .)com \
    --cc=mgorman@suse$(echo .)de \
    --cc=mhocko@kernel$(echo .)org \
    --cc=miguel.ojeda.sandonis@gmail$(echo .)com \
    --cc=minchan@kernel$(echo .)org \
    --cc=mingo@redhat$(echo .)com \
    --cc=mjguzik@gmail$(echo .)com \
    --cc=neeraj.upadhyay@kernel$(echo .)org \
    --cc=neil@brown$(echo .)name \
    --cc=neilb@ownmail$(echo .)net \
    --cc=netdev@vger$(echo .)kernel.org \
    --cc=ngupta@vflare$(echo .)org \
    --cc=ojeda@kernel$(echo .)org \
    --cc=okorniev@redhat$(echo .)com \
    --cc=oleg@redhat$(echo .)com \
    --cc=paulmck@kernel$(echo .)org \
    --cc=penberg@kernel$(echo .)org \
    --cc=peterz@infradead$(echo .)org \
    --cc=petr.pavlu@suse$(echo .)com \
    --cc=qiang.zhang@linux$(echo .)dev \
    --cc=rcu@vger$(echo .)kernel.org \
    --cc=richard.weiyang@gmail$(echo .)com \
    --cc=rientjes@google$(echo .)com \
    --cc=rodrigosiqueiramelo@gmail$(echo .)com \
    --cc=rostedt@goodmis$(echo .)org \
    --cc=rppt@kernel$(echo .)org \
    --cc=rust-for-linux@vger$(echo .)kernel.org \
    --cc=samitolvanen@google$(echo .)com \
    --cc=sashal@kernel$(echo .)org \
    --cc=shakeel.butt@linux$(echo .)dev \
    --cc=sj@kernel$(echo .)org \
    --cc=sumit.semwal@linaro$(echo .)org \
    --cc=surenb@google$(echo .)com \
    --cc=tglx@linutronix$(echo .)de \
    --cc=thomas.weissschuh@linutronix$(echo .)de \
    --cc=tim.c.chen@linux$(echo .)intel.com \
    --cc=tj@kernel$(echo .)org \
    --cc=tmgross@umich$(echo .)edu \
    --cc=tom@talpey$(echo .)com \
    --cc=torvalds@linux-foundation$(echo .)org \
    --cc=trondmy@kernel$(echo .)org \
    --cc=tytso@mit$(echo .)edu \
    --cc=urezki@gmail$(echo .)com \
    --cc=usamaarif642@gmail$(echo .)com \
    --cc=vbabka@suse$(echo .)cz \
    --cc=vdavydov.dev@gmail$(echo .)com \
    --cc=vincent.guittot@linaro$(echo .)org \
    --cc=vschneid@redhat$(echo .)com \
    --cc=wangfushuai@baidu$(echo .)com \
    --cc=wangkefeng.wang@huawei$(echo .)com \
    --cc=will@kernel$(echo .)org \
    --cc=willy@infradead$(echo .)org \
    --cc=wsa+renesas@sang-engineering$(echo .)com \
    --cc=x86@kernel$(echo .)org \
    --cc=yeoreum.yun@arm$(echo .)com \
    --cc=ysk@kzalloc$(echo .)com \
    --cc=yunseong.kim@ericsson$(echo .)com \
    --cc=yuzhao@google$(echo .)com \
    --cc=ziy@nvidia$(echo .)com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox