From: Andy Shevchenko <andriy.shevchenko@linux•intel.com>
To: Arnd Bergmann <arnd@arndb•de>
Cc: "Alejandro Colomar (man-pages)" <alx.manpages@gmail•com>,
LKML <linux-kernel@vger•kernel.org>,
Ajit Khaparde <ajit.khaparde@broadcom•com>,
Andrew Morton <akpm@linux-foundation•org>,
Bjorn Andersson <bjorn.andersson@linaro•org>,
Borislav Petkov <bp@suse•de>, Corey Minyard <cminyard@mvista•com>,
Chris Mason <clm@fb•com>,
Christian Brauner <christian.brauner@ubuntu•com>,
David Sterba <dsterba@suse•com>,
Jani Nikula <jani.nikula@linux•intel.com>,
Jason Wang <jasowang@redhat•com>,
Jitendra Bhivare <jitendra.bhivare@broadcom•com>,
John Hubbard <jhubbard@nvidia•com>,
"John S . Gruber" <JohnSGruber@gmail•com>,
Jonathan Cameron <Jonathan.Cameron@huawei•com>,
Joonas Lahtinen <joonas.lahtinen@linux•intel.com>,
Josef Bacik <josef@toxicpanda•com>,
Kees Cook <keescook@chromium•org>,
Ketan Mukadam <ketan.mukadam@broadcom•com>,
Len Brown <lenb@kernel•org>,
"Michael S. Tsirkin" <mst@redhat•com>,
Miguel Ojeda <ojeda@kernel•org>,
Mike Rapoport <rppt@linux•ibm.com>,
Nick Desaulniers <ndesaulniers@google•com>,
"Rafael J. Wysocki" <rafael@kernel•org>,
Rasmus Villemoes <linux@rasmusvillemoes•dk>,
Rodrigo Vivi <rodrigo.vivi@intel•com>,
Russell King <linux@armlinux•org.uk>,
Somnath Kotur <somnath.kotur@broadcom•com>,
Sriharsha Basavapatna <sriharsha.basavapatna@broadcom•com>,
Subbu Seetharaman <subbu.seetharaman@broadcom•com>,
Intel Graphics <intel-gfx@lists•freedesktop.org>,
ACPI Devel Maling List <linux-acpi@vger•kernel.org>,
Linux ARM <linux-arm-kernel@lists•infradead.org>,
linux-btrfs <linux-btrfs@vger•kernel.org>,
linux-scsi <linux-scsi@vger•kernel.org>,
Networking <netdev@vger•kernel.org>,
"open list:DRM DRIVER FOR QEMU'S CIRRUS DEVICE"
<virtualization@lists•linux-foundation.org>
Subject: Re: [PATCH 00/17] Add memberof(), split some headers, and slightly simplify code
Date: Fri, 19 Nov 2021 18:10:06 +0200 [thread overview]
Message-ID: <YZfMXlqvG52ls2TE@smile.fi.intel.com> (raw)
In-Reply-To: <CAK8P3a2yVXw9gf8-BNvX_rzectNoiy0MqGKvBcXydiUSrc_fCA@mail.gmail.com>
On Fri, Nov 19, 2021 at 04:57:46PM +0100, Arnd Bergmann wrote:
> On Fri, Nov 19, 2021 at 4:06 PM Alejandro Colomar (man-pages)
> <alx.manpages@gmail•com> wrote:
> > On 11/19/21 15:47, Arnd Bergmann wrote:
> > > On Fri, Nov 19, 2021 at 12:36 PM Alejandro Colomar
> >
> > Yes, I would like to untangle the dependencies.
> >
> > The main reason I started doing this splitting
> > is because I wouldn't be able to include
> > <linux/stddef.h> in some headers,
> > because it pulled too much stuff that broke unrelated things.
> >
> > So that's why I started from there.
> >
> > I for example would like to get NULL in memberof()
> > without puling anything else,
> > so <linux/NULL.h> makes sense for that.
> >
> > It's clear that every .c wants NULL,
> > but it's not so clear that every .c wants
> > everything that <linux/stddef.h> pulls indirectly.
>
> From what I can tell, linux/stddef.h is tiny, I don't think it's really
> worth optimizing this part. I have spent some time last year
> trying to untangle some of the more interesting headers, but ended
> up not completing this as there are some really hard problems
> once you start getting to the interesting bits.
>
> The approach I tried was roughly:
>
> - For each header in the kernel, create a preprocessed version
> that includes all the indirect includes, from that start a set
> of lookup tables that record which header is eventually included
> by which ones, and the size of each preprocessed header in
> bytes
>
> - For a given kernel configuration (e.g. defconfig or allmodconfig)
> that I'm most interested in, look at which files are built, and what
> the direct includes are in the source files.
>
> - Sort the headers by the product of the number of direct includes
> and the preprocessed size: the largest ones are those that are
> worth looking at first.
>
> - use graphviz to visualize the directed graph showing the includes
> between the top 100 headers in that list. You get something like
> I had in [1], or the version afterwards at [2].
>
> - split out unneeded indirect includes from the headers in the center
> of that graph, typically by splitting out struct definitions.
>
> - repeat.
>
> The main problem with this approach is that as soon as you start
> actually reducing the unneeded indirect includes, you end up with
> countless .c files that no longer build because they are missing a
> direct include for something that was always included somewhere
> deep underneath, so I needed a second set of scripts to add
> direct includes to every .c file.
Can't it be done with cocci support?
> On the plus side, I did see something on the order of a 30%
> compile speed improvement with clang, which is insane
> given that this only removed dead definitions.
Thumb up!
> > But I'll note that linux/fs.h, linux/sched.h, linux/mm.h are
> > interesting headers for further splitting.
> >
> >
> > BTW, I also have a longstanding doubt about
> > how header files are organized in the kernel,
> > and which headers can and cannot be included
> > from which other files.
> >
> > For example I see that files in samples or scripts or tools,
> > that redefine many things such as offsetof() or ARRAY_SIZE(),
> > and I don't know if there's a good reason for that,
> > or if I should simply remove all that stuff and
> > include <linux/offsetof.h> everywhere I see offsetof() being used.
>
> The main issue here is that user space code should not
> include anything outside of include/uapi/ and arch/*/include/uapi/
>
> offsetof() is defined in include/linux/stddef.h, so this is by
> definition not accessible here. It appears that there is also
> an include/uapi/linux/stddef.h that is really strange because
> it includes linux/compiler_types.h, which in turn is outside
> of uapi/. This should probably be fixed.
>
> Arnd
>
> [1] https://drive.google.com/file/d/14IKifYDadg2W5fMsefxr4373jizo9bLl/view?usp=sharing
> [2] https://drive.google.com/file/d/1pWQcv3_ZXGqZB8ogV-JOfoV-WJN2UNnd/view?usp=sharing
--
With Best Regards,
Andy Shevchenko
next prev parent reply other threads:[~2021-11-19 16:10 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-11-19 11:36 [PATCH 00/17] Add memberof(), split some headers, and slightly simplify code Alejandro Colomar
2021-11-19 11:36 ` [PATCH 02/17] Use memberof(T, m) instead of explicit NULL dereference Alejandro Colomar
2021-11-23 18:07 ` Rafael J. Wysocki
2021-11-19 12:47 ` [PATCH 00/17] Add memberof(), split some headers, and slightly simplify code Jani Nikula
2021-11-19 13:16 ` Alejandro Colomar (man-pages)
2021-11-19 13:48 ` Jani Nikula
2021-11-19 14:54 ` Andy Shevchenko
2021-11-19 14:47 ` Arnd Bergmann
2021-11-19 15:06 ` Alejandro Colomar (man-pages)
2021-11-19 15:34 ` Andy Shevchenko
2021-11-19 15:38 ` Alejandro Colomar (man-pages)
2021-11-19 15:57 ` Arnd Bergmann
2021-11-19 16:10 ` Andy Shevchenko [this message]
2021-11-19 16:18 ` Arnd Bergmann
2021-11-19 16:22 ` Alejandro Colomar (man-pages)
2021-11-19 16:27 ` Arnd Bergmann
2021-11-19 16:35 ` Andy Shevchenko
2021-11-22 12:36 ` Jonathan Cameron
2021-11-19 16:12 ` Alejandro Colomar (man-pages)
2021-11-19 16:25 ` Arnd Bergmann
2021-11-19 16:37 ` Andy Shevchenko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YZfMXlqvG52ls2TE@smile.fi.intel.com \
--to=andriy.shevchenko@linux$(echo .)intel.com \
--cc=JohnSGruber@gmail$(echo .)com \
--cc=Jonathan.Cameron@huawei$(echo .)com \
--cc=ajit.khaparde@broadcom$(echo .)com \
--cc=akpm@linux-foundation$(echo .)org \
--cc=alx.manpages@gmail$(echo .)com \
--cc=arnd@arndb$(echo .)de \
--cc=bjorn.andersson@linaro$(echo .)org \
--cc=bp@suse$(echo .)de \
--cc=christian.brauner@ubuntu$(echo .)com \
--cc=clm@fb$(echo .)com \
--cc=cminyard@mvista$(echo .)com \
--cc=dsterba@suse$(echo .)com \
--cc=intel-gfx@lists$(echo .)freedesktop.org \
--cc=jani.nikula@linux$(echo .)intel.com \
--cc=jasowang@redhat$(echo .)com \
--cc=jhubbard@nvidia$(echo .)com \
--cc=jitendra.bhivare@broadcom$(echo .)com \
--cc=joonas.lahtinen@linux$(echo .)intel.com \
--cc=josef@toxicpanda$(echo .)com \
--cc=keescook@chromium$(echo .)org \
--cc=ketan.mukadam@broadcom$(echo .)com \
--cc=lenb@kernel$(echo .)org \
--cc=linux-acpi@vger$(echo .)kernel.org \
--cc=linux-arm-kernel@lists$(echo .)infradead.org \
--cc=linux-btrfs@vger$(echo .)kernel.org \
--cc=linux-kernel@vger$(echo .)kernel.org \
--cc=linux-scsi@vger$(echo .)kernel.org \
--cc=linux@armlinux$(echo .)org.uk \
--cc=linux@rasmusvillemoes$(echo .)dk \
--cc=mst@redhat$(echo .)com \
--cc=ndesaulniers@google$(echo .)com \
--cc=netdev@vger$(echo .)kernel.org \
--cc=ojeda@kernel$(echo .)org \
--cc=rafael@kernel$(echo .)org \
--cc=rodrigo.vivi@intel$(echo .)com \
--cc=rppt@linux$(echo .)ibm.com \
--cc=somnath.kotur@broadcom$(echo .)com \
--cc=sriharsha.basavapatna@broadcom$(echo .)com \
--cc=subbu.seetharaman@broadcom$(echo .)com \
--cc=virtualization@lists$(echo .)linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox