Subject: Re: [PATCH] Makefile: fix bugs in coccicheck and speed it up

From: Ævar Arnfjörð Bjarmason <[email-masked]>
Date: Wed, 03 Mar 2021 12:45:20 +0100

On Tue, Mar 02 2021, Ævar Arnfjörð Bjarmason wrote:

A few notes, for a probable re-roll sometime later:

> I've often wondered why "make coccicheck" takes so long. This change
> speeds it up by ~2x and makes it use much less memory. Or a reduction
> of a max of around ~2GB per-process (under the old
> SPATCH_BATCH_SIZE=0) to around ~200MB.

I've also tested this with e.g. "strace -c -f -e trace=openat ",
before/after with just contrib/coccinelle/swap.cocci.patch:

    192612
    13106

Grepping e.g. for strbuf.h shows that when running "make coccicheck" we
open it 12 times as expected (we have 12 rules)

    strace -f -e trace=openat make -j8 coccicheck 2>&1|grep 'openat.*"strbuf\.h"' | wc -l
    12

(Is there a more direct way to filter this type of thing with strace?)

Before:

    $ strace -f -e trace=openat make -j8 coccicheck 2>&1|grep 'openat.*"strbuf\.h"' | wc -l
    137

> [...]
> I'm also the whole "cat $@.log" introduced in [3]. Since we don't call
> this in a loop anymore (and xargs will early-exit) we can just rely on
> standard V=1 for debugging issues.

Noticed by Denton in the side-thread, should be:

    I'm also removing the whole "cat $@.log" feature introduced in...

> +		xargs \
> +			-n 32 $(SPATCH_XARGS_FLAGS) \

As an aside this relies on xargs not barfing on "xargs -n 1 -n 2",
i.e. taking 2 over 1. I tested it on Linux/AIX/OpenBSD/Solaris, works
consistently on all of them.