Subject: Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format

From: Arnaldo Carvalho de Melo <[email-masked]>
Date: Thu, 7 Jun 2018 16:30:29 -0300

Em Thu, Jun 07, 2018 at 12:05:10PM -0700, Martin KaFai Lau escreveu:
> On Thu, Jun 07, 2018 at 11:03:37AM -0300, Arnaldo Carvalho de Melo wrote:
> > Em Thu, Jun 07, 2018 at 10:54:01AM -0300, Arnaldo Carvalho de Melo escreveu:
> > > Em Tue, Jun 05, 2018 at 02:25:48PM -0700, Martin KaFai Lau escreveu:
> > > > [ btw, the latest commit (1 commit) should be 94a11b59e592 ].

> > So, the commit log message for the pahole patch is non-existent:

> > https://github.com/iamkafai/pahole/commit/94a11b59e5920908085bfc8d24c92f95c8ffceaf

> > we should do better in describing what is done and how, I'm staring
> > with a message you sent to the kernel part:
> > --
> > This patch introduces BPF Type Format (BTF).

> > BTF (BPF Type Format) is the meta data format which describes
> > the data types of BPF program/map.  Hence, it basically focus
> > on the C programming language which the modern BPF is primary
> > using.  The first use case is to provide a generic pretty print
> > capability for a BPF map.

> I will add details in the next github respin/push.

Ok, but I can do that if there is nothing else to do in the code at this
stage :-)
 
> > Now I'm going to do the step-by-step guide on testing the feature just
> > introduced, and will try to convert from dwarf to BTF and back, compare
> > the pahole output for types encoded in DWARF and BTF, etc.

> > If you have something ressembling this already, please share.

> The pahole only has the encoder part.  I tested with the verbose output
> from the "pahole -V -J".  Loading the btf to the kernel is also tested.

Ok, so here it goes my first stab at testing, using perf's BPF
integration:

[root@jouet bpf]# cat hello.c
#include "stdio.h"

int syscall_enter(openat)(void *ctx)
{
	puts("Hello, world\n");
	return 0;
}
[root@jouet bpf]# cat ~/.perfconfig
[llvm]
dump-obj = true
[root@jouet bpf]# perf trace -e open*,hello.c touch /tmp/hello.BTF
LLVM: dumping hello.o
     0.017 (         ): __bpf_stdout__:Hello, world
     0.019 ( 0.011 ms): touch/28147 openat(dfd: CWD, filename: /etc/ld.so.cache, flags: CLOEXEC           ) = 3
     0.053 (         ): __bpf_stdout__:Hello, world
     0.055 ( 0.011 ms): touch/28147 openat(dfd: CWD, filename: /lib64/libc.so.6, flags: CLOEXEC           ) = 3
     0.354 ( 0.012 ms): touch/28147 open(filename: /usr/lib/locale/locale-archive, flags: CLOEXEC         ) = 3
     0.411 (         ): __bpf_stdout__:Hello, world
     0.412 ( 0.198 ms): touch/28147 openat(dfd: CWD, filename: /tmp/hello.BTF, flags: CREAT|NOCTTY|NONBLOCK|WRONLY, mode: IRUGO|IWUGO) = 3
[root@jouet bpf]# 
[root@jouet bpf]# file hello.o
hello.o: ELF 64-bit LSB relocatable, *unknown arch 0xf7* version 1 (SYSV), not stripped
[root@jouet bpf]# pahole --btf_encode hello.o
pahole: hello.o: No debugging information found
[root@jouet bpf]#

[root@jouet bpf]# readelf -s hello.o

Symbol table '.symtab' contains 5 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND 
     1: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT    7 __bpf_stdout__
     2: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT    5 _license
     3: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT    6 _version
     4: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT    3 syscall_enter_openat
[root@jouet bpf]#
[root@jouet bpf]# readelf -SW hello.o
There are 10 section headers, starting at offset 0x1f8:

Section Headers:
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
  [ 1] .strtab           STRTAB          0000000000000000 000178 00007f 00      0   0  1
  [ 2] .text             PROGBITS        0000000000000000 000040 000000 00  AX  0   0  4
  [ 3] syscalls:sys_enter_openat PROGBITS        0000000000000000 000040 000088 00  AX  0   0  8
  [ 4] .relsyscalls:sys_enter_openat REL             0000000000000000 000168 000010 10      9   3  8
  [ 5] license           PROGBITS        0000000000000000 0000c8 000004 00  WA  0   0  1
  [ 6] version           PROGBITS        0000000000000000 0000cc 000004 00  WA  0   0  4
  [ 7] maps              PROGBITS        0000000000000000 0000d0 000010 00  WA  0   0  4
  [ 8] .rodata.str1.1    PROGBITS        0000000000000000 0000e0 00000e 01 AMS  0   0  1
  [ 9] .symtab           SYMTAB          0000000000000000 0000f0 000078 18      1   1  8
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
  L (link order), O (extra OS processing required), G (group), T (TLS),
  C (compressed), x (unknown), o (OS specific), E (exclude),
  p (processor specific)
[root@jouet bpf]#

Humm, lemme try something, add -g to clang-opt:

[root@jouet bpf]# cat ~/.perfconfig
[llvm]
dump-obj = true
clang-opt = -g
[root@jouet bpf]# perf trace -e open*,hello.c touch /tmp/hello.BTF
LLVM: dumping hello.o
     0.185 (         ): __bpf_stdout__:Hello, world
     0.188 ( 0.009 ms): touch/19670 openat(dfd: CWD, filename: /etc/ld.so.cache, flags: CLOEXEC           ) = 3
     0.219 (         ): __bpf_stdout__:Hello, world
     0.220 ( 0.011 ms): touch/19670 openat(dfd: CWD, filename: /lib64/libc.so.6, flags: CLOEXEC           ) = 3
     0.480 ( 0.095 ms): touch/19670 open(filename: /usr/lib/locale/locale-archive, flags: CLOEXEC         ) = 3
     0.624 (         ): __bpf_stdout__:Hello, world
     0.626 ( 0.011 ms): touch/19670 openat(dfd: CWD, filename: /tmp/hello.BTF, flags: CREAT|NOCTTY|NONBLOCK|WRONLY, mode: IRUGO|IWUGO) = 3
[root@jouet bpf]# file hello.o
hello.o: ELF 64-bit LSB relocatable, *unknown arch 0xf7* version 1 (SYSV), with debug_info, not stripped
[root@jouet bpf]#

Much better:

[root@jouet bpf]# readelf -SW hello.o
There are 25 section headers, starting at offset 0xc70:

Section Headers:
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
  [ 1] .strtab           STRTAB          0000000000000000 000b50 000119 00      0   0  1
  [ 2] .text             PROGBITS        0000000000000000 000040 000000 00  AX  0   0  4
  [ 3] syscalls:sys_enter_openat PROGBITS        0000000000000000 000040 000088 00  AX  0   0  8
  [ 4] .relsyscalls:sys_enter_openat REL             0000000000000000 000910 000010 10     24   3  8
  [ 5] license           PROGBITS        0000000000000000 0000c8 000004 00  WA  0   0  1
  [ 6] version           PROGBITS        0000000000000000 0000cc 000004 00  WA  0   0  4
  [ 7] maps              PROGBITS        0000000000000000 0000d0 000010 00  WA  0   0  4
  [ 8] .rodata.str1.1    PROGBITS        0000000000000000 0000e0 00000e 01 AMS  0   0  1
  [ 9] .debug_str        PROGBITS        0000000000000000 0000ee 00010e 01  MS  0   0  1
  [10] .debug_loc        PROGBITS        0000000000000000 0001fc 000023 00      0   0  1
  [11] .debug_abbrev     PROGBITS        0000000000000000 00021f 0000e3 00      0   0  1
  [12] .debug_info       PROGBITS        0000000000000000 000302 00015e 00      0   0  1
  [13] .rel.debug_info   REL             0000000000000000 000920 0001e0 10     24  12  8
  [14] .debug_ranges     PROGBITS        0000000000000000 000460 000030 00      0   0  1
  [15] .debug_macinfo    PROGBITS        0000000000000000 000490 000001 00      0   0  1
  [16] .debug_pubnames   PROGBITS        0000000000000000 000491 00006e 00      0   0  1
  [17] .rel.debug_pubnames REL             0000000000000000 000b00 000010 10     24  16  8
  [18] .debug_pubtypes   PROGBITS        0000000000000000 0004ff 00005a 00      0   0  1
  [19] .rel.debug_pubtypes REL             0000000000000000 000b10 000010 10     24  18  8
  [20] .debug_frame      PROGBITS        0000000000000000 000560 000028 00      0   0  8
  [21] .rel.debug_frame  REL             0000000000000000 000b20 000020 10     24  20  8
  [22] .debug_line       PROGBITS        0000000000000000 000588 00006e 00      0   0  1
  [23] .rel.debug_line   REL             0000000000000000 000b40 000010 10     24  22  8
  [24] .symtab           SYMTAB          0000000000000000 0005f8 000318 18      1  29  8
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
  L (link order), O (extra OS processing required), G (group), T (TLS),
  C (compressed), x (unknown), o (OS specific), E (exclude),
  p (processor specific)
[root@jouet bpf]# 

[root@jouet bpf]# readelf -s hello.o

Symbol table '.symtab' contains 33 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND 
     1: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT    9 
     2: 000000000000002d     0 NOTYPE  LOCAL  DEFAULT    9 
     3: 0000000000000044     0 NOTYPE  LOCAL  DEFAULT    9 
     4: 0000000000000053     0 NOTYPE  LOCAL  DEFAULT    9 
     5: 000000000000005c     0 NOTYPE  LOCAL  DEFAULT    9 
     6: 0000000000000061     0 NOTYPE  LOCAL  DEFAULT    9 
     7: 000000000000006a     0 NOTYPE  LOCAL  DEFAULT    9 
     8: 0000000000000073     0 NOTYPE  LOCAL  DEFAULT    9 
     9: 0000000000000077     0 NOTYPE  LOCAL  DEFAULT    9 
    10: 0000000000000086     0 NOTYPE  LOCAL  DEFAULT    9 
    11: 000000000000008b     0 NOTYPE  LOCAL  DEFAULT    9 
    12: 0000000000000098     0 NOTYPE  LOCAL  DEFAULT    9 
    13: 00000000000000a1     0 NOTYPE  LOCAL  DEFAULT    9 
    14: 00000000000000ac     0 NOTYPE  LOCAL  DEFAULT    9 
    15: 00000000000000b8     0 NOTYPE  LOCAL  DEFAULT    9 
    16: 00000000000000c4     0 NOTYPE  LOCAL  DEFAULT    9 
    17: 00000000000000d6     0 NOTYPE  LOCAL  DEFAULT    9 
    18: 00000000000000e8     0 NOTYPE  LOCAL  DEFAULT    9 
    19: 00000000000000fd     0 NOTYPE  LOCAL  DEFAULT    9 
    20: 0000000000000101     0 NOTYPE  LOCAL  DEFAULT    9 
    21: 0000000000000107     0 NOTYPE  LOCAL  DEFAULT    9 
    22: 0000000000000000     0 SECTION LOCAL  DEFAULT    3 
    23: 0000000000000000     0 SECTION LOCAL  DEFAULT   10 
    24: 0000000000000000     0 SECTION LOCAL  DEFAULT   11 
    25: 0000000000000000     0 SECTION LOCAL  DEFAULT   12 
    26: 0000000000000000     0 SECTION LOCAL  DEFAULT   14 
    27: 0000000000000000     0 SECTION LOCAL  DEFAULT   20 
    28: 0000000000000000     0 SECTION LOCAL  DEFAULT   22 
    29: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT    7 __bpf_stdout__
    30: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT    5 _license
    31: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT    6 _version
    32: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT    3 syscall_enter_openat
[root@jouet bpf]# 

Progress:

[root@jouet bpf]# pahole --btf_encode hello.o
sh: llvm-objcopy: command not found
Failed to encode BTF
[root@jouet bpf]# 

[acme@jouet perf]$ llvm-
llvm-ar          llvm-cov         llvm-dis         llvm-lib         llvm-modextract  llvm-profdata    llvm-split
llvm-as          llvm-c-test      llvm-dlltool     llvm-link        llvm-mt          llvm-ranlib      llvm-stress
llvm-bcanalyzer  llvm-cvtres      llvm-dsymutil    llvm-lto         llvm-nm          llvm-readelf     llvm-strings
llvm-cat         llvm-cxxdump     llvm-dwarfdump   llvm-lto2        llvm-objdump     llvm-readobj     llvm-symbolizer
llvm-config      llvm-cxxfilt     llvm-dwp         llvm-mc          llvm-opt-report  llvm-rtdyld      llvm-tblgen
llvm-config-64   llvm-diff        llvm-extract     llvm-mcmarkup    llvm-pdbutil     llvm-size        llvm-xray
[acme@jouet perf]$ llvm-

So this must be available in a newer llvm version? Which one?

[root@jouet bpf]# clang -v
clang version 5.0.1 (tags/RELEASE_501/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/7
Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/7
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/7
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64
[root@jouet bpf]#

[acme@jouet perf]$ rpm -q clang llvm
clang-5.0.1-5.fc27.x86_64
llvm-5.0.1-6.fc27.x86_64
[acme@jouet perf]$ 
[root@jouet bpf]# clang -v
clang version 5.0.1 (tags/RELEASE_501/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/7
Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/7
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/7
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64
[root@jouet bpf]#

[acme@jouet perf]$ rpm -q clang llvm
clang-5.0.1-5.fc27.x86_64
llvm-5.0.1-6.fc27.x86_64
[acme@jouet perf]$ 

- Arnaldo