Subject: [linux-next:master] [splice, net] 2dc334f1a6: netperf.Throughput_Mbps 29.9% improvement

From: kernel test robot <[email-masked]>
Date: Tue, 20 Jun 2023 10:29:50 +0800

Hello,

kernel test robot noticed a 29.9% improvement of netperf.Throughput_Mbps on:

commit: 2dc334f1a63a8839b88483a3e73c0f27c9c1791c ("splice, net: Use sendmsg(MSG_SPLICE_PAGES) rather than ->sendpage()")
https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master

testcase: netperf
test machine: 128 threads 4 sockets Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz (Ice Lake) with 256G memory
parameters:

	ip: ipv4
	runtime: 300s
	nr_threads: 200%
	cluster: cs-localhost
	send_size: 10K
	test: TCP_SENDFILE
	cpufreq_governor: performance

Details are as below:
-------------------------------------------------------------------------------------------------->

To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        sudo bin/lkp install job.yaml           # job file is attached in this email
        bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run
        sudo bin/lkp run generated-yaml-file

        # if come across any failure that blocks the test,
        # please remove ~/.lkp and /lkp dir to run from a clean state.

=========================================================================================
cluster/compiler/cpufreq_governor/ip/kconfig/nr_threads/rootfs/runtime/send_size/tbox_group/test/testcase:
  cs-localhost/gcc-12/performance/ipv4/x86_64-rhel-8.3/200%/debian-11.1-x86_64-20220510.cgz/300s/10K/lkp-icl-2sp2/TCP_SENDFILE/netperf

commit: 
  81840b3b91 ("tls: Allow MSG_SPLICE_PAGES but treat it as normal sendmsg")
  2dc334f1a6 ("splice, net: Use sendmsg(MSG_SPLICE_PAGES) rather than ->sendpage()")

81840b3b91aad060 2dc334f1a63a8839b88483a3e73 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
   5702242 ±  6%     -15.6%    4814280 ±  4%  cpuidle..usage
      1180 ± 15%     -45.2%     646.67 ± 45%  perf-c2c.HITM.local
  16580465            +3.3%   17124667        vmstat.system.cs
    220034            -9.8%     198521        vmstat.system.in
  -2613383           +86.0%   -4860317        sched_debug.cfs_rq:/.spread0.avg
     88.31 ± 24%     +70.3%     150.36 ±  8%  sched_debug.cfs_rq:/.util_est_enqueued.min
      4501 ±  4%     +13.3%       5101 ±  4%  sched_debug.cpu.avg_idle.min
   5657476 ±  6%     -15.7%    4769566 ±  4%  turbostat.C1
  67713300            -9.9%   61042056        turbostat.IRQ
     11318 ±  6%     -19.3%       9130 ±  5%  turbostat.POLL
     67414 ±  3%      -9.6%      60931 ±  8%  proc-vmstat.nr_active_anon
     67414 ±  3%      -9.6%      60931 ±  8%  proc-vmstat.nr_zone_active_anon
     39564 ± 22%     -21.8%      30940 ±  3%  proc-vmstat.numa_pages_migrated
     39564 ± 22%     -21.8%      30940 ±  3%  proc-vmstat.pgmigrate_success
      5026           +29.9%       6530        netperf.Throughput_Mbps
   1286708           +29.9%    1671781        netperf.Throughput_total_Mbps
 2.541e+09            +3.3%  2.626e+09        netperf.time.involuntary_context_switches
      7339            -3.2%       7105        netperf.time.percent_of_cpu_this_job_got
     21534            -3.7%      20739        netperf.time.system_time
    817.24           +10.6%     903.64        netperf.time.user_time
      7000 ± 10%     +64.1%      11486 ±  9%  netperf.time.voluntary_context_switches
 4.712e+09           +29.9%  6.122e+09        netperf.workload
      0.99 ±  2%     -37.8%       0.62 ±  7%  perf-stat.i.MPKI
 5.941e+10            -1.4%  5.856e+10        perf-stat.i.branch-instructions
      1.03            -0.1        0.92        perf-stat.i.branch-miss-rate%
   6.1e+08           -12.1%  5.361e+08        perf-stat.i.branch-misses
     10.08 ±  3%      +7.0       17.03 ±  5%  perf-stat.i.cache-miss-rate%
 3.046e+08 ±  3%     -39.6%   1.84e+08 ±  7%  perf-stat.i.cache-references
  16798255            +3.3%   17348981        perf-stat.i.context-switches
      1.04            +1.8%       1.06        perf-stat.i.cpi
      1831 ±  3%     -12.2%       1608 ±  2%  perf-stat.i.cpu-migrations
     12505 ±  3%      +9.8%      13733 ±  7%  perf-stat.i.cycles-between-cache-misses
 8.983e+10            -2.4%   8.77e+10        perf-stat.i.dTLB-loads
      0.01 ± 10%      +0.0        0.01 ±  2%  perf-stat.i.dTLB-store-miss-rate%
   4562369 ± 10%     +23.5%    5633059 ±  2%  perf-stat.i.dTLB-store-misses
 5.131e+10            -3.2%  4.969e+10        perf-stat.i.dTLB-stores
 3.139e+11            -1.5%  3.092e+11        perf-stat.i.instructions
      0.97            -1.7%       0.95        perf-stat.i.ipc
    249.05          +186.8%     714.20 ±  3%  perf-stat.i.metric.K/sec
      1568            -2.4%       1531        perf-stat.i.metric.M/sec
      0.97 ±  3%     -38.7%       0.60 ±  7%  perf-stat.overall.MPKI
      1.03            -0.1        0.92        perf-stat.overall.branch-miss-rate%
      8.89 ±  3%      +4.6       13.46 ±  3%  perf-stat.overall.cache-miss-rate%
      1.04            +1.7%       1.06        perf-stat.overall.cpi
     12031 ±  3%     +10.2%      13256 ±  7%  perf-stat.overall.cycles-between-cache-misses
      0.01 ± 10%      +0.0        0.01 ±  2%  perf-stat.overall.dTLB-store-miss-rate%
      0.96            -1.7%       0.95        perf-stat.overall.ipc
     20265           -24.2%      15352        perf-stat.overall.path-length
 5.919e+10            -1.4%  5.835e+10        perf-stat.ps.branch-instructions
 6.077e+08           -12.1%  5.342e+08        perf-stat.ps.branch-misses
 3.038e+08 ±  3%     -39.6%  1.835e+08 ±  7%  perf-stat.ps.cache-references
  16735247            +3.3%   17287047        perf-stat.ps.context-switches
      1835 ±  2%     -11.8%       1617 ±  2%  perf-stat.ps.cpu-migrations
  8.95e+10            -2.4%  8.739e+10        perf-stat.ps.dTLB-loads
   4543807 ± 10%     +23.5%    5610443 ±  2%  perf-stat.ps.dTLB-store-misses
 5.113e+10            -3.2%  4.951e+10        perf-stat.ps.dTLB-stores
 3.128e+11            -1.5%  3.081e+11        perf-stat.ps.instructions
 9.549e+13            -1.6%  9.399e+13        perf-stat.total.instructions
     46.72           -46.7        0.00        perf-profile.calltrace.cycles-pp.generic_splice_sendpage.direct_splice_actor.splice_direct_to_actor.do_splice_direct.do_sendfile
     46.31           -46.3        0.00        perf-profile.calltrace.cycles-pp.__splice_from_pipe.generic_splice_sendpage.direct_splice_actor.splice_direct_to_actor.do_splice_direct
     44.84           -44.8        0.00        perf-profile.calltrace.cycles-pp.pipe_to_sendpage.__splice_from_pipe.generic_splice_sendpage.direct_splice_actor.splice_direct_to_actor
     44.42           -44.4        0.00        perf-profile.calltrace.cycles-pp.sock_sendpage.pipe_to_sendpage.__splice_from_pipe.generic_splice_sendpage.direct_splice_actor
     44.08           -44.1        0.00        perf-profile.calltrace.cycles-pp.kernel_sendpage.sock_sendpage.pipe_to_sendpage.__splice_from_pipe.generic_splice_sendpage
     43.65           -43.6        0.00        perf-profile.calltrace.cycles-pp.inet_sendpage.kernel_sendpage.sock_sendpage.pipe_to_sendpage.__splice_from_pipe
     43.10           -43.1        0.00        perf-profile.calltrace.cycles-pp.tcp_sendpage.inet_sendpage.kernel_sendpage.sock_sendpage.pipe_to_sendpage
     32.84           -32.8        0.00        perf-profile.calltrace.cycles-pp.tcp_sendpage_locked.tcp_sendpage.inet_sendpage.kernel_sendpage.sock_sendpage
     31.27           -31.3        0.00        perf-profile.calltrace.cycles-pp.tcp_sendmsg_locked.tcp_sendpage_locked.tcp_sendpage.inet_sendpage.kernel_sendpage
     13.69           -13.7        0.00        perf-profile.calltrace.cycles-pp.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendpage_locked.tcp_sendpage.inet_sendpage
     13.38           -13.4        0.00        perf-profile.calltrace.cycles-pp.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendpage_locked.tcp_sendpage
     10.07           -10.1        0.00        perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendpage_locked
      8.24            -8.2        0.00        perf-profile.calltrace.cycles-pp.release_sock.tcp_sendpage.inet_sendpage.kernel_sendpage.sock_sendpage
      7.47            -7.5        0.00        perf-profile.calltrace.cycles-pp.__release_sock.release_sock.tcp_sendpage.inet_sendpage.kernel_sendpage
      5.08            -5.1        0.00        perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.__release_sock.release_sock.tcp_sendpage.inet_sendpage
      5.05            -5.0        0.00        perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock.tcp_sendpage
     46.88            -4.6       42.24        perf-profile.calltrace.cycles-pp.direct_splice_actor.splice_direct_to_actor.do_splice_direct.do_sendfile.__x64_sys_sendfile64
     54.68            -2.7       51.97        perf-profile.calltrace.cycles-pp.splice_direct_to_actor.do_splice_direct.do_sendfile.__x64_sys_sendfile64.do_syscall_64
     55.18            -2.6       52.58        perf-profile.calltrace.cycles-pp.do_splice_direct.do_sendfile.__x64_sys_sendfile64.do_syscall_64.entry_SYSCALL_64_after_hwframe
     62.26            -2.3       59.96        perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.sendfile.sendfile_tcp_stream.main.__libc_start_main
     61.81            -2.3       59.55        perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.sendfile.sendfile_tcp_stream.main
     58.87            -2.2       56.66        perf-profile.calltrace.cycles-pp.__x64_sys_sendfile64.do_syscall_64.entry_SYSCALL_64_after_hwframe.sendfile.sendfile_tcp_stream
     58.05            -2.2       55.86        perf-profile.calltrace.cycles-pp.do_sendfile.__x64_sys_sendfile64.do_syscall_64.entry_SYSCALL_64_after_hwframe.sendfile
     62.97            -2.1       60.82        perf-profile.calltrace.cycles-pp.sendfile.sendfile_tcp_stream.main.__libc_start_main
     64.15            -1.9       62.26        perf-profile.calltrace.cycles-pp.sendfile_tcp_stream.main.__libc_start_main
      8.78            -0.9        7.90        perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg.sock_recvmsg
      7.96            -0.8        7.11        perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg
      9.12            -0.8        8.32        perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked
      7.14            -0.8        6.36        perf-profile.calltrace.cycles-pp.ip_finish_output2.__ip_queue_xmit.__tcp_transmit_skb.tcp_recvmsg_locked.tcp_recvmsg
      6.81            -0.8        6.04        perf-profile.calltrace.cycles-pp.__dev_queue_xmit.ip_finish_output2.__ip_queue_xmit.__tcp_transmit_skb.tcp_recvmsg_locked
      9.56 ±  3%      -0.5        9.04        perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish
      9.97 ±  2%      -0.5        9.45        perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.__netif_receive_skb_one_core
      1.66 ±  4%      -0.5        1.17 ±  3%  perf-profile.calltrace.cycles-pp.mem_cgroup_charge_skmem.__sk_mem_raise_allocated.__sk_mem_schedule.tcp_wmem_schedule.tcp_sendmsg_locked
      1.24 ± 14%      -0.4        0.82        perf-profile.calltrace.cycles-pp.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu
     12.65 ±  2%      -0.4       12.28        perf-profile.calltrace.cycles-pp.ip_local_deliver_finish.__netif_receive_skb_one_core.process_backlog.__napi_poll.net_rx_action
     12.56 ±  2%      -0.4       12.20        perf-profile.calltrace.cycles-pp.ip_protocol_deliver_rcu.ip_local_deliver_finish.__netif_receive_skb_one_core.process_backlog.__napi_poll
     12.19 ±  2%      -0.4       11.83        perf-profile.calltrace.cycles-pp.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.__netif_receive_skb_one_core.process_backlog
     17.14            -0.3       16.86        perf-profile.calltrace.cycles-pp.__do_softirq.do_softirq.__local_bh_enable_ip.__dev_queue_xmit.ip_finish_output2
     17.45            -0.3       17.16        perf-profile.calltrace.cycles-pp.do_softirq.__local_bh_enable_ip.__dev_queue_xmit.ip_finish_output2.__ip_queue_xmit
      2.54            -0.1        2.43        perf-profile.calltrace.cycles-pp.exit_to_user_mode_loop.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
      2.40            -0.1        2.30        perf-profile.calltrace.cycles-pp.schedule.exit_to_user_mode_loop.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64
      2.31            -0.1        2.20        perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_user_mode_loop.exit_to_user_mode_prepare.syscall_exit_to_user_mode
      0.68            -0.0        0.65        perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.sendfile
      0.55 ±  2%      +0.0        0.60 ±  2%  perf-profile.calltrace.cycles-pp.security_file_permission.do_sendfile.__x64_sys_sendfile64.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.55            +0.0        0.60        perf-profile.calltrace.cycles-pp.touch_atime.splice_direct_to_actor.do_splice_direct.do_sendfile.__x64_sys_sendfile64
      0.83            +0.1        0.89        perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule.schedule_timeout.wait_woken
      1.19            +0.1        1.30        perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.sendfile
      1.19            +0.1        1.30        perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.sendfile
      1.20            +0.1        1.32        perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.recv
      1.20            +0.1        1.32        perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.recv
      1.20            +0.1        1.32        perf-profile.calltrace.cycles-pp.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe.recv
      0.73            +0.1        0.86        perf-profile.calltrace.cycles-pp.__cond_resched.kmem_cache_alloc_node.__alloc_skb.tcp_stream_alloc_skb.tcp_sendmsg_locked
      0.64 ±  2%      +0.1        0.79        perf-profile.calltrace.cycles-pp.__schedule.__cond_resched.kmem_cache_alloc_node.__alloc_skb.tcp_stream_alloc_skb
      0.56            +0.2        0.71        perf-profile.calltrace.cycles-pp._copy_from_user.__x64_sys_sendfile64.do_syscall_64.entry_SYSCALL_64_after_hwframe.sendfile
      0.53            +0.2        0.68        perf-profile.calltrace.cycles-pp.skb_release_data.__kfree_skb.tcp_clean_rtx_queue.tcp_ack.tcp_rcv_established
      0.55            +0.2        0.70        perf-profile.calltrace.cycles-pp.__kfree_skb.tcp_clean_rtx_queue.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv
      2.28            +0.2        2.44        perf-profile.calltrace.cycles-pp.recv
      2.20            +0.2        2.38        perf-profile.calltrace.cycles-pp.sendfile
      0.74            +0.2        0.93        perf-profile.calltrace.cycles-pp.__fsnotify_parent.do_sendfile.__x64_sys_sendfile64.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.42 ± 44%      +0.2        0.64        perf-profile.calltrace.cycles-pp.__x64_sys_sendfile64.do_syscall_64.entry_SYSCALL_64_after_hwframe.sendfile
      5.84            +0.3        6.10        perf-profile.calltrace.cycles-pp.__schedule.schedule.schedule_timeout.wait_woken.sk_wait_data
      2.68 ±  2%      +0.3        2.94        perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.__wake_up_common.__wake_up_common_lock.sock_def_readable
      6.02            +0.3        6.30        perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.wait_woken.sk_wait_data.tcp_recvmsg_locked
      6.28            +0.3        6.58        perf-profile.calltrace.cycles-pp.schedule_timeout.wait_woken.sk_wait_data.tcp_recvmsg_locked.tcp_recvmsg
      0.84            +0.3        1.14        perf-profile.calltrace.cycles-pp.do_splice_to.splice_direct_to_actor.do_splice_direct.do_sendfile.__x64_sys_sendfile64
      1.15            +0.3        1.44        perf-profile.calltrace.cycles-pp.copy_page_to_iter_pipe.filemap_read.generic_file_splice_read.splice_direct_to_actor.do_splice_direct
      0.27 ±100%      +0.3        0.57 ±  5%  perf-profile.calltrace.cycles-pp.ip_rcv.__netif_receive_skb_one_core.process_backlog.__napi_poll.net_rx_action
      6.55            +0.3        6.86        perf-profile.calltrace.cycles-pp.wait_woken.sk_wait_data.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg
      1.32            +0.3        1.64        perf-profile.calltrace.cycles-pp.filemap_get_read_batch.filemap_get_pages.filemap_read.generic_file_splice_read.splice_direct_to_actor
      1.58            +0.3        1.90        perf-profile.calltrace.cycles-pp.tcp_clean_rtx_queue.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.__release_sock
      1.23 ± 14%      +0.4        1.58        perf-profile.calltrace.cycles-pp.__check_object_size.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg_locked
      0.28 ±100%      +0.4        0.64        perf-profile.calltrace.cycles-pp.switch_fpu_return.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.63            +0.4        2.00        perf-profile.calltrace.cycles-pp.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg_locked.tcp_recvmsg
      7.69            +0.4        8.08        perf-profile.calltrace.cycles-pp.sk_wait_data.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg.sock_recvmsg
      1.54            +0.4        1.93        perf-profile.calltrace.cycles-pp.filemap_get_pages.filemap_read.generic_file_splice_read.splice_direct_to_actor.do_splice_direct
      1.98            +0.4        2.37        perf-profile.calltrace.cycles-pp.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock
      0.32 ±100%      +0.4        0.72        perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.recv
      0.36 ±100%      +0.4        0.80        perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.recv.process_requests
      0.49 ± 73%      +0.4        0.94        perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.schedule_timeout.wait_woken
      0.34 ±100%      +0.4        0.79 ±  3%  perf-profile.calltrace.cycles-pp.__sk_mem_reduce_allocated.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg.sock_recvmsg
      0.65 ± 51%      +0.5        1.10        perf-profile.calltrace.cycles-pp.check_heap_object.__check_object_size.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter
      0.00            +0.5        0.51        perf-profile.calltrace.cycles-pp.__entry_text_start.sendfile.sendfile_tcp_stream.main.__libc_start_main
      0.00            +0.5        0.51        perf-profile.calltrace.cycles-pp.__fget_light.do_sendfile.__x64_sys_sendfile64.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00            +0.5        0.52        perf-profile.calltrace.cycles-pp.__virt_addr_valid.check_heap_object.__check_object_size.simple_copy_to_iter.__skb_datagram_iter
      0.00            +0.5        0.52        perf-profile.calltrace.cycles-pp.restore_fpregs_from_fpstate.switch_fpu_return.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64
      0.00            +0.5        0.52        perf-profile.calltrace.cycles-pp.tcp_current_mss.tcp_send_mss.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
      0.00            +0.5        0.53        perf-profile.calltrace.cycles-pp.prepare_task_switch.__schedule.schedule.schedule_timeout.wait_woken
      0.00            +0.5        0.55        perf-profile.calltrace.cycles-pp.touch_atime.generic_file_splice_read.splice_direct_to_actor.do_splice_direct.do_sendfile
      0.00            +0.6        0.60        perf-profile.calltrace.cycles-pp.page_cache_pipe_buf_release.splice_to_socket.direct_splice_actor.splice_direct_to_actor.do_splice_direct
     14.75            +0.7       15.40        perf-profile.calltrace.cycles-pp.__dev_queue_xmit.ip_finish_output2.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit
      3.56 ±  2%      +0.7        4.23        perf-profile.calltrace.cycles-pp.ip_finish_output2.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.tcp_sendmsg_locked
      0.00            +0.7        0.68        perf-profile.calltrace.cycles-pp.tcp_send_mss.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.splice_to_socket
      0.00            +0.7        0.73        perf-profile.calltrace.cycles-pp.__schedule.__cond_resched.__release_sock.release_sock.tcp_sendmsg
      0.00            +0.8        0.80        perf-profile.calltrace.cycles-pp.__cond_resched.__release_sock.release_sock.tcp_sendmsg.sock_sendmsg
      1.62 ± 13%      +0.8        2.42        perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.__wake_up_common.__wake_up_common_lock
      3.33 ±  2%      +0.9        4.20        perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_rcv_established
      3.68 ±  2%      +1.0        4.66        perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_rcv_established.tcp_v4_do_rcv
      0.00            +1.0        1.04        perf-profile.calltrace.cycles-pp.skb_append_pagefrags.skb_splice_from_iter.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
      1.21 ± 19%      +1.1        2.28        perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.__wake_up_common
      4.12 ±  2%      +1.1        5.23        perf-profile.calltrace.cycles-pp.tcp_write_xmit.__tcp_push_pending_frames.tcp_rcv_established.tcp_v4_do_rcv.__release_sock
      4.16 ±  2%      +1.1        5.28        perf-profile.calltrace.cycles-pp.__tcp_push_pending_frames.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock
      0.00            +1.2        1.22        perf-profile.calltrace.cycles-pp.kmem_cache_alloc_node.__alloc_skb.tcp_stream_alloc_skb.tcp_sendmsg_locked.tcp_sendmsg
      4.79            +1.2        6.03        perf-profile.calltrace.cycles-pp.filemap_read.generic_file_splice_read.splice_direct_to_actor.do_splice_direct.do_sendfile
      0.00            +1.4        1.40        perf-profile.calltrace.cycles-pp.iov_iter_extract_pages.skb_splice_from_iter.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
      5.53            +1.4        6.94        perf-profile.calltrace.cycles-pp.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg_locked
      5.99            +1.5        7.51        perf-profile.calltrace.cycles-pp._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg_locked.tcp_recvmsg
      5.97            +1.5        7.52        perf-profile.calltrace.cycles-pp.generic_file_splice_read.splice_direct_to_actor.do_splice_direct.do_sendfile.__x64_sys_sendfile64
     27.79            +1.6       29.36        perf-profile.calltrace.cycles-pp.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg.sock_recvmsg.__sys_recvfrom
     28.74            +1.6       30.33        perf-profile.calltrace.cycles-pp.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe.recv.process_requests
     28.43            +1.6       30.04        perf-profile.calltrace.cycles-pp.tcp_recvmsg.inet_recvmsg.sock_recvmsg.__sys_recvfrom.__x64_sys_recvfrom
     28.62            +1.6       30.23        perf-profile.calltrace.cycles-pp.inet_recvmsg.sock_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64
      0.00            +1.6        1.63        perf-profile.calltrace.cycles-pp.__sk_mem_raise_allocated.__sk_mem_schedule.tcp_wmem_schedule.tcp_sendmsg_locked.tcp_sendmsg
     29.69            +1.6       31.33        perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.recv.process_requests.spawn_child
     28.94            +1.6       30.59        perf-profile.calltrace.cycles-pp.sock_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe
     29.94            +1.7       31.60        perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.recv.process_requests.spawn_child.accept_connection
     29.80            +1.7       31.50        perf-profile.calltrace.cycles-pp.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe.recv
      0.00            +1.7        1.73        perf-profile.calltrace.cycles-pp.__sk_mem_schedule.tcp_wmem_schedule.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
      0.00            +1.8        1.80        perf-profile.calltrace.cycles-pp.tcp_wmem_schedule.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.splice_to_socket
      0.00            +2.0        1.98        perf-profile.calltrace.cycles-pp.__alloc_skb.tcp_stream_alloc_skb.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
      8.52            +2.0       10.56        perf-profile.calltrace.cycles-pp.skb_copy_datagram_iter.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg.sock_recvmsg
      8.41            +2.0       10.46        perf-profile.calltrace.cycles-pp.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg
      0.00            +2.7        2.66        perf-profile.calltrace.cycles-pp.tcp_stream_alloc_skb.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.splice_to_socket
      0.00            +3.6        3.62        perf-profile.calltrace.cycles-pp.skb_splice_from_iter.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.splice_to_socket
      0.00            +4.5        4.53        perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.tcp_sendmsg_locked.tcp_sendmsg
      0.00            +5.0        4.98        perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
      0.00            +5.6        5.57        perf-profile.calltrace.cycles-pp.tcp_write_xmit.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.splice_to_socket
     25.85 ± 24%      +6.2       32.04        perf-profile.calltrace.cycles-pp.recv.process_requests.spawn_child.accept_connection.accept_connections
      0.00            +6.3        6.30        perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock.tcp_sendmsg
      0.00            +6.3        6.34        perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.__release_sock.release_sock.tcp_sendmsg.sock_sendmsg
     24.39 ± 26%      +8.2       32.62        perf-profile.calltrace.cycles-pp.accept_connections.main.__libc_start_main
     24.39 ± 26%      +8.2       32.62        perf-profile.calltrace.cycles-pp.accept_connection.accept_connections.main.__libc_start_main
     24.39 ± 26%      +8.2       32.62        perf-profile.calltrace.cycles-pp.spawn_child.accept_connection.accept_connections.main.__libc_start_main
     24.39 ± 26%      +8.2       32.62        perf-profile.calltrace.cycles-pp.process_requests.spawn_child.accept_connection.accept_connections.main
      0.00            +9.2        9.20        perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg
      0.00            +9.3        9.30        perf-profile.calltrace.cycles-pp.__release_sock.release_sock.tcp_sendmsg.sock_sendmsg.splice_to_socket
      0.00            +9.7        9.69        perf-profile.calltrace.cycles-pp.release_sock.tcp_sendmsg.sock_sendmsg.splice_to_socket.direct_splice_actor
      0.00           +11.0       11.00        perf-profile.calltrace.cycles-pp.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
      0.00           +11.2       11.15        perf-profile.calltrace.cycles-pp.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.splice_to_socket
      0.00           +27.1       27.07        perf-profile.calltrace.cycles-pp.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.splice_to_socket.direct_splice_actor
      0.00           +38.2       38.19        perf-profile.calltrace.cycles-pp.tcp_sendmsg.sock_sendmsg.splice_to_socket.direct_splice_actor.splice_direct_to_actor
      0.00           +39.1       39.07        perf-profile.calltrace.cycles-pp.sock_sendmsg.splice_to_socket.direct_splice_actor.splice_direct_to_actor.do_splice_direct
      0.00           +42.0       42.00        perf-profile.calltrace.cycles-pp.splice_to_socket.direct_splice_actor.splice_direct_to_actor.do_splice_direct.do_sendfile
     47.09           -47.1        0.00        perf-profile.children.cycles-pp.generic_splice_sendpage
     46.72           -46.7        0.00        perf-profile.children.cycles-pp.__splice_from_pipe
     45.19           -45.2        0.00        perf-profile.children.cycles-pp.pipe_to_sendpage
     44.78           -44.8        0.00        perf-profile.children.cycles-pp.sock_sendpage
     44.44           -44.4        0.00        perf-profile.children.cycles-pp.kernel_sendpage
     44.05           -44.0        0.00        perf-profile.children.cycles-pp.inet_sendpage
     43.51           -43.5        0.00        perf-profile.children.cycles-pp.tcp_sendpage
     33.39           -33.4        0.00        perf-profile.children.cycles-pp.tcp_sendpage_locked
     47.23            -4.9       42.36        perf-profile.children.cycles-pp.direct_splice_actor
     31.96            -4.4       27.58        perf-profile.children.cycles-pp.tcp_sendmsg_locked
     55.06            -2.9       52.12        perf-profile.children.cycles-pp.splice_direct_to_actor
     55.56            -2.8       52.73        perf-profile.children.cycles-pp.do_splice_direct
     58.50            -2.4       56.10        perf-profile.children.cycles-pp.do_sendfile
     59.70            -2.2       57.49        perf-profile.children.cycles-pp.__x64_sys_sendfile64
     64.44            -2.1       62.38        perf-profile.children.cycles-pp.sendfile_tcp_stream
     65.75            -2.0       63.71        perf-profile.children.cycles-pp.sendfile
     18.06            -1.5       16.55        perf-profile.children.cycles-pp.__tcp_push_pending_frames
      2.79 ±  3%      -1.0        1.83        perf-profile.children.cycles-pp.tcp_wmem_schedule
      1.58            -0.9        0.70        perf-profile.children.cycles-pp.tcp_send_mss
      2.74 ±  3%      -0.7        2.01        perf-profile.children.cycles-pp.__sk_mem_schedule
      1.28            -0.7        0.61        perf-profile.children.cycles-pp.tcp_current_mss
      2.59 ±  3%      -0.7        1.92        perf-profile.children.cycles-pp.__sk_mem_raise_allocated
     94.99            -0.6       94.39        perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
      1.63 ±  3%      -0.6        1.06        perf-profile.children.cycles-pp.ktime_get
     94.31            -0.5       93.76        perf-profile.children.cycles-pp.do_syscall_64
      1.52            -0.5        0.99        perf-profile.children.cycles-pp._raw_spin_lock_bh
     22.52            -0.5       22.01        perf-profile.children.cycles-pp.tcp_write_xmit
     18.12            -0.5       17.63        perf-profile.children.cycles-pp.__local_bh_enable_ip
      1.02            -0.5        0.55        perf-profile.children.cycles-pp.lock_sock_nested
      2.63 ±  4%      -0.4        2.23 ±  3%  perf-profile.children.cycles-pp.mem_cgroup_charge_skmem
     12.71 ±  2%      -0.4       12.32        perf-profile.children.cycles-pp.ip_local_deliver_finish
     12.64 ±  2%      -0.4       12.25        perf-profile.children.cycles-pp.ip_protocol_deliver_rcu
     12.34 ±  2%      -0.4       11.96        perf-profile.children.cycles-pp.tcp_v4_rcv
      1.09            -0.4        0.72        perf-profile.children.cycles-pp.read_tsc
     17.26            -0.3       16.94        perf-profile.children.cycles-pp.__do_softirq
      0.42            -0.3        0.10 ±  3%  perf-profile.children.cycles-pp.tcp_rate_check_app_limited
      0.49            -0.2        0.24        perf-profile.children.cycles-pp.tcp_tso_segs
      0.35 ±  2%      -0.2        0.12 ±  4%  perf-profile.children.cycles-pp.tcp_push
      1.02 ±  4%      -0.2        0.82 ±  6%  perf-profile.children.cycles-pp.select_task_rq
      0.39 ±  8%      -0.2        0.19 ±  5%  perf-profile.children.cycles-pp.ipv4_mtu
      0.93 ±  3%      -0.2        0.73 ±  7%  perf-profile.children.cycles-pp.select_task_rq_fair
      3.80            -0.1        3.66        perf-profile.children.cycles-pp.tcp_ack
      0.47 ±  2%      -0.1        0.35 ±  2%  perf-profile.children.cycles-pp.tcp_established_options
      0.72            -0.1        0.60 ±  2%  perf-profile.children.cycles-pp.try_charge_memcg
      2.60            -0.1        2.48        perf-profile.children.cycles-pp.exit_to_user_mode_loop
      0.19 ±  2%      -0.1        0.08 ±  4%  perf-profile.children.cycles-pp.tcp_downgrade_zcopy_pure
      0.19 ±  2%      -0.1        0.09 ±  5%  perf-profile.children.cycles-pp.tcp_mtu_probe
      0.74            -0.1        0.66        perf-profile.children.cycles-pp.__tcp_send_ack
      0.58 ±  5%      -0.1        0.49 ±  3%  perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
      0.54 ±  6%      -0.1        0.46 ±  3%  perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
      0.51 ±  6%      -0.1        0.43 ±  3%  perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
      0.20            -0.1        0.12        perf-profile.children.cycles-pp._raw_spin_unlock_bh
      0.49 ±  6%      -0.1        0.42 ±  3%  perf-profile.children.cycles-pp.hrtimer_interrupt
      0.46 ±  6%      -0.1        0.39 ±  3%  perf-profile.children.cycles-pp.__hrtimer_run_queues
      0.16 ±  3%      -0.1        0.09 ±  4%  perf-profile.children.cycles-pp.tcp_chrono_stop
      0.51            -0.1        0.45        perf-profile.children.cycles-pp.check_preempt_curr
      0.14 ±  3%      -0.1        0.08        perf-profile.children.cycles-pp.woken_wake_function
      0.38 ±  8%      -0.1        0.32 ±  4%  perf-profile.children.cycles-pp.update_process_times
      0.42            -0.1        0.36 ±  2%  perf-profile.children.cycles-pp.check_preempt_wakeup
      0.19 ±  6%      -0.1        0.14 ±  3%  perf-profile.children.cycles-pp.perf_event_task_tick
      0.38 ±  8%      -0.1        0.33 ±  4%  perf-profile.children.cycles-pp.tick_sched_handle
      0.18 ±  8%      -0.1        0.13 ±  3%  perf-profile.children.cycles-pp.perf_adjust_freq_unthr_context
      0.13 ±  2%      -0.0        0.08 ±  4%  perf-profile.children.cycles-pp.tcp_release_cb
      0.31 ±  8%      -0.0        0.26 ±  3%  perf-profile.children.cycles-pp.scheduler_tick
      0.26 ±  6%      -0.0        0.22 ±  7%  perf-profile.children.cycles-pp.skb_release_head_state
      0.14 ±  3%      -0.0        0.10 ±  5%  perf-profile.children.cycles-pp.iov_iter_bvec
      0.10            -0.0        0.06 ±  6%  perf-profile.children.cycles-pp.tcp_update_pacing_rate
      0.68            -0.0        0.65        perf-profile.children.cycles-pp.kmem_cache_free
      0.10 ±  3%      -0.0        0.07 ±  5%  perf-profile.children.cycles-pp.tcp_rate_skb_delivered
      0.66            -0.0        0.63        perf-profile.children.cycles-pp.sk_reset_timer
      0.30 ±  2%      -0.0        0.28        perf-profile.children.cycles-pp.rcu_all_qs
      0.30 ±  2%      -0.0        0.28        perf-profile.children.cycles-pp.update_rq_clock_task
      0.13            -0.0        0.11        perf-profile.children.cycles-pp.update_irq_load_avg
      0.19            -0.0        0.17 ±  2%  perf-profile.children.cycles-pp.tcp_ack_update_rtt
      0.16 ±  2%      -0.0        0.14        perf-profile.children.cycles-pp.sock_put
      0.07            -0.0        0.06        perf-profile.children.cycles-pp.tcp_v4_send_check
      0.10            +0.0        0.11        perf-profile.children.cycles-pp.rb_insert_color
      0.08            +0.0        0.09        perf-profile.children.cycles-pp.save_fpregs_to_fpstate
      0.11            +0.0        0.12 ±  3%  perf-profile.children.cycles-pp.kfree_skbmem
      0.09            +0.0        0.10 ±  3%  perf-profile.children.cycles-pp.__tcp_ack_snd_check
      0.07 ±  7%      +0.0        0.08        perf-profile.children.cycles-pp.__fdget
      0.06 ±  6%      +0.0        0.07 ±  5%  perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack
      0.44            +0.0        0.46        perf-profile.children.cycles-pp.netperf_sendfile
      0.29 ±  2%      +0.0        0.31 ±  2%  perf-profile.children.cycles-pp.sockfd_lookup_light
      0.13 ±  3%      +0.0        0.15 ±  3%  perf-profile.children.cycles-pp.tcp_v4_fill_cb
      0.13 ±  2%      +0.0        0.14 ±  3%  perf-profile.children.cycles-pp.tcp_skb_entail
      0.08 ±  4%      +0.0        0.10        perf-profile.children.cycles-pp.xas_descend
      0.16 ±  4%      +0.0        0.18 ±  2%  perf-profile.children.cycles-pp.security_socket_recvmsg
      0.06            +0.0        0.08        perf-profile.children.cycles-pp.check_cfs_rq_runtime
      0.39            +0.0        0.41        perf-profile.children.cycles-pp.__list_del_entry_valid
      0.60            +0.0        0.62        perf-profile.children.cycles-pp.put_prev_entity
      0.30            +0.0        0.32        perf-profile.children.cycles-pp.remove_wait_queue
      0.08 ±  5%      +0.0        0.11 ±  3%  perf-profile.children.cycles-pp.tcp_rate_skb_sent
      0.06 ±  9%      +0.0        0.08 ±  4%  perf-profile.children.cycles-pp.tcp_small_queue_check
      0.10 ±  5%      +0.0        0.12 ±  3%  perf-profile.children.cycles-pp.xas_start
      0.10 ±  3%      +0.0        0.13 ±  2%  perf-profile.children.cycles-pp.lock_timer_base
      0.33 ±  2%      +0.0        0.35 ±  2%  perf-profile.children.cycles-pp.refill_stock
      0.06 ±  7%      +0.0        0.09        perf-profile.children.cycles-pp.demo_interval_tick
      0.10            +0.0        0.13 ±  2%  perf-profile.children.cycles-pp.make_vfsgid
      0.14 ±  4%      +0.0        0.17 ±  2%  perf-profile.children.cycles-pp.ktime_get_coarse_real_ts64
      0.10 ±  4%      +0.0        0.13 ±  3%  perf-profile.children.cycles-pp.__get_task_ioprio
      0.10 ±  4%      +0.0        0.13 ±  3%  perf-profile.children.cycles-pp.demo_stream_interval
      0.47 ±  2%      +0.0        0.50        perf-profile.children.cycles-pp.tcp_schedule_loss_probe
      0.16            +0.0        0.19 ±  3%  perf-profile.children.cycles-pp.generic_file_read_iter
      0.10 ±  3%      +0.0        0.13        perf-profile.children.cycles-pp.make_vfsuid
      0.24 ±  2%      +0.0        0.27        perf-profile.children.cycles-pp.is_vmalloc_addr
      0.58 ±  2%      +0.0        0.61 ±  3%  perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
      0.83            +0.0        0.86        perf-profile.children.cycles-pp.__switch_to
      0.14 ±  6%      +0.0        0.18        perf-profile.children.cycles-pp.folio_mark_accessed
      0.13 ±  3%      +0.0        0.17 ±  2%  perf-profile.children.cycles-pp.check_stack_object
      0.66            +0.0        0.70        perf-profile.children.cycles-pp.__update_load_avg_se
      0.08 ±  6%      +0.0        0.12 ±  4%  perf-profile.children.cycles-pp.check_spread
      0.28 ±  2%      +0.0        0.32        perf-profile.children.cycles-pp.copy_page_to_iter
      0.39 ±  2%      +0.0        0.43 ±  4%  perf-profile.children.cycles-pp.recv_tcp_stream
      0.20 ±  6%      +0.0        0.24 ±  6%  perf-profile.children.cycles-pp.tcp_add_backlog
      0.19 ±  2%      +0.0        0.24 ±  2%  perf-profile.children.cycles-pp.__sk_dst_check
      0.18 ±  5%      +0.0        0.23 ±  4%  perf-profile.children.cycles-pp.cpuacct_charge
      0.01 ±223%      +0.0        0.06 ±  9%  perf-profile.children.cycles-pp.page_counter_try_charge
      0.00            +0.1        0.05        perf-profile.children.cycles-pp.get_pipe_info
      0.00            +0.1        0.05        perf-profile.children.cycles-pp.enqueue_timer
      0.17 ±  3%      +0.1        0.22        perf-profile.children.cycles-pp.rw_verify_area
      0.06 ±  6%      +0.1        0.11        perf-profile.children.cycles-pp.tcp_urg
      0.12 ±  3%      +0.1        0.17 ±  2%  perf-profile.children.cycles-pp.rb_next
      0.75 ±  4%      +0.1        0.80        perf-profile.children.cycles-pp.___perf_sw_event
      0.17 ±  2%      +0.1        0.22 ±  2%  perf-profile.children.cycles-pp.fsnotify_perm
      0.22            +0.1        0.28        perf-profile.children.cycles-pp.pick_next_entity
      0.00            +0.1        0.06 ±  8%  perf-profile.children.cycles-pp.tcp_send_delayed_ack
      0.00            +0.1        0.06 ±  6%  perf-profile.children.cycles-pp.iov_iter_pipe
      0.74            +0.1        0.79        perf-profile.children.cycles-pp._raw_spin_lock_irqsave
      0.24            +0.1        0.30        perf-profile.children.cycles-pp.xas_load
      0.16 ±  2%      +0.1        0.22 ±  2%  perf-profile.children.cycles-pp.page_cache_pipe_buf_confirm
      0.25 ±  2%      +0.1        0.31        perf-profile.children.cycles-pp.sanity
      1.03 ±  2%      +0.1        1.09        perf-profile.children.cycles-pp.prepare_task_switch
      0.04 ± 44%      +0.1        0.11 ±  3%  perf-profile.children.cycles-pp.switch_ldt
      0.30            +0.1        0.37        perf-profile.children.cycles-pp.__put_user_8
      0.41            +0.1        0.48        perf-profile.children.cycles-pp.os_xsave
      0.11 ±  4%      +0.1        0.19        perf-profile.children.cycles-pp.tcp_rearm_rto
      1.06            +0.1        1.13        perf-profile.children.cycles-pp.skb_release_data
      0.60            +0.1        0.68        perf-profile.children.cycles-pp.tcp_event_new_data_sent
      0.12 ±  3%      +0.1        0.20        perf-profile.children.cycles-pp.rb_erase
      3.97            +0.1        4.05        perf-profile.children.cycles-pp.syscall_exit_to_user_mode
      0.46            +0.1        0.54        perf-profile.children.cycles-pp.__virt_addr_valid
      1.85            +0.1        1.94        perf-profile.children.cycles-pp.update_load_avg
      0.60 ±  7%      +0.1        0.68 ±  7%  perf-profile.children.cycles-pp.mem_cgroup_uncharge_skmem
      0.28            +0.1        0.37        perf-profile.children.cycles-pp.aa_file_perm
      0.13 ±  3%      +0.1        0.22 ±  4%  perf-profile.children.cycles-pp.security_sock_rcv_skb
      3.71            +0.1        3.80        perf-profile.children.cycles-pp.exit_to_user_mode_prepare
      0.45            +0.1        0.54        perf-profile.children.cycles-pp.iov_iter_advance
      0.43            +0.1        0.52        perf-profile.children.cycles-pp.entry_SYSRETQ_unsafe_stack
      0.22            +0.1        0.33 ±  2%  perf-profile.children.cycles-pp.sk_filter_trim_cap
      0.68            +0.1        0.79        perf-profile.children.cycles-pp.__fget_light
      0.00            +0.1        0.12        perf-profile.children.cycles-pp.inet_sendmsg
      0.50            +0.1        0.63        perf-profile.children.cycles-pp.page_cache_pipe_buf_release
      0.48            +0.1        0.61        perf-profile.children.cycles-pp.current_time
      0.00            +0.1        0.12 ±  4%  perf-profile.children.cycles-pp.apparmor_socket_sock_rcv_skb
      0.00            +0.1        0.13 ±  3%  perf-profile.children.cycles-pp.sock_from_file
      1.08 ±  5%      +0.1        1.21 ±  2%  perf-profile.children.cycles-pp.enqueue_entity
      1.31 ±  5%      +0.1        1.45 ±  2%  perf-profile.children.cycles-pp.__sk_mem_reduce_allocated
      0.60            +0.1        0.74        perf-profile.children.cycles-pp.__entry_text_start
      0.57            +0.2        0.72        perf-profile.children.cycles-pp._copy_from_user
      2.22            +0.2        2.37        perf-profile.children.cycles-pp.switch_mm_irqs_off
      2.78            +0.2        2.94        perf-profile.children.cycles-pp.__alloc_skb
      8.46            +0.2        8.62        perf-profile.children.cycles-pp.schedule
      0.91            +0.2        1.08        perf-profile.children.cycles-pp.apparmor_file_permission
      2.07            +0.2        2.25        perf-profile.children.cycles-pp.kmem_cache_alloc_node
      3.58            +0.2        3.76        perf-profile.children.cycles-pp.skb_splice_from_iter
      0.74            +0.2        0.92        perf-profile.children.cycles-pp.restore_fpregs_from_fpstate
      1.01            +0.2        1.20        perf-profile.children.cycles-pp.check_heap_object
      0.83            +0.2        1.03        perf-profile.children.cycles-pp.set_next_entity
      0.94            +0.2        1.15        perf-profile.children.cycles-pp.switch_fpu_return
      1.17            +0.2        1.42        perf-profile.children.cycles-pp.security_file_permission
      1.14            +0.3        1.40        perf-profile.children.cycles-pp.atime_needs_update
      2.71 ±  2%      +0.3        2.97        perf-profile.children.cycles-pp.ttwu_do_activate
      2.05 ±  3%      +0.3        2.31        perf-profile.children.cycles-pp.enqueue_task_fair
      0.80            +0.3        1.08        perf-profile.children.cycles-pp.skb_append_pagefrags
      1.42            +0.3        1.70        perf-profile.children.cycles-pp.touch_atime
      6.30            +0.3        6.59        perf-profile.children.cycles-pp.schedule_timeout
      1.13            +0.3        1.42        perf-profile.children.cycles-pp.__fsnotify_parent
      2.15 ±  3%      +0.3        2.44        perf-profile.children.cycles-pp.activate_task
      0.86            +0.3        1.17        perf-profile.children.cycles-pp.do_splice_to
      6.57            +0.3        6.88        perf-profile.children.cycles-pp.wait_woken
      1.20            +0.3        1.51        perf-profile.children.cycles-pp.copy_page_to_iter_pipe
      1.16            +0.3        1.48        perf-profile.children.cycles-pp.iov_iter_extract_pages
      2.25            +0.3        2.58        perf-profile.children.cycles-pp.pick_next_task_fair
      1.34            +0.3        1.67        perf-profile.children.cycles-pp.filemap_get_read_batch
      1.59            +0.4        1.95        perf-profile.children.cycles-pp.__check_object_size
      2.66            +0.4        3.02        perf-profile.children.cycles-pp.tcp_stream_alloc_skb
      0.11 ±  6%      +0.4        0.48        perf-profile.children.cycles-pp.aa_sk_perm
      7.75            +0.4        8.13        perf-profile.children.cycles-pp.sk_wait_data
      1.68            +0.4        2.06        perf-profile.children.cycles-pp.simple_copy_to_iter
      1.57            +0.4        1.96        perf-profile.children.cycles-pp.filemap_get_pages
      2.63            +0.4        3.06        perf-profile.children.cycles-pp.__cond_resched
      0.00            +0.5        0.52        perf-profile.children.cycles-pp.security_socket_sendmsg
      9.99            +0.6       10.59        perf-profile.children.cycles-pp.__schedule
     14.76 ±  2%      +0.7       15.47        perf-profile.children.cycles-pp.tcp_rcv_established
     15.16            +0.7       15.89        perf-profile.children.cycles-pp.tcp_v4_do_rcv
      4.90            +1.3        6.16        perf-profile.children.cycles-pp.filemap_read
      5.66            +1.4        7.10        perf-profile.children.cycles-pp.copyout
      8.96            +1.5       10.44        perf-profile.children.cycles-pp.release_sock
      6.04            +1.5        7.57        perf-profile.children.cycles-pp._copy_to_iter
      6.04            +1.6        7.59        perf-profile.children.cycles-pp.generic_file_splice_read
     27.83            +1.6       29.41        perf-profile.children.cycles-pp.tcp_recvmsg_locked
     28.49            +1.6       30.10        perf-profile.children.cycles-pp.tcp_recvmsg
     28.63            +1.6       30.24        perf-profile.children.cycles-pp.inet_recvmsg
     28.96            +1.6       30.60        perf-profile.children.cycles-pp.sock_recvmsg
     29.97            +1.7       31.66        perf-profile.children.cycles-pp.__x64_sys_recvfrom
     29.83            +1.7       31.52        perf-profile.children.cycles-pp.__sys_recvfrom
     30.87            +1.8       32.62        perf-profile.children.cycles-pp.accept_connection
     30.87            +1.8       32.62        perf-profile.children.cycles-pp.spawn_child
     30.87            +1.8       32.62        perf-profile.children.cycles-pp.process_requests
     32.83            +1.9       34.70        perf-profile.children.cycles-pp.recv
      7.78            +1.9        9.68        perf-profile.children.cycles-pp.__release_sock
      8.53            +2.0       10.57        perf-profile.children.cycles-pp.skb_copy_datagram_iter
      8.47            +2.1       10.54        perf-profile.children.cycles-pp.__skb_datagram_iter
     26.30 ± 24%      +6.3       32.62        perf-profile.children.cycles-pp.accept_connections
      0.00           +38.4       38.35        perf-profile.children.cycles-pp.tcp_sendmsg
      0.00           +39.2       39.19        perf-profile.children.cycles-pp.sock_sendmsg
      0.00           +42.3       42.26        perf-profile.children.cycles-pp.splice_to_socket
      2.03            -1.0        1.06        perf-profile.self.cycles-pp.tcp_sendmsg_locked
      1.43            -0.6        0.87        perf-profile.self.cycles-pp.tcp_write_xmit
      1.35            -0.5        0.86        perf-profile.self.cycles-pp._raw_spin_lock_bh
      1.04            -0.4        0.69        perf-profile.self.cycles-pp.read_tsc
      1.65            -0.4        1.30        perf-profile.self.cycles-pp.skb_splice_from_iter
      0.56            -0.3        0.26        perf-profile.self.cycles-pp.tcp_current_mss
      0.36            -0.3        0.09        perf-profile.self.cycles-pp.tcp_rate_check_app_limited
      1.72 ±  7%      -0.3        1.47 ±  5%  perf-profile.self.cycles-pp.mem_cgroup_charge_skmem
      0.46            -0.2        0.22 ±  3%  perf-profile.self.cycles-pp.tcp_tso_segs
      0.32 ±  2%      -0.2        0.11 ±  4%  perf-profile.self.cycles-pp.tcp_push
      0.52 ± 10%      -0.2        0.33        perf-profile.self.cycles-pp.ktime_get
      0.32            -0.2        0.13 ±  3%  perf-profile.self.cycles-pp.tcp_send_mss
      0.36 ±  9%      -0.2        0.18 ±  6%  perf-profile.self.cycles-pp.ipv4_mtu
      0.64 ±  2%      -0.2        0.47 ±  2%  perf-profile.self.cycles-pp.__sk_mem_raise_allocated
      0.52            -0.1        0.37 ±  2%  perf-profile.self.cycles-pp.__local_bh_enable_ip
      0.22            -0.1        0.08 ±  6%  perf-profile.self.cycles-pp.tcp_wmem_schedule
      0.29 ±  3%      -0.1        0.15 ±  2%  perf-profile.self.cycles-pp.select_task_rq_fair
      0.63            -0.1        0.51 ±  2%  perf-profile.self.cycles-pp.try_charge_memcg
      0.31            -0.1        0.20 ±  2%  perf-profile.self.cycles-pp.release_sock
      0.27            -0.1        0.16 ±  3%  perf-profile.self.cycles-pp.__tcp_push_pending_frames
      0.43 ±  2%      -0.1        0.32        perf-profile.self.cycles-pp.tcp_established_options
      0.16 ±  2%      -0.1        0.08 ±  6%  perf-profile.self.cycles-pp.tcp_mtu_probe
      0.48            -0.1        0.39        perf-profile.self.cycles-pp.do_syscall_64
      0.14 ±  2%      -0.1        0.06        perf-profile.self.cycles-pp.tcp_downgrade_zcopy_pure
      0.14 ±  3%      -0.1        0.07 ±  5%  perf-profile.self.cycles-pp.lock_sock_nested
      0.13            -0.1        0.08 ±  6%  perf-profile.self.cycles-pp.woken_wake_function
      0.12 ±  4%      -0.1        0.07        perf-profile.self.cycles-pp.tcp_chrono_stop
      0.13 ±  3%      -0.1        0.08        perf-profile.self.cycles-pp._raw_spin_unlock_bh
      0.14 ±  3%      -0.1        0.08 ±  5%  perf-profile.self.cycles-pp.__sk_mem_schedule
      0.82            -0.0        0.77        perf-profile.self.cycles-pp.tcp_ack
      0.34 ±  2%      -0.0        0.29        perf-profile.self.cycles-pp.__mod_timer
      0.70            -0.0        0.65        perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
      0.14 ±  3%      -0.0        0.10 ±  3%  perf-profile.self.cycles-pp.direct_splice_actor
      0.10 ±  3%      -0.0        0.06        perf-profile.self.cycles-pp.tcp_release_cb
      0.09 ± 14%      -0.0        0.05 ±  8%  perf-profile.self.cycles-pp.perf_adjust_freq_unthr_context
      0.18 ±  3%      -0.0        0.14 ±  3%  perf-profile.self.cycles-pp.check_preempt_wakeup
      0.10 ±  5%      -0.0        0.06 ±  6%  perf-profile.self.cycles-pp.tcp_update_pacing_rate
      0.56            -0.0        0.53        perf-profile.self.cycles-pp.__ip_queue_xmit
      0.11            -0.0        0.08        perf-profile.self.cycles-pp.iov_iter_bvec
      0.50            -0.0        0.47        perf-profile.self.cycles-pp.__do_softirq
      0.66            -0.0        0.63        perf-profile.self.cycles-pp.kmem_cache_free
      0.45            -0.0        0.42        perf-profile.self.cycles-pp.tcp_clean_rtx_queue
      0.13 ±  3%      -0.0        0.10        perf-profile.self.cycles-pp.update_irq_load_avg
      0.49            -0.0        0.46        perf-profile.self.cycles-pp.net_rx_action
      0.34 ±  2%      -0.0        0.32 ±  2%  perf-profile.self.cycles-pp.__x64_sys_sendfile64
      0.09 ±  5%      -0.0        0.07 ±  5%  perf-profile.self.cycles-pp.tcp_rate_skb_delivered
      0.22            -0.0        0.20 ±  2%  perf-profile.self.cycles-pp.rcu_all_qs
      0.14 ±  3%      -0.0        0.13 ±  3%  perf-profile.self.cycles-pp.sock_put
      0.14 ±  3%      -0.0        0.12 ±  3%  perf-profile.self.cycles-pp.syscall_exit_to_user_mode_prepare
      0.20            -0.0        0.19 ±  2%  perf-profile.self.cycles-pp.tcp_event_new_data_sent
      0.10 ±  4%      -0.0        0.09 ±  5%  perf-profile.self.cycles-pp.tcp_rtt_estimator
      0.06            -0.0        0.05        perf-profile.self.cycles-pp.remove_wait_queue
      0.14            +0.0        0.15        perf-profile.self.cycles-pp.inet_recvmsg
      0.10            +0.0        0.11        perf-profile.self.cycles-pp.kfree_skbmem
      0.07            +0.0        0.08        perf-profile.self.cycles-pp.xas_load
      0.06            +0.0        0.07        perf-profile.self.cycles-pp.save_fpregs_to_fpstate
      0.26            +0.0        0.27        perf-profile.self.cycles-pp.schedule_timeout
      0.14 ±  3%      +0.0        0.16 ±  3%  perf-profile.self.cycles-pp.__rdgsbase_inactive
      0.10 ±  4%      +0.0        0.12 ±  4%  perf-profile.self.cycles-pp.tcp_skb_entail
      0.10 ±  3%      +0.0        0.11        perf-profile.self.cycles-pp.__release_sock
      0.12 ±  3%      +0.0        0.13        perf-profile.self.cycles-pp.__list_add_valid
      0.28            +0.0        0.29        perf-profile.self.cycles-pp.refill_stock
      0.08 ±  4%      +0.0        0.10 ±  5%  perf-profile.self.cycles-pp.rb_insert_color
      0.17 ±  2%      +0.0        0.19        perf-profile.self.cycles-pp.set_next_entity
      0.09            +0.0        0.10 ±  4%  perf-profile.self.cycles-pp.tcp_stream_alloc_skb
      0.07 ±  5%      +0.0        0.08 ±  5%  perf-profile.self.cycles-pp.ttwu_do_activate
      0.15 ±  3%      +0.0        0.16 ±  2%  perf-profile.self.cycles-pp.__netif_receive_skb_one_core
      0.07            +0.0        0.09 ±  5%  perf-profile.self.cycles-pp.xas_descend
      0.05 ±  8%      +0.0        0.07        perf-profile.self.cycles-pp.demo_stream_interval
      0.12 ±  3%      +0.0        0.14 ±  4%  perf-profile.self.cycles-pp.tcp_v4_fill_cb
      0.18 ±  2%      +0.0        0.20        perf-profile.self.cycles-pp.do_splice_direct
      0.34            +0.0        0.36        perf-profile.self.cycles-pp.__list_del_entry_valid
      0.17 ±  3%      +0.0        0.19        perf-profile.self.cycles-pp.tcp_schedule_loss_probe
      0.21            +0.0        0.23 ±  2%  perf-profile.self.cycles-pp.schedule
      0.20            +0.0        0.22 ±  2%  perf-profile.self.cycles-pp.switch_fpu_return
      0.05 ±  7%      +0.0        0.07        perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack
      0.27 ±  2%      +0.0        0.29        perf-profile.self.cycles-pp.wait_woken
      0.09 ±  4%      +0.0        0.11 ±  3%  perf-profile.self.cycles-pp.xas_start
      0.28            +0.0        0.30        perf-profile.self.cycles-pp.touch_atime
      0.11 ±  5%      +0.0        0.13 ±  2%  perf-profile.self.cycles-pp.ktime_get_coarse_real_ts64
      0.05 ±  7%      +0.0        0.07 ±  6%  perf-profile.self.cycles-pp.demo_interval_tick
      0.09 ±  6%      +0.0        0.11 ±  4%  perf-profile.self.cycles-pp.sk_filter_trim_cap
      0.08 ±  6%      +0.0        0.10 ±  3%  perf-profile.self.cycles-pp.make_vfsgid
      0.23 ±  2%      +0.0        0.26 ±  2%  perf-profile.self.cycles-pp.recv
      0.08 ±  4%      +0.0        0.10 ±  4%  perf-profile.self.cycles-pp.tcp_rate_skb_sent
      0.07 ±  6%      +0.0        0.10 ±  3%  perf-profile.self.cycles-pp.make_vfsuid
      0.21            +0.0        0.24        perf-profile.self.cycles-pp.is_vmalloc_addr
      0.10 ±  4%      +0.0        0.12 ±  3%  perf-profile.self.cycles-pp.__get_task_ioprio
      0.15            +0.0        0.18 ±  2%  perf-profile.self.cycles-pp.generic_file_read_iter
      0.50            +0.0        0.54        perf-profile.self.cycles-pp.__sys_recvfrom
      0.10            +0.0        0.13 ±  5%  perf-profile.self.cycles-pp.check_stack_object
      0.79            +0.0        0.82        perf-profile.self.cycles-pp.__switch_to
      0.11 ±  4%      +0.0        0.15 ±  3%  perf-profile.self.cycles-pp.simple_copy_to_iter
      0.25 ±  2%      +0.0        0.28        perf-profile.self.cycles-pp.copy_page_to_iter
      0.09 ±  5%      +0.0        0.13 ±  2%  perf-profile.self.cycles-pp.activate_task
      0.03 ± 70%      +0.0        0.07 ±  5%  perf-profile.self.cycles-pp.tcp_small_queue_check
      0.11 ±  3%      +0.0        0.15        perf-profile.self.cycles-pp.folio_mark_accessed
      0.60            +0.0        0.64        perf-profile.self.cycles-pp.__update_load_avg_se
      0.32 ±  3%      +0.0        0.36 ±  4%  perf-profile.self.cycles-pp.recv_tcp_stream
      0.11 ±  3%      +0.0        0.15 ±  2%  perf-profile.self.cycles-pp.__sk_dst_check
      0.06 ±  8%      +0.0        0.10 ±  4%  perf-profile.self.cycles-pp.check_spread
      0.13 ±  2%      +0.0        0.17        perf-profile.self.cycles-pp.rw_verify_area
      0.37            +0.0        0.42        perf-profile.self.cycles-pp.splice_direct_to_actor
      0.12            +0.0        0.16 ±  2%  perf-profile.self.cycles-pp.do_splice_to
      0.18 ±  6%      +0.0        0.22 ±  5%  perf-profile.self.cycles-pp.cpuacct_charge
      0.13            +0.0        0.18 ±  4%  perf-profile.self.cycles-pp.page_cache_pipe_buf_confirm
      0.15 ±  2%      +0.0        0.20 ±  2%  perf-profile.self.cycles-pp.fsnotify_perm
      0.20 ±  2%      +0.0        0.25        perf-profile.self.cycles-pp.pick_next_entity
      0.10 ±  4%      +0.1        0.16 ±  3%  perf-profile.self.cycles-pp.rb_next
      0.05            +0.1        0.10        perf-profile.self.cycles-pp.tcp_urg
      0.00            +0.1        0.05        perf-profile.self.cycles-pp.check_cfs_rq_runtime
      0.00            +0.1        0.05        perf-profile.self.cycles-pp.tcp_send_delayed_ack
      0.00            +0.1        0.05        perf-profile.self.cycles-pp.tcp_grow_window
      0.20            +0.1        0.25        perf-profile.self.cycles-pp.__entry_text_start
      0.67 ±  4%      +0.1        0.72        perf-profile.self.cycles-pp.___perf_sw_event
      0.66            +0.1        0.72        perf-profile.self.cycles-pp.ip_finish_output2
      0.22 ±  2%      +0.1        0.27        perf-profile.self.cycles-pp.filemap_get_pages
      0.20 ±  2%      +0.1        0.25        perf-profile.self.cycles-pp.tcp_data_queue
      0.22 ±  2%      +0.1        0.28 ±  2%  perf-profile.self.cycles-pp.sanity
      0.70            +0.1        0.76        perf-profile.self.cycles-pp._raw_spin_lock_irqsave
      0.10 ±  5%      +0.1        0.16        perf-profile.self.cycles-pp.tcp_rearm_rto
      1.24            +0.1        1.31        perf-profile.self.cycles-pp.__schedule
      0.29            +0.1        0.36 ±  2%  perf-profile.self.cycles-pp.__put_user_8
      0.41            +0.1        0.48        perf-profile.self.cycles-pp.os_xsave
      0.39            +0.1        0.46        perf-profile.self.cycles-pp._copy_to_iter
      0.10 ±  6%      +0.1        0.17 ±  2%  perf-profile.self.cycles-pp.rb_erase
      0.00            +0.1        0.08 ±  6%  perf-profile.self.cycles-pp.inet_sendmsg
      0.23            +0.1        0.31        perf-profile.self.cycles-pp.aa_file_perm
      0.33 ±  2%      +0.1        0.40        perf-profile.self.cycles-pp.check_heap_object
      0.61            +0.1        0.68 ±  2%  perf-profile.self.cycles-pp.apparmor_file_permission
      0.30            +0.1        0.38        perf-profile.self.cycles-pp.security_file_permission
      0.42            +0.1        0.50        perf-profile.self.cycles-pp.__virt_addr_valid
      0.41            +0.1        0.50        perf-profile.self.cycles-pp.iov_iter_advance
      0.42            +0.1        0.51        perf-profile.self.cycles-pp.entry_SYSRETQ_unsafe_stack
      0.50            +0.1        0.59        perf-profile.self.cycles-pp.atime_needs_update
      0.27 ±  6%      +0.1        0.36 ±  2%  perf-profile.self.cycles-pp.enqueue_task_fair
      0.34            +0.1        0.44        perf-profile.self.cycles-pp.current_time
      0.37            +0.1        0.46        perf-profile.self.cycles-pp.sendfile
      0.00            +0.1        0.10        perf-profile.self.cycles-pp.switch_ldt
      0.69            +0.1        0.79        perf-profile.self.cycles-pp.skb_release_data
      0.66            +0.1        0.76        perf-profile.self.cycles-pp.__fget_light
      0.00            +0.1        0.11 ±  6%  perf-profile.self.cycles-pp.apparmor_socket_sock_rcv_skb
      0.41            +0.1        0.52        perf-profile.self.cycles-pp.generic_file_splice_read
      0.48            +0.1        0.59        perf-profile.self.cycles-pp.page_cache_pipe_buf_release
      0.62            +0.1        0.74        perf-profile.self.cycles-pp.__skb_datagram_iter
      0.00            +0.1        0.12 ±  4%  perf-profile.self.cycles-pp.sock_from_file
      0.00            +0.1        0.12 ±  3%  perf-profile.self.cycles-pp.security_socket_sendmsg
      0.47            +0.1        0.61        perf-profile.self.cycles-pp.__check_object_size
      0.44            +0.1        0.58        perf-profile.self.cycles-pp.sendfile_tcp_stream
      0.56            +0.1        0.71        perf-profile.self.cycles-pp._copy_from_user
      2.18            +0.2        2.33        perf-profile.self.cycles-pp.switch_mm_irqs_off
      0.00            +0.2        0.18 ±  2%  perf-profile.self.cycles-pp.tcp_sendmsg
      0.73            +0.2        0.92        perf-profile.self.cycles-pp.restore_fpregs_from_fpstate
      0.71            +0.2        0.92        perf-profile.self.cycles-pp.iov_iter_extract_pages
      0.00            +0.2        0.22 ±  2%  perf-profile.self.cycles-pp.sock_sendmsg
      0.96            +0.2        1.20        perf-profile.self.cycles-pp.copy_page_to_iter_pipe
      0.75            +0.3        1.01        perf-profile.self.cycles-pp.skb_append_pagefrags
      1.10            +0.3        1.38        perf-profile.self.cycles-pp.filemap_get_read_batch
      0.08 ±  4%      +0.3        0.36        perf-profile.self.cycles-pp.aa_sk_perm
      1.10            +0.3        1.38        perf-profile.self.cycles-pp.__fsnotify_parent
      1.33            +0.4        1.70        perf-profile.self.cycles-pp.filemap_read
      5.60            +1.4        7.04        perf-profile.self.cycles-pp.copyout
      0.00            +2.0        2.05        perf-profile.self.cycles-pp.splice_to_socket

Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki